First, a disclaimer: I love cURL, but there are times when you simply do not want/need to use third-party libraries. This post has nothing to do with cURL. It has everything to do with making HTTP requests with the native API’s for doing HTTP requests on Windows and OS X.
If you are here, odds are that you’ve just spent the last few hours in a futile attempt to understand either the WinInet or CFNetwork documentation. In fairness, the documentation from Microsoft and Apple is decent enough … But let’s face it, you want a quick and dirty example and there are none.
The following snippets are complete working examples. They are simple HTTP GET requests from start to finish. They DO NOT take into consideration proxy servers or HTTP auth. They DO NOT show you how to do an HTTP post. Once you get the basics though, it’s really not that far of a leap to get the rest.
If anyone would like to dive into those subject in more detail, we certainly can. Let me know, and we can talk.
Note that there’s not a lot of error handling in the code below. Just take a look at the API docs for information about what these functions return, and how to respond to errors.
Windows via WinInet:
#include <Windows.h> #include <WinInet.h> #include <iostream> #include <string> int main(int argc, char *argv[]) { HINTERNET hInternet = InternetOpenW(L"MyUserAgent", INTERNET_OPEN_TYPE_DIRECT, NULL, NULL, 0); if( hInternet==NULL ) { std::cout << "InternetOpenW failed with error code " << GetLastError() << std::endl; } else { HINTERNET hConnect = InternetConnectW(hInternet, L"www.your_server.com", 80, NULL, NULL, INTERNET_SERVICE_HTTP, 0, NULL); if( hConnect==NULL ) { std::cout << "InternetConnectW failed with error code " << GetLastError() << std::endl; } else { const wchar_t* parrAcceptTypes[] = { L"text/*", NULL }; HINTERNET hRequest = HttpOpenRequestW(hConnect, L"GET", L"", NULL, NULL, parrAcceptTypes, 0, 0); if( hRequest==NULL ) { std::cout << "HttpOpenRequestW failed with error code " << GetLastError() << std::endl; } else { BOOL bRequestSent = HttpSendRequestW(hRequest, NULL, 0, NULL, 0); if( !bRequestSent ) { std::cout << "HttpSendRequestW failed with error code " << GetLastError() << std::endl; } else { std::string strResponse; const int nBuffSize = 1024; char buff[nBuffSize]; BOOL bKeepReading = true; DWORD dwBytesRead = -1; while(bKeepReading && dwBytesRead!=0) { bKeepReading = InternetReadFile( hRequest, buff, nBuffSize, &dwBytesRead ); strResponse.append(buff, dwBytesRead); } std::cout << strResponse << std::endl; } InternetCloseHandle(hRequest); } InternetCloseHandle(hConnect); } InternetCloseHandle(hInternet); } return 0; }
OS X via CFNetwork*
#include <iostream> #include <fstream> #include <CoreFoundation/CoreFoundation.h> #include <CFNetwork/CFNetwork.h> #include <CFNetwork/CFHTTPStream.h> int main(int argc, char *argv[]) { CFURLRef cfUrl = CFURLCreateWithString(kCFAllocatorDefault, CFSTR("http://www.foo.com/stuff"), NULL); CFHTTPMessageRef cfHttpReq = CFHTTPMessageCreateRequest(kCFAllocatorDefault, CFSTR("GET"), cfUrl, kCFHTTPVersion1_1); CFReadStreamRef readStream = CFReadStreamCreateForHTTPRequest(kCFAllocatorDefault, cfHttpReq); CFReadStreamOpen(readStream); CFMutableDataRef cfResp = CFDataCreateMutable(kCFAllocatorDefault, 0); CFIndex numBytesRead; do { const int nBuffSize = 1024; UInt8 buff[nBuffSize]; numBytesRead = CFReadStreamRead(readStream, buff, nBuffSize); if( numBytesRead > 0 ) { CFDataAppendBytes(cfResp, buff, numBytesRead); } else if( numBytesRead < 0 ) { CFStreamError error = CFReadStreamGetError(readStream); std::cout << error.error << std::endl; } } while( numBytesRead > 0 ); CFReadStreamClose(readStream); // to write to file, uncomment code below. //std::ofstream oFile; //oFile.open("/Users/cbarnes/Desktop/file.out", std::ios::out|std::ios::binary); //oFile.write( (const char*)CFDataGetBytePtr(cfResp), CFDataGetLength(cfResp)); CFRelease(cfUrl); CFRelease(cfHttpReq); CFRelease(readStream); CFRelease(cfResp); return 0; }
* To keep things simple, this code does not use run loops, polling, or callbacks. It just blocks until the request is complete. If you’d like to see an example of such, let me know.
This code for OSX does not seem to work when the url given is http://the.earth.li/~sgtatham/putty/0.62/x86/putty.zip. I just added a little code to the above as shown below
data = (const char*)CFDataGetBytePtr(cfResp);
fp = fopen(“/var/tmp/Update.zip”, “w”);
if (fp == NULL) {
printf(“ACnnot be opened\n”);
} else {
fwrite(data, length, 1, fp);
fclose(fp);
}
printf (“Download done\n”);
Its always some bytes missing so the zip is always corrupt. Please help. Also do give a example to how to schedule this is runloop and makes is async.
Varun, thanks for the input!
Let me take a look and I’ll post back shortly.
Ok Varun, I’ve updated the code above.
Instead of creating a new CFHttpMessageRef object for the response, I just created a CFMutableDataRef and appended to it. Then, you can write those bytes out to file.
I’m not sure exactly what was is happening with the CFHttpMessageRef. My best guess would be that it has something to do with the NULL terminator char being somewhere in the data and being treated like a string. I’m really don’t know though. I’ll investigate further, but if anyone else has any ideas, please share!
Also, if you are downloading BIG files, you could run into memory problems. For example, you probably wouldn’t want to download a 5gb DVD iso image with this code. In this case, just read the bytes from the HTTP stream and immediately write them out to the file as you go. As opposed to appending them to a CFMutableDataRef.
Concerning async http requests: Since you asked nicely, I’ll do a new post in the next day or two.
Asynchronous version of the OS X code above can be found here:
http://cryptofreek.org/2013/04/15/asynchronous-http-request-using-cfnetwork-framework/
Thanks Cole, that fixed the issue. I think you other async example write it to disc directly! Thats good! I will do the same in this case as well.
hai i have used your code to establish a secure connection for sending a small text data to a server
error part of code :-
while(bKeepWriting && dwBytesWrite!=0)
{
bKeepWriting = InternetWriteFile( hRequest, buff, nBuffSize, &dwBytesWrite );
strResponse.append(buff, dwBytesWrite);
}
especially i dont know about amp , for what we are using that
unfortunately i stuck up with the following errors :-
error C2065: ‘amp’ : undeclared identifier
error C2143: syntax error : missing ‘)’ before ‘;’
fatal error C1903: unable to recover from previous error(s); stopping compilation
please guide me to overcome the error
Well, let’s put together our knowledge of compiler warnings, C/C++ syntax, and HTML encoding…
The problem is that in the HTML world, an ampersand (&) is encoded by using “& a m p ;” (spaces added so that the browser will not escape the char sequence). And for whatever reason, WordPress or the Javascript library showing this code isn’t escaping those characters. So, when you are copy/pasting you are getting the entire HTML encoding for the ampersand. This is not valid C syntax.
Anywhere you see “& a m p ;”, replace it with “&”. I’ve tried to correct it in the code above, but whether or not it displays properly might depend on whatever browser you are using.