The example project and source files are available on Google Code; C++ code for Win32, with a Visual Studio 2008 project file.

OAuth POST Requests

Doing a POST request isn’t much different from doing a GET request, but there are still a few important things that you need to get right and, like all things OAuth, if there is a single character out of place somewhere it won’t work.

First one simple cosmetic change for clarity, I renamed the OAuthParameters type to HTTPParameters to reflect its general nature. I’m using it for holding GET and POST query parameters as well as those used for OAuth.

typedef std::map<wstring, wstring> HTTPParameters;

This name change is reflected throughout the code. Next we add a new URL constant for accessing the status update API on twitter:

const wstring UserStatusUpdateUrl = L"http://twitter.com/statuses/update.xml";

I added a quick interface to prompt the user for their status update after retrieving the user’s timeline:

	wprintf(L"Enter your status update (Enter to skip): ");

	wchar_t buf[500] = {L""};
	_getws_s(buf, SIZEOF(buf));

	wstring input = buf;
	if(input.size() > 0)
	{
		wprintf(L"\r\nUpdating status: %s\r\n", input.c_str());

		HTTPParameters postParams;
		postParams[L"status"] = UrlEncode(input);

		wstring submitResult = OAuthWebRequestSubmit(UserStatusUpdateUrl, L"POST", &postParams, ConsumerKey, ConsumerSecret,
			oauthAccessToken, oauthAccessTokenSecret);

		wprintf(L"\r\nUpdate Result:\r\n%s\r\n", submitResult.c_str());
	}

After checking if the user has entered an update and not an empty string, the first step is to build an HTTPParameters object and assign the user’s text to the 'status' parameter. There are a number of other possible parameters which could also be added to postParams, as documented on the Twitter API statuses/update reference page.

Note that the status text is URL Encoded immediately, before it is put into the parameters object. I originally had made the mistake of leaving this until later, when I build the POST query string to send over HTTP. This is incorrect (as I’ve implemented things), because when we generate the OAuth signature later, we need to sign the POST query values as they are sent to the server. In this case, that means URL Encoded, and since I was only URL encoding the data as I sent it in the POST request, the signed data didn’t match (it wasn’t URL encoded). One alternative would have been to do the URL encoding in multiple places, when I generate the OAuth signature, and when generating the POST form data. Doing it once removes decreases your chances of forgetting or doing things differently in multiple places later.

There are two other changes of note here, one is that we are passing "POST" as the HTTP request method, indicating that we are passing parameters as the request body, and not as part of the URL as in GET requests. The second is I’ve added a new parameter to OAuthWebRequestSubmit, following the request method is an optional pointer to an HTTPParameters list, here we provide postParams which contains the status value.

This second change is also reflected throughout the rest of the example code, where all of the other calls are using the "GET" HTTP request method I am simply passing NULL for the unused postParameters value.

The updated OAuthWebRequestSubmit:

wstring OAuthWebRequestSubmit(
    const wstring& url,
	const wstring& httpMethod,
	const HTTPParameters* postParameters,
    const wstring& consumerKey,
    const wstring& consumerSecret,
    const wstring& oauthToken = L"",
    const wstring& oauthTokenSecret = L"",
    const wstring& pin = L""
    )
{
    wstring query = UrlGetQuery(url);
    HTTPParameters getParameters = ParseQueryString(query);

    HTTPParameters oauthSignedParameters = BuildSignedOAuthParameters(
        getParameters,
        url,
        httpMethod,
		postParameters,
        consumerKey, consumerSecret,
        oauthToken, oauthTokenSecret,
        pin );
	return OAuthWebRequestSignedSubmit(oauthSignedParameters, url, httpMethod, postParameters);
}

The only difference is that we are now passing the new postParameters value to BuildSignedOAuthParameters and OAuthWebRequestSignedSubmit (formerly also named OAuthWebRequestSubmit, which wasn’t a great choice for clarity), as well as passing the httpMethod to OAuthWebRequestSignedSubmit which now takes the HTTP request method as an argument instead of hard coding it to "GET" internally.

BuildSignedOAuthParameters actually does something useful with the new postParameters value:

	// create a parameter list containing both oauth and original parameters
	// this will be used to create the parameter signature
	HTTPParameters allParameters = requestParameters;
	if(Compare(httpMethod, L"POST", false) && postParameters)
	{
		allParameters.insert(postParameters->begin(), postParameters->end());
	}
	allParameters.insert(oauthParameters.begin(), oauthParameters.end());

	// prepare a signature base, a carefully formatted string containing
	// all of the necessary information needed to generate a valid signature
	wstring normalUrl = OAuthNormalizeUrl(url);
	wstring normalizedParameters = OAuthNormalizeRequestParameters(allParameters);
	wstring signatureBase = OAuthConcatenateRequestElements(httpMethod, normalUrl, normalizedParameters);

	// obtain a signature and add it to header requestParameters
	wstring signature = OAuthCreateSignature(signatureBase, consumerSecret, requestTokenSecret);
	oauthParameters[L"oauth_signature"] = signature;

	return oauthParameters;

Where before we were taking the request parameters (parsed from the URL) and adding in the OAuth parameters (that we just generated) and combining them into the same list, we are now adding the postParameters into the mix as well, but only if it’s a "POST" request, and there are any postParameters available to be added. As you can see, this master list is then sent to OAuthNormalizeRequestParameters whose return value is then concatenated and ultimately included in the OAuthCreateSignature process.

Finally, after the signature is generated we pass our two new parameters to OAuthWebRequestSignedSubmit where the actual working of doing the POSTing happens:

wstring OAuthWebRequestSignedSubmit(
	const HTTPParameters& oauthParameters,
	const wstring& url,
	const wstring& httpMethod,
	const HTTPParameters* postParameters
	)
{
	_TRACE("OAuthWebRequestSignedSubmit(%s)", url.c_str());

	wstring oauthHeader = L"Authorization: OAuth ";
	oauthHeader += OAuthBuildHeader(oauthParameters);
	oauthHeader += L"\r\n";

	_TRACE("%s", oauthHeader.c_str());

	wchar_t host[1024*4] = {};
	wchar_t path[1024*4] = {};

	URL_COMPONENTS components = { sizeof(URL_COMPONENTS) };

	components.lpszHostName = host;
	components.dwHostNameLength = SIZEOF(host);

	components.lpszUrlPath = path;
	components.dwUrlPathLength = SIZEOF(path);

	wstring normalUrl = url;

	BOOL crackUrlOk = InternetCrackUrl(url.c_str(), url.size(), 0, &components);
	_ASSERTE(crackUrlOk);

	wstring result;

	// TODO you'd probably want to InternetOpen only once at app initialization
	HINTERNET hINet = InternetOpen(L"tc2/1.0",
		INTERNET_OPEN_TYPE_PRECONFIG,
		NULL,
		NULL,
		0 );
	_ASSERTE( hINet != NULL );
	if ( hINet != NULL )
	{
		// TODO add support for HTTPS requests
		HINTERNET hConnection = InternetConnect(
			hINet,
			host,
			components.nPort,
			NULL,
			NULL,
			INTERNET_SERVICE_HTTP,
			0, 0 );
		_ASSERTE(hConnection != NULL);
		if ( hConnection != NULL)
		{
			HINTERNET hData = HttpOpenRequest( hConnection,
				httpMethod.c_str(),
				path,
				NULL,
				NULL,
				NULL,
				INTERNET_FLAG_KEEP_CONNECTION | INTERNET_FLAG_NO_CACHE_WRITE | INTERNET_FLAG_NO_AUTH | INTERNET_FLAG_RELOAD,
				0 );
			_ASSERTE(hData != NULL);
			if ( hData != NULL )
			{
				BOOL oauthHeaderOk = HttpAddRequestHeaders(hData,
					oauthHeader.c_str(),
					oauthHeader.size(),
					0);
				_ASSERTE(oauthHeaderOk);

				// NOTE POST requests are supported, but the MIME type is hardcoded to application/x-www-form-urlencoded (aka. form data)
				// TODO implement support for posting image, raw or other data types
				string postDataUTF8;
				if(Compare(httpMethod, L"POST", false) && postParameters)
				{
					wstring contentHeader = L"Content-Type: application/x-www-form-urlencoded\r\n";
					BOOL contentHeaderOk = HttpAddRequestHeaders(hData,
						contentHeader.c_str(),
						contentHeader.size(),
						0);
					_ASSERTE(contentHeaderOk);

					postDataUTF8 = WideToUTF8(BuildQueryString(*postParameters));
					_TRACE("POST DATA: %S", postDataUTF8.c_str());
				}

				BOOL sendOk = HttpSendRequest( hData, NULL, 0, (LPVOID)(postDataUTF8.size() > 0 ? postDataUTF8.c_str() : NULL), postDataUTF8.size());
				_ASSERTE(sendOk);

				// TODO dynamically allocate return buffer
				BYTE buffer[1024*32] = {};
				DWORD dwRead = 0;
				while(
					InternetReadFile( hData, buffer, SIZEOF(buffer) - 1, &dwRead ) &&
					dwRead > 0
					)
				{
					buffer[dwRead] = 0;
					result += UTF8ToWide((char*)buffer);
				}

				_TRACE("%s", result.c_str());

				InternetCloseHandle(hData);
			}
			InternetCloseHandle(hConnection);
		}
		InternetCloseHandle(hINet);
	}

	return result;
}

Another minor cosmetic change, I extracted the OAuth header parameter formatting out into its own function OAuthBuildHeader.

Farther down you can see in the call to HttpOpenRequest I am no longer blindly passing "GET", and instead am passing in the new httpMethod parameter value. Currently "GET" and "POST" are the only values likely to work.

And finally, just before calling HttpSendRequest, we again check to see if we’re processing a "POST" request, and if so, we take care of a couple of tasks.

First, we need to add a header indicating the MIME type of the data that we’ll be posting, in this case application/x-www-form-urlencoded which is the data format used for submitting forms on websites.

Next we need to format the POST parameters into the application/x-www-form-urlencoded data format, which is simple. It’s essentially the same as the query part of an URL used in a GET request (var1=blah&var2=foo&var3=john). This is taken care of by the helper function BuildQueryString:

// parameters must already be URL encoded before calling BuildQueryString
wstring BuildQueryString( const HTTPParameters &parameters )
{
	wstring query;

	for(HTTPParameters::const_iterator it = parameters.begin();
		it != parameters.end();
		++it)
	{
		_TRACE("%s = %s", it->first.c_str(), it->second.c_str());

		if(it != parameters.begin())
		{
			query += L"&";
		}

		wstring pair;
		pair += it->first + L"=" + it->second + L"";
		query += pair;
	}
	return query;
}

The resulting query string must then be UTF8 encoded. When we send the query string as part of a GET request, the WinInet APIs convert the entire URL to UTF8 for us, parameters and all. Not so with POST data, which WinInet treats as raw binary and will pass along unchanged.

I forgot to do this the first time, and just sent my wide string data buffer as the POST data. This resulted in an invalid signature error from Twitter since the data was UTF8 encoded as part of the signing process and therefor the signature didn’t match the raw wide string data that I sent with the POST (and which wouldn’t have worked anyway since the twitter API would almost certainly only accept UTF8 encoded POST data).

Last but not least, we pass the UTF8 encoded form data string to HttpSendRequest, or pass NULL if there is no data to send.

Tweeted

And that should be it. Assuming everything has gone right so far, you have just sent a tweet out into the twitterverse. The return value from the call (if it succeeds) should be a copy of the full tweet in XML format. One thing to note while testing is that (according to the Twitter API docs) multiple tweets in a row with exactly the same text will be ignored, so make sure to change it up each time.

Since UrlEncode converts to UTF8 as an intermediate format as part of the encoding process (when we first added the status text to the HTTPParameters object), Unicode status text should work fine. I tested it with Japanese and had no problems, but I had to hard code the string as I wasn’t able to enter Japanese into my console window. A utility I found in order to do the test without saving my source file in Unicode is Richard Ishida’s Unicode Code Converter, which looks super handy for a variety of Unicode related tasks.

The only networking capability provided is via the Windows Communication Foundation, a high level interface, which will provide only single request/response access to HTTP.

So that means no IRC, or any other protocol which doesn’t operate over HTTP.  I’m not sure how email will be implemented, but I imagine over HTTP gateways or simply through APIs not exposed to 3rd party applications.

WP7 also doesn’t have multitasking support, repeating the historical choice of the earlier iPhones.  This is also a bit of a shot to connection oriented protocols like IRC, where disconnecting and reconnecting has a noticeably detrimental impact on the user experience.

From the sound of various MS posts and comments, they do intend to add socket support eventually, but not in the first release.  For the sake of the platform I hope limitations like this will not kill it before MS  gets a chance to release all the features they want.

At this point it looks like my choices are, simply wait for a future revision of the Windows Phone platform, or waste time and effort implementing some hack temporary solution like an IRC client that operates over an HTTP gateway, which has it’s own problems, namely having to run an HTTP gateway.

Windows Phone 7 looks like it has potential, but I’m a bit worried about its chances against iPhone and Android if it’s coming out of the gate with such glaring limitations.

The example project and source files are available on Google Code; C++ code for Win32, with a Visual Studio 2008 project file.  Update: Part 3 builds on the code in this section, however the older code as discussed here is still available under tags/blog-part-2.

Making OAuth Requests

Now lets take a look at what OAuthWebRequest actually does. The steps are essentially the same for any request, however not all of the parameters are used all the time.

wstring OAuthWebRequest(
    const wstring& url,
    const wstring& httpMethod,
    const wstring& consumerKey,
    const wstring& consumerSecret,
    const wstring& oauthToken = L"",
    const wstring& oauthTokenSecret = L"",
    const wstring& pin = L""
    )
{
    wstring query = UrlGetQuery(url);
    OAuthParameters originalParameters = ParseQueryString(query);

    OAuthParameters oauthSignedParameters = BuildSignedOAuthParameters(
        originalParameters,
        url, httpMethod,
        consumerKey, consumerSecret,
        oauthToken, oauthTokenSecret,
        pin );
    return OAuthWebRequestSubmit(oauthSignedParameters, url);
}

The first step is pretty basic, we call UrlGetQuery which simply extracts the “query” part from the provided URL, if any. None of the URLs we are using for this example actually need any parameters, but I stuck some extras in one of them just to prove this code works:

http://twitter.com/oauth/request_token?some_other_parameter=hello&another_one=goodbye#meep

The unused parameters will just be ignored by the API, but since they’re there, they need to be properly included in the OAuth signature process.

ParseQueryString splits the query section of the URL up into invidual key/value pairs and stores them in an OAuthParameters, which is simply a typedef for a map of strings:

typedef std::map<wstring, wstring> OAuthParameters;

We pass these parameters (if any) to BuildSignedOAuthParameters which generates all of the additional information that makes an OAuth request OAuth and not just another HTTP call.

OAuthParameters BuildSignedOAuthParameters(
    const OAuthParameters& requestParameters,
    const wstring& url,
    const wstring& httpMethod,
    const wstring& consumerKey,
    const wstring& consumerSecret,
    const wstring& requestToken = L"",
    const wstring& requestTokenSecret = L"",
    const wstring& pin = L"" )
{
    wstring timestamp = OAuthCreateTimestamp();
    wstring nonce = OAuthCreateNonce();

    // create oauth requestParameters
    OAuthParameters oauthParameters;

    oauthParameters[L"oauth_timestamp"] = timestamp;
    oauthParameters[L"oauth_nonce"] = nonce;
    oauthParameters[L"oauth_version"] = L"1.0";
    oauthParameters[L"oauth_signature_method"] = L"HMAC-SHA1";
    oauthParameters[L"oauth_consumer_key"] = consumerKey;

    // add the request token if found
    if (!requestToken.empty())
    {
        oauthParameters[L"oauth_token"] = requestToken;
    }

    // add the authorization pin if found
    if (!pin.empty())
    {
        oauthParameters[L"oauth_verifier"] = pin;
    }

    // create a parameter list containing both oauth and original parameters
    // this will be used to create the parameter signature
    OAuthParameters allParameters = requestParameters;
    allParameters.insert(oauthParameters.begin(), oauthParameters.end());

    // prepare a signature base, a carefully formatted string containing
    // all of the necessary information needed to generate a valid signature
    wstring normalUrl = OAuthNormalizeUrl(url);
    wstring normalizedParameters = OAuthNormalizeRequestParameters(allParameters);
    wstring signatureBase = OAuthConcatenateRequestElements(httpMethod, normalUrl, normalizedParameters);

    // obtain a signature and add it to header requestParameters
    wstring signature = OAuthCreateSignature(signatureBase, consumerSecret, requestTokenSecret);
    oauthParameters[L"oauth_signature"] = signature;

    return oauthParameters;
}

The first couple of steps are pretty straightforward, OAuthCreateTimestamp simply formats the current time value in seconds as a string, and OAuthCreateNonce creates a nonce value, a random string of characters used only for this request.

wstring OAuthCreateNonce()
{
    wchar_t ALPHANUMERIC[] = L"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
    wstring nonce;

    for(int i = 0; i <= 16; ++i)
    {
        nonce += ALPHANUMERIC[rand() % (SIZEOF(ALPHANUMERIC) - 1)]; // don't count null terminator in array
    }
    return nonce;
}

wstring OAuthCreateTimestamp()
{
    __time64_t utcNow;
    __time64_t ret = _time64(&utcNow);
    _ASSERTE(ret != -1);

    wchar_t buf[100] = {};
    swprintf_s(buf, SIZEOF(buf), L"%I64u", utcNow);

    return buf;
}

Next, BuildSignedOAuthParameters collects the nonce and timestamp value, along with the OAuth version we’re using, the signature method (Twitter only accepts the HMAC-SHA1 signature method), and our Consumer Key. In addition, if requestToken is set (may contain either a Request Token or Access Token) we add that into the list; and if a PIN is provided for authorization, we add that as well.

We currently have two lists of parameters, one that contains anything that was included in the original URL, and one that contains all of the needed OAuth values. We create a new set of parameters called allParameters, which includes both lists.

Now that we have a master list of parameters, we begin constructing a Signature Base String, according to the OAuth specification. Basically this is a single string that combines the URL, the query and OAuth parameters, and the HTTP method (“GET” or “POST”) in a known and re-producable format. The server is going to perform all of the same steps in an attempt to create exactly the same string, and validate our signature. If even a single character is the wrong case, missing, or out of place, then the whole signature will be invalidated, and the request will be rejected by the server.

We pass our master parameter list to OAuthNormalizeRequestParameters which creates a single string containing all of the key/value pairs, in sorted order.

wstring OAuthNormalizeRequestParameters( const OAuthParameters& requestParameters )
{
    list<wstring> sorted;
    for(OAuthParameters::const_iterator it = requestParameters.begin();
        it != requestParameters.end();
        ++it)
    {
        wstring param = it->first + L"=" + it->second;
        sorted.push_back(param);
    }
    sorted.sort();

    wstring params;
    for(list<wstring>::iterator it = sorted.begin(); it != sorted.end(); ++it)
    {
        if(params.size() > 0)
        {
            params += L"&";
        }
        params += *it;
    }

    return params;
}

We also need to normalize the URL, which consists of stripping off the query section, and making sure the port number is included if it’s non-standard (eg. not 80 for HTTP), and NOT included if it IS standard.

wstring OAuthNormalizeUrl( const wstring& url )
{
    wchar_t scheme[1024*4] = {};
    wchar_t host[1024*4] = {};
    wchar_t path[1024*4] = {};

    URL_COMPONENTS components = { sizeof(URL_COMPONENTS) };

    components.lpszScheme = scheme;
    components.dwSchemeLength = SIZEOF(scheme);

    components.lpszHostName = host;
    components.dwHostNameLength = SIZEOF(host);

    components.lpszUrlPath = path;
    components.dwUrlPathLength = SIZEOF(path);

    wstring normalUrl = url;

    BOOL crackUrlOk = InternetCrackUrl(url.c_str(), url.size(), 0, &components);
    _ASSERTE(crackUrlOk);
    if(crackUrlOk)
    {
        wchar_t port[10] = {};

        // The port number must only be included if it is non-standard
        if((Compare(scheme, L"http", false) && components.nPort != 80) ||
            (Compare(scheme, L"https", false) && components.nPort != 443))
        {
            swprintf_s(port, SIZEOF(port), L":%u", components.nPort);
        }

        // InternetCrackUrl includes ? and # elements in the path,
        // which we need to strip off
        wstring pathOnly = path;
        wstring::size_type q = pathOnly.find_first_of(L"#?");
        if(q != wstring::npos)
        {
            pathOnly = pathOnly.substr(0, q);
        }

        normalUrl = wstring(scheme) + L"://" + host + port + pathOnly;
    }
    return normalUrl;
}

OAuthNormalizeUrl splits apart the URL using InternetCrackUrl, and then recombines only the parts we need.

We’ve got all our normalized information ready, so we can stick it all together in a single string using OAuthConcatenateRequestElements, first the HTTP method, then the normalized URL, then the normalized parameters.

wstring OAuthConcatenateRequestElements( const wstring& httpMethod, wstring url, const wstring& parameters )
{
    wstring escapedUrl = UrlEncode(url);
    wstring escapedParameters = UrlEncode(parameters);

    wstring ret = httpMethod + L"&" + escapedUrl + L"&" + escapedParameters;
    return ret;
}

The URL and parameters need to be URL encoded before they are concatenated. OAuth has specific requirements about how this should be done, namely:

1. Alphanumeric characters (letters and digits), as well as – . _ ~ (dash, period, underscore, and tilde), MUST NOT be encoded. They must remain as they are. These are confusingly named the Unreserved Characters.
2. All other characters MUST be encoded.
3. The resulting encoded hexadecimal values must be in UPPERCASE, %1A, not %1a.

// char2hex and urlencode from http://www.zedwood.com/article/111/cpp-urlencode-function
// modified according to http://oauth.net/core/1.0a/#encoding_parameters
//
//5.1.  Parameter Encoding
//
//All parameter names and values are escaped using the [RFC3986]
//percent-encoding (%xx) mechanism. Characters not in the unreserved character set
//MUST be encoded. Characters in the unreserved character set MUST NOT be encoded.
//Hexadecimal characters in encodings MUST be upper case.
//Text names and values MUST be encoded as UTF-8
// octets before percent-encoding them per [RFC3629].
//
//  unreserved = ALPHA, DIGIT, '-', '.', '_', '~'

string char2hex( char dec )
{
    char dig1 = (dec&0xF0)>>4;
    char dig2 = (dec&0x0F);
    if ( 0<= dig1 && dig1<= 9) dig1+=48;    //0,48 in ascii
    if (10<= dig1 && dig1<=15) dig1+=65-10; //A,65 in ascii
    if ( 0<= dig2 && dig2<= 9) dig2+=48;
    if (10<= dig2 && dig2<=15) dig2+=65-10;

    string r;
    r.append( &dig1, 1);
    r.append( &dig2, 1);
    return r;
}

string urlencode(const string &c)
{

    string escaped;
    int max = c.length();
    for(int i=0; i<max; i++)
    {
        if ( (48 <= c[i] && c[i] <= 57) ||//0-9
            (65 <= c[i] && c[i] <= 90) ||//ABC...XYZ
            (97 <= c[i] && c[i] <= 122) || //abc...xyz
            (c[i]=='~' || c[i]=='-' || c[i]=='_' || c[i]=='.')
            )
        {
            escaped.append( &c[i], 1);
        }
        else
        {
            escaped.append("%");
            escaped.append( char2hex(c[i]) );//converts char 255 to string "FF"
        }
    }
    return escaped;
}

wstring UrlEncode( const wstring& url )
{
    return UTF8ToWide(urlencode(WideToUTF8(url)));
}

We’re almost there. We have our signature base, and we need to sign it, and add the signature to our list of OAuth parameters. To do this we call OAuthCreateSignature.

wstring OAuthCreateSignature( const wstring& signatureBase, const wstring& consumerSecret, const wstring& requestTokenSecret )
{
    // URL encode key elements http://oauth.net/core/1.0/#anchor16
    wstring escapedConsumerSecret = UrlEncode(consumerSecret);
    wstring escapedTokenSecret = UrlEncode(requestTokenSecret);

    wstring key = escapedConsumerSecret + L"&" + escapedTokenSecret;
    string keyBytes = WideToUTF8(key);

    string data = WideToUTF8(signatureBase);
    string hash = HMACSHA1(keyBytes, data);
    wstring signature = Base64String(hash);

    // You must encode the URI for safe net travel
    signature = UrlEncode(signature);
    return signature;
}

To create the key for signing, we URL encode the Consumer Secret, and the Access Token Secret, and then concatenate them with an &. Then we pass this key data, and our signature base that we want to sign, to HMACSHA1 which will create a signature in the form of a binary hash value. The hash value is then Base64 encoded and then URL encoded, before being returned. In the interest of keeping this article from being even longer than it already is, I won’t include the Base64 code here as it is nothing out of the ordinary, you can check it out on the Google Code project in Base64Coder.cpp or the original file provided by Microsoft.

HMACSHA1 gave me the most trouble, as I am not particularly well versed in cryptographic subjects, and don’t recall ever using the Crypto API before. Ultimately, my solution was based on this delightful example on MSDN, Example C Program: Creating an HMAC, with a key change (oh, a pun!) to how the key is created, based on a NetBSD support source file of all places, crypto_cryptoapi.c

Presented for you in all it’s terrible glory, HMACSHA1…

string HMACSHA1( const string& keyBytes, const string& data )
{
    // based on http://msdn.microsoft.com/en-us/library/aa382379%28v=VS.85%29.aspx

    string hash;

    //--------------------------------------------------------------------
    // Declare variables.
    //
    // hProv:           Handle to a cryptographic service provider (CSP).
    //                  This example retrieves the default provider for
    //                  the PROV_RSA_FULL provider type.
    // hHash:           Handle to the hash object needed to create a hash.
    // hKey:            Handle to a symmetric key. This example creates a
    //                  key for the RC4 algorithm.
    // hHmacHash:       Handle to an HMAC hash.
    // pbHash:          Pointer to the hash.
    // dwDataLen:       Length, in bytes, of the hash.
    // Data1:           Password string used to create a symmetric key.
    // Data2:           Message string to be hashed.
    // HmacInfo:        Instance of an HMAC_INFO structure that contains
    //                  information about the HMAC hash.
    //
    HCRYPTPROV  hProv       = NULL;
    HCRYPTHASH  hHash       = NULL;
    HCRYPTKEY   hKey        = NULL;
    HCRYPTHASH  hHmacHash   = NULL;
    PBYTE       pbHash      = NULL;
    DWORD       dwDataLen   = 0;
    //BYTE        Data1[]     = {0x70,0x61,0x73,0x73,0x77,0x6F,0x72,0x64};
    //BYTE        Data2[]     = {0x6D,0x65,0x73,0x73,0x61,0x67,0x65};
    HMAC_INFO   HmacInfo;

    //--------------------------------------------------------------------
    // Zero the HMAC_INFO structure and use the SHA1 algorithm for
    // hashing.

    ZeroMemory(&HmacInfo, sizeof(HmacInfo));
    HmacInfo.HashAlgid = CALG_SHA1;

    //--------------------------------------------------------------------
    // Acquire a handle to the default RSA cryptographic service provider.

    if (!CryptAcquireContext(
        &hProv,                   // handle of the CSP
        NULL,                     // key container name
        NULL,                     // CSP name
        PROV_RSA_FULL,            // provider type
        CRYPT_VERIFYCONTEXT))     // no key access is requested
    {
        _TRACE(" Error in AcquireContext 0x%08x \n",
            GetLastError());
        goto ErrorExit;
    }

    //--------------------------------------------------------------------
    // Derive a symmetric key from a hash object by performing the
    // following steps:
    //    1. Call CryptCreateHash to retrieve a handle to a hash object.
    //    2. Call CryptHashData to add a text string (password) to the
    //       hash object.
    //    3. Call CryptDeriveKey to create the symmetric key from the
    //       hashed password derived in step 2.
    // You will use the key later to create an HMAC hash object.

    if (!CryptCreateHash(
        hProv,                    // handle of the CSP
        CALG_SHA1,                // hash algorithm to use
        0,                        // hash key
        0,                        // reserved
        &hHash))                  // address of hash object handle
    {
        _TRACE("Error in CryptCreateHash 0x%08x \n",
            GetLastError());
        goto ErrorExit;
    }

    if (!CryptHashData(
        hHash,                    // handle of the hash object
        (BYTE*)keyBytes.c_str(),                    // password to hash
        keyBytes.size(),            // number of bytes of data to add
        0))                       // flags
    {
        _TRACE("Error in CryptHashData 0x%08x \n",
            GetLastError());
        goto ErrorExit;
    }

    // key creation based on
    // http://mirror.leaseweb.com/NetBSD/NetBSD-release-5-0/src/dist/wpa/src/crypto/crypto_cryptoapi.c
    struct {
        BLOBHEADER hdr;
        DWORD len;
        BYTE key[1024]; // TODO might want to dynamically allocate this, Should Be Fine though
    } key_blob;

    key_blob.hdr.bType = PLAINTEXTKEYBLOB;
    key_blob.hdr.bVersion = CUR_BLOB_VERSION;
    key_blob.hdr.reserved = 0;
    /*
    * Note: RC2 is not really used, but that can be used to
    * import HMAC keys of up to 16 byte long.
    * CRYPT_IPSEC_HMAC_KEY flag for CryptImportKey() is needed to
    * be able to import longer keys (HMAC-SHA1 uses 20-byte key).
    */
    key_blob.hdr.aiKeyAlg = CALG_RC2;
    key_blob.len = keyBytes.size();
    ZeroMemory(key_blob.key, sizeof(key_blob.key));

    _ASSERTE(keyBytes.size() <= SIZEOF(key_blob.key));
    CopyMemory(key_blob.key, keyBytes.c_str(), min(keyBytes.size(), SIZEOF(key_blob.key)));

    if (!CryptImportKey(
        hProv,
        (BYTE *)&key_blob,
        sizeof(key_blob),
        0,
        CRYPT_IPSEC_HMAC_KEY,
        &hKey))
    {
        _TRACE("Error in CryptImportKey 0x%08x \n", GetLastError());
        goto ErrorExit;
    }

    //--------------------------------------------------------------------
    // Create an HMAC by performing the following steps:
    //    1. Call CryptCreateHash to create a hash object and retrieve
    //       a handle to it.
    //    2. Call CryptSetHashParam to set the instance of the HMAC_INFO
    //       structure into the hash object.
    //    3. Call CryptHashData to compute a hash of the message.
    //    4. Call CryptGetHashParam to retrieve the size, in bytes, of
    //       the hash.
    //    5. Call malloc to allocate memory for the hash.
    //    6. Call CryptGetHashParam again to retrieve the HMAC hash.

    if (!CryptCreateHash(
        hProv,                    // handle of the CSP.
        CALG_HMAC,                // HMAC hash algorithm ID
        hKey,                     // key for the hash (see above)
        0,                        // reserved
        &hHmacHash))              // address of the hash handle
    {
        _TRACE("Error in CryptCreateHash 0x%08x \n",
            GetLastError());
        goto ErrorExit;
    }

    if (!CryptSetHashParam(
        hHmacHash,                // handle of the HMAC hash object
        HP_HMAC_INFO,             // setting an HMAC_INFO object
        (BYTE*)&HmacInfo,         // the HMAC_INFO object
        0))                       // reserved
    {
        _TRACE("Error in CryptSetHashParam 0x%08x \n",
            GetLastError());
        goto ErrorExit;
    }

    if (!CryptHashData(
        hHmacHash,                // handle of the HMAC hash object
        (BYTE*)data.c_str(),                    // message to hash
        data.size(),            // number of bytes of data to add
        0))                       // flags
    {
        _TRACE("Error in CryptHashData 0x%08x \n",
            GetLastError());
        goto ErrorExit;
    }

    //--------------------------------------------------------------------
    // Call CryptGetHashParam twice. Call it the first time to retrieve
    // the size, in bytes, of the hash. Allocate memory. Then call
    // CryptGetHashParam again to retrieve the hash value.

    if (!CryptGetHashParam(
        hHmacHash,                // handle of the HMAC hash object
        HP_HASHVAL,               // query on the hash value
        NULL,                     // filled on second call
        &dwDataLen,               // length, in bytes, of the hash
        0))
    {
        _TRACE("Error in CryptGetHashParam 0x%08x \n",
            GetLastError());
        goto ErrorExit;
    }

    pbHash = (BYTE*)malloc(dwDataLen);
    if(NULL == pbHash)
    {
        _TRACE("unable to allocate memory\n");
        goto ErrorExit;
    }

    if (!CryptGetHashParam(
        hHmacHash,                 // handle of the HMAC hash object
        HP_HASHVAL,                // query on the hash value
        pbHash,                    // pointer to the HMAC hash value
        &dwDataLen,                // length, in bytes, of the hash
        0))
    {
        _TRACE("Error in CryptGetHashParam 0x%08x \n", GetLastError());
        goto ErrorExit;
    }

    for(DWORD i = 0 ; i < dwDataLen ; i++)
    {
        hash.push_back((char)pbHash[i]);
    }

    // Free resources.
    // lol goto
ErrorExit:
    if(hHmacHash)
        CryptDestroyHash(hHmacHash);
    if(hKey)
        CryptDestroyKey(hKey);
    if(hHash)
        CryptDestroyHash(hHash);
    if(hProv)
        CryptReleaseContext(hProv, 0);
    if(pbHash)
        free(pbHash);

    return hash;
}

Note in particular the use of CryptImportKey to use the key data we provide, instead of using CryptDeriveKey to create one as is done in the MSDN example.

Phew! The hard stuff is all done. As mentioned earlier, the signed hash is Base64 encoded, then URL encoded, and passed back to BuildSignedOAuthParameters which adds it to the list of OAuth parameters, and returns the final signed list.

All we need to do now is pass the signed list of OAuth parameters, and the original URL to OAuthWebRequestSubmit to actually make the HTTP connection, and get the reply.

wstring OAuthWebRequestSubmit(
                              const OAuthParameters& parameters,
                              const wstring& url
                              )
{
    _TRACE("OAuthWebRequestSubmit(%s)", url.c_str());

    wstring oauthHeader = L"Authorization: OAuth ";

    for(OAuthParameters::const_iterator it = parameters.begin();
        it != parameters.end();
        ++it)
    {
        _TRACE("%s = %s", it->first.c_str(), it->second.c_str());

        if(it != parameters.begin())
        {
            oauthHeader += L",";
        }

        wstring pair;
        pair += it->first + L"=\"" + it->second + L"\"";
        oauthHeader += pair;
    }
    oauthHeader += L"\r\n";

    _TRACE("%s", oauthHeader.c_str());

    wchar_t host[1024*4] = {};
    wchar_t path[1024*4] = {};

    URL_COMPONENTS components = { sizeof(URL_COMPONENTS) };

    components.lpszHostName = host;
    components.dwHostNameLength = SIZEOF(host);

    components.lpszUrlPath = path;
    components.dwUrlPathLength = SIZEOF(path);

    wstring normalUrl = url;

    BOOL crackUrlOk = InternetCrackUrl(url.c_str(), url.size(), 0, &components);
    _ASSERTE(crackUrlOk);

    wstring result;

    // TODO you'd probably want to InternetOpen only once at app initialization
    HINTERNET hINet = InternetOpen(L"tc2/1.0",
        INTERNET_OPEN_TYPE_PRECONFIG,
        NULL,
        NULL,
        0 );
    _ASSERTE( hINet != NULL );
    if ( hINet != NULL )
    {
        // TODO add support for HTTPS requests
        HINTERNET hConnection = InternetConnect(
            hINet,
            host,
            components.nPort,
            NULL,
            NULL,
            INTERNET_SERVICE_HTTP,
            0, 0 );
        _ASSERTE(hConnection != NULL);
        if ( hConnection != NULL)
        {
            // TODO add support for handling POST requests
            HINTERNET hData = HttpOpenRequest( hConnection,
                L"GET",
                path,
                NULL,
                NULL,
                NULL,
                INTERNET_FLAG_KEEP_CONNECTION | INTERNET_FLAG_NO_CACHE_WRITE | INTERNET_FLAG_NO_AUTH | INTERNET_FLAG_RELOAD,
                0 );
            _ASSERTE(hData != NULL);
            if ( hData != NULL )
            {
                BOOL addHeadersOk = HttpAddRequestHeaders(hData,
                    oauthHeader.c_str(),
                    oauthHeader.size(),
                    0);
                _ASSERTE(addHeadersOk);

                BOOL sendOk = HttpSendRequest( hData, NULL, 0, NULL, 0);
                _ASSERTE(sendOk);

                BYTE buffer[1024*32] = {};
                DWORD dwRead = 0;
                while(
                    InternetReadFile( hData, buffer, SIZEOF(buffer) - 1, &dwRead ) &&
                    dwRead > 0
                    )
                {
                    buffer[dwRead] = 0;
                    result += UTF8ToWide((char*)buffer);
                }

                _TRACE("%s", result.c_str());

                InternetCloseHandle(hData);
            }
            InternetCloseHandle(hConnection);
        }
        InternetCloseHandle(hINet);
    }

    return result;
}

Notice that the OAuth parameters are all provided as part of a custom header named “Authorization: “. The query parameters, while they were included in the OAuth signing process, are not sent as part of the OAuth custom header, they are sent as usual in the path of the “GET” request (or in the body of the request if using “POST”).

The result is read into a buffer, and then returned. The result will be truncated at 32KB because I was lazy. You’ll want to allocate it dynamically. The example code also doesn’t deal with any of the possible HTTP responses, or other WinInet error codes.

The End

And that’s all there is to it! Ok, it was a bit longer than I expected. The response you get back from the web request, obviously depends on what URL you requested. You might get back a Request Token, or an Access Token, the user’s feed, or possibly an error code or other result.

The Twitter API Documentation awaits.

Update: Part 3 covers POST requests and updating your twitter status.

There seems to be lackluster support for Twitter in the C++ community.  I haven’t yet seen a single Twitter client for Windows written in C++ that was… acceptable.

Accessing the Twitter API using Basic Authentication is pretty straightforward, however Basic Authentication is going to be turned off on August 16th, 2010 (last I checked), and after that will require all applications to use OAuth.

Having looked into what it would take to implement OAuth, I’m not surprised that few people have jumped at the opportunity to do so.  I had to scrounge around the net for several hours in order to piece together the necessary information and code required to implement bare bones OAuth functionality in C++, it’s fairly involved, and not at all well documented.

Existing Libraries

There are currently only two C++ Twitter libraries listed on Twitter’s library list, QTwitLib, and Twitcurl.  Neither appears to be under active development, they’re both GPL (which I’m not a big fan of), QTwitLib uses Qt (which I’m not a big fan of), and neither support OAuth anyway.

There is only one C++ OAuth library, liboauth, listed on Twitter’s OAuth library list, and it requires OpenSSL or Mozilla’s NSS.  OpenSSL requires Perl and NASM to be installed to build, and while I haven’t looked at NSS much, it doesn’t appear to be sync-and-go either.  At least liboauth is available under an MIT License, so this is something I might return to later.  It might be possible to remove HTTPS support and drop the dependencies without too much headache.

The Solution

So the perfectly rational (and conveniently most interesting) solution, is to try and write up some OAuth code from scratch… or from bits and pieces anyway.  Be warned, this isn’t going to be pretty.

Register Your App

The project should work out of the box, as I’ve included test keys registered to me that will work just fine, however if you’re going to alter the program at all, it would be better to register your own application that way I won’t get all the credit/blame, it’s quick and easy.

Log into Twitter’s website and hit their Register An Application page.  Choose a name for your app (you can change it later), and it can’t include the word “twitter”.  Select “Client” not “Browser”, choose “Read & Write” (if you want to be able to post), and you can leave “Use Twitter for login” unchecked.

Once your application is created, make a note of the Consumer Key and Consumer Secret, these can be found under the Application Detail link from the application edit page.  These values uniquely identify your program to Twitter.

Download The Code

The example project and source files are available on Google Code.

The code is C++ for Win32, with a Visual Studio 2008 project file.

Use The Code

The code I’ve provided is not great and only superficially tested, please do not actually use it for anything, although you’re free to do so at your own risk.  I wrote it mostly in a single 8 hour stretch on a Friday night (yay Dr. Pepper), all the while googling for docs and samples on OAuth, and trying to figure out how to calculate an HMAC-SH1 hash.  That being said, it does work more or less.

Take your Consumer Key and Consumer Secret you got when you registered your app, and plug them into the similarly named variables at the top of tc2.cpp:

wstring ConsumerKey = L"abc";
wstring ConsumerSecret = L"ABC";

At this point you should be able to compile and run the program.  I didn’t originally intend for it to have a user interface at all, so I recommend running it in Debug mode under a debugger, as helpful trace information is displayed in the debug output window.

1. When you run the application, it will attempt to connect to twitter, launch the Twitter authorization page in your web browser, and then prompt you on the console to enter the PIN that will be provided to you on the Twitter website.
2. Go to your browser (it may not come to the foreground automatically) and find the authorization page that may have opened in a new tab.  If your browser didn’t load the authorization page at all, look in the console window for “Launching http://twitter.com….” and copy and paste the entire URL into your browser.
3. On the authorization page, Allow your application access to your Twitter account, after which you will be provided the PIN.
4. Type the pin into the console app and hit Enter.  The app will display the authorization information, and then request your user timeline, dumping it in XML on the console.

Now that you’ve gone through this process once, the program will have saved your authorization to a file called tc2_saved.txt in the current directory (probably the project directory).  The next time you run the program, instead of re-authorizing with Twitter, it will simply re-load access token that was saved last time, and access your user time-line directly.

If you want to re-run the authorization flow, just delete tc2_saved.txt.

Anyone that has access to the contents of the saved authorization file can post as you on Twitter. This also goes for debug traces or other output that includes any access tokens and secrets.

High Level Flow

Stepping through the code will be the best way to figure it out, the app isn’t that big so it won’t take too long, but I’ll try to give a general overview of what’s going on.

OAuthWebRequest is the workhorse of the example application, every request we make to the Twitter API is through this function.

1. We call OAuthWebRequest on http://twitter.com/oauth/request_token, providing only our Consumer Key and Consumer Secret.  The reply from request_token gives us a temporary Request Token consisting of “oauth_token” and “oauth_token_secret”.
2. We launch a web browser to http://twitter.com/oauth/authorize?oauth_token=ABC123 and pass in the value of “oauth_token”.  The user authorizes the app and they are given a PIN number which they enter into our app.
3. We call OAuthWebRequest on http://twitter.com/oauth/access_token providing our temporary Request Token plus the PIN number we got from the user, and we get back our permanent Access Token values, “oauth_token” and “oauth_token_secret”.  We save our Access Token to disk for future use.
4. We can now call OAuthWebRequest whenever we want to access the Twitter API as the authorized user by using the saved Access Token.  In the example, we just request the user’s main timeline at http://twitter.com/statuses/user_timeline.xml and then dump it to the screen.
   

However most of the “OAuth stuff” happens in BuildSignedOAuthParameters which generates all of the additional information that makes an OAuth request what it is, and not just another HTTP call.

I’ll leave things at that for now, and may go over the code in more thorough details in a follow up post, or on the Google Code project wiki.

Update: Continue to Part 2

1) It’s written in Adobe AIR.  Task Manager reveals that twhirl is second only to Outlook in using up my precious, precious RAM.

AIR

I might overlook this if I were running it on my main PC with 4GB (the future is now!), but I run email, IRC, messaging clients, etc. on my laptop which has only 1GB.

Which reminds me of DestroyTwitter which boasts, among other things, of having “an unbelievably small footprint”.  How does memory usage of as low as 25MB sound?  Pretty believable, and a bit high for something that only has to display a few lines of text and some icons.

That's unpossible!

2) It’s out of date, doesn’t support the new ReTweet functionality for example, and doesn’t appear to be under active development, as Seesmic has moved on to Seesmic Desktop.  Recently I’ve been discovering other bugs, that I just didn’t notice before or because the Twitter API is changing in subtle ways that twhirl isn’t liking.

3) And the final nail in the coffin, as far as I know, there are no plans to add OAuth support to twhirl.  Which is going to become a problem on August 16th, when Twitter disables basic authentication and requires all clients to use OAuth.

I think it’s about time the C++ community took a hint from apps like uTorrent, and started developing lightweight native Twitter clients for Windows.

Update: Oops, I had said DestroyTwitter is .NET, it’s actually AIR too.  A few other clients, like Blu and Seesmic for Windows are .NET.

Pocket IRC on Windows Mobile 5

Pocket IRC my IRC client for Pocket PC and Windows Mobile devices.  Version 1.3 has been released removing registration requirements and is now available for free, previously $14.95 USD.

This release doesn’t add a lot of new functionality, but it had significant re-factoring internally a couple of years ago, and not a whole lot of testing since then, so fingers crossed.

The free release is a stepping stone to open sourcing the code.  I haven’t decided yet where or how I’ll be releasing the source, so in the meantime I figured I might as well make it free.  Likely it will be a public read-only SVN repository, maybe on Google Code, and probably under an MIT or similar license.