More often than not, life as a software developer involves finding inventive ways to work around various impediments that, one feels, shouldn't have been there in the first place. But hey! that's life. As a developer.
A few days ago I had to debug a problem with an application that interfaces with a third party system, using a web service. The application recently started behaving incorrectly, and the log files only showed an unhelpful generic error from the web service.
If you've ever worked with SOAP web services you'll probably know that a very useful way to see what's going on is to inspect the HTTP traffic, i.e. look at the actual HTTP request made by the client and the response returned by the server, using a tool like tcpmon. That, after all, is the great advantage of human readable protocols like HTTP and SOAP.
Except that, for security reasons, the web service at hand uses HTTPS, so all traffic is encrypted.
Fair enough, I thought, surely we'll have access to a test server that uses plain HTTP for debugging? Er, of course we don't! And who knows how long it could take to get this third party to give us one, if at all possible...
So let's get creative, I though. Maybe there are some proxies out there that let you inspect HTTPS traffic as well. They would have to work as a man in the middle attacker, presenting themselves as a server to the client, and as a client to the server. The client sends a request to the proxy using HTTPS, the proxy decrypts the request so it can be inspected, then re-encrypts it and sends it to the server through a second, separate HTTPS connection.
That turned out to be possible, but quite tricky to set up because, unsurprisingly, the client will not trust the SSL certificate returned by the proxy unless forced to do so. So avoiding all the trial and error I had to go through, here are the steps to make it work:
After all this, it's finally possible to inspect web service requests and responses.