More often than not, life as a software developer involves finding inventive ways to work around various impediments that, one feels, shouldn't have been there in the first place. But hey! that's life. As a developer.

A few days ago I had to debug a problem with an application that interfaces with a third party system, using a web service. The application recently started behaving incorrectly, and the log files only showed an unhelpful generic error from the web service.

If you've ever worked with SOAP web services you'll probably know that a very useful way to see what's going on is to inspect the HTTP traffic, i.e. look at the actual HTTP request made by the client and the response returned by the server, using a tool like tcpmon. That, after all, is the great advantage of human readable protocols like HTTP and SOAP.

Except that, for security reasons, the web service at hand uses HTTPS, so all traffic is encrypted.

Fair enough, I thought, surely we'll have access to a test server that uses plain HTTP for debugging? Er, of course we don't! And who knows how long it could take to get this third party to give us one, if at all possible...

So let's get creative, I though. Maybe there are some proxies out there that let you inspect HTTPS traffic as well. They would have to work as a man in the middle attacker, presenting themselves as a server to the client, and as a client to the server. The client sends a request to the proxy using HTTPS, the proxy decrypts the request so it can be inspected, then re-encrypts it and sends it to the server through a second, separate HTTPS connection.

That turned out to be possible, but quite tricky to set up because, unsurprisingly, the client will not trust the SSL certificate returned by the proxy unless forced to do so. So avoiding all the trial and error I had to go through, here are the steps to make it work:

1. Download Burp Suite, a tool that can act as HTTS proxy
2. Unpack and start the Burp proxy java -jar burpsuite_v1.1.jar; by default it will listen on port 8080
3. In Firefox, install the Cert Viewer Plus extension that allows to export SSL certificates
4. Configure Firefox to use localhost:8080 as SSL proxy
5. Load a HTTPS page, e.g. the same Cert Viewer Plus page
6. Firefox will display a warning dialogue saying that PortSwigger (the self-signed certificate used by Burp) is not trusted. Examine the certificate and export it as PortSwigger.pem
7. Create a certificate store for Java and import the PortSwigger certificate with keytool -import -file PortSwigger.pem -keystore test-keystore; the tool will ask for a password for the new keystore
8. Set the following system properties so that Java will use Burp as its HTTPS proxy and trust its certificate:
   

   -Dhttp.proxyHost=localhost
   -Dhttp.proxyPort=8080
   -Djavax.net.ssl.trustStore=/path/to/test-keystore
   -Djavax.net.ssl.trustStorePassword=secret

9. Add the following code to avoid an exception being thrown because the proxy certificate hostname doesn't match the target hostname:
   

   HostnameVerifier hostnameVerifier = new HostnameVerifier() {
     public boolean verify(String hostname, SSLSession session) {
       System.err.println("WARNING! hostname mismatch: "  + hostname);
       return true;
     }
   };
   HttpsURLConnection.setDefaultHostnameVerifier(hostnameVerifier);

After all this, it's finally possible to inspect web service requests and responses.