JSCoverage, cobertura and selenium

I spent a few hours this morning trying to add code coverage to our javascript code base. I came across the pretty well written jscoverage tool and liked the look of the results. What I didn't like was the output format, I needed a way to collect the results for use in continuous integration. We use hudson and the cobertura plugin successfully for both c++ and java projects so I took a crack at converting the jscoverage data into something usable in our system.

Here's what I came up with: JSCoverage to Cobertura

I based my solution off of the reporting code embedded in the jscoverage-server generated files. That report function was the main difference between the regular jscoverage output and the jscoverage-server output, by sending this code with the request we can use the stock jscoverage generated files. My scripts are written in ruby but could be easily ported to other languages, the key is really the function that is uploaded and evaluated inside the browser, that returns a hunk of json that is a map of filenames to arrays of line hit counts. I call the collect_coverage function in the teardown of each of my selenium tests and since the @@coverage variable is shared across all of the test cases all of the runs are merged together so I get a good picture of the coverage. Obviously there is no branch coverage but if jscoverage were to get that capability it would be relatively straight forward to add it to the cobertura output.

One problem is that the collection of data is only 'test-driver' initiated, this means its impossible to test 'exit' behavior or tests that exercise more than one page (since the coverage information is kept in the DOM it is lost on reload). One way to handle this would be inverting the control and having a POST occur before the page is closed (which is what jscoverage-server aimed to accomplish) but that requires alot more plumbing work which makes it less flexible, more likely to break. Plus this is not much of an issue when running javascript unit tests since the page is never reloaded.

Using this setup we now have the javascript coverage of our unit and acceptance tests run by selenium converted to the cobertura xml format and tracked in hudson, isn't open source great! Let me know if there are issues with script in the comments.