So I'm currently working on some code which has been written but needs refactoring to add some additional business requirements - there are very few tests. So I started by wrapping the code that I want to work with a "golden master"/characterisation test below is the pseudo code for this:
[Test, Ignore (Run this once at the begining)]
Serialize result to xml
Save to file
Serialize result to xml
Compare with master & assert correctness
I save the file so that I can see the results side by side - sometimes this is helpful rather than assertion failure in the test runner.
If you've never heard of the "golden master"/characterisation test then it's a testing technique to ensure that as you refactor some legacy code the output is the same. Anyhow, as I started the refactor the code to add the new functionality, I noticed a couple of bugs. Now while you've got the "golden master" I would advise against fixing these bugs until a little later in the refatoring stage, I prefer to highlight and fix these bugs by using tests. This is where you should be going - as I don't tend to like to run comparison tests in a CI environment.
A couple of things happened while running through the refactoring - the "golden master" didn't have enough test combinations in. I therefore had to reconfigure the data in the database to give this additional test combinations. I only did this once I had everything passing with the original "golden master" that I had created. This gave me the confidence that the changes to the data would generate the necessary failing assertion.
Once I'd got to a point where I was happy that "most" of the combinations had been covered. I then started to add the tests which will run in the CI environment. Essentially, as this is a service with a db, I start by creating an in memory database, adding some data and asserting that the service response contains these bits of data. I also use random data (guids) generated each test to add some testing invariance...
Now is when I want to highlight the bugs that I've found before as I now feel confident that the current refactored code is the same as it was before, but now I can add the tests to fix these bugs. Insert the data as required then check that the output is incorrect - this will obviously create a red test which can now be fixed with confidence.
As the tests grow the "golden master" can be removed and deleted from the code base.
I've used the "golden master" a number of times within production code with no tests, if you find yourself in this position, then see if you can wrap the original code with a "golden master" and then start refactoring. I think you'll be surprised how effect they are...