Thursday, March 05, 2015

Software Quality a Perspective

I've been recently talking to former colleagues about work.  We discuss code and perceived quality in code.  We tend to discuss the merits of good code and what it really looks like.  Each time I come back to the same place in my mind.  Good code is mainly about collective code ownership & having a shared understanding of what you are doing.  So how does one go about helping manifest this?

I think software development is a 80/20 game...  Depending on which you play it governs the impact on this collective code ownership and what some may deem as code quality.

If you work in the following manner 80% code and 20% communication - your code bases will all have a mish mash of styles, poor tests, good tests and really anything in between.  Certain code will be well understood by certain people, and other bits will be understood a lot less by other people.  Some people will open code bases and swoon at it's clarity for them but others will open it up and want to instantly refactor it!

I'm more than convinced that working the other way 20% code and 80% communication gets a better shared code understanding.  Now I don't mean that you all sit around in meetings and discuss what good code looks likes.  I suggest that you pair about 80% of your time, this is the communication piece that is lacking the other way around.  This conversation drives a shared understanding through your code bases.  It also helps ideas flow from mind to mind, while producing code to a certain degree.

Communication doesn't just start or end with pairing, it starts by having the team be a team.  This is something to focus on above and beyond everything else initially.  Once you have this everything else will flow...  More on this in another post.

Thursday, February 19, 2015

Start with what you do now

Well I managed to get out to Lean Agile Manchester last night about Workplace visualisation and catch up with Carl Phillips of ao.  We had an excellent burger in the Northern Quarter before heading over to madlab, therefore delaying our arrival - oops sorry guys!  We spent quite a bit of time chatting about old times & then got down to the problem at hand.

Weirdly the first example was very reminiscent of where Carl & I started our journey at ao nearly four years ago.  Although, some of the key points where slightly different - i.e. number of developers, the location of the constraints in the department.

A rough guide of the scenario:

  • Specialists with hand-offs, bottleneck in test, multi-tasking galore
  • Small changes and incidents getting in the way of more strategic projects
  • 4 devs, 1 QA, 1 BA

We came up with a board similar to this:

We introduced WIP limits around the requirements section and set this at 1 as testing tended to be the bottleneck.  Reflecting this morning - introducing the WIP limits might have been a little premature...

As we where happy with the previous board, Ian challenged us to look at the digital agency scenario.

We started to map out the current value stream for the digital agency.  A rough guide for the scenario:

  • Multitasking on client projects, fixed scope and deadlines, 
  • lots of rework particularly because of UX
  • lots of late changes of requirements
  • 3rd party dependencies and a long support handover for client projects
  • 4 devs, 1 QA, 1 BA

This time we used swim lanes.  It looked similar but not exactly like this:

 These actually added inherent WIP limits because a swim lane could only have one card in within the column.  We introduced different two additional classes of service (we had a "feature" class of service for project work):

  1. A defect which blocked the current ticket and got tracked across the board.
  2. CR's which would track across the board too.

We had a differing opinion on whether the "client support hand over" should be an additional column or a ticket - we ended up going with a column.

What I realised about this second scenario was that we mapped what was currently happening, although we introduced some inherent WIP limits due to board layout, but these where not forced upon the board - i.e. with numbers above the card columns.

Visualisation is good first step on understanding how work flows through a system, but remember to not force concepts on to the current environment.  "Start with what you do now".

Wednesday, February 11, 2015

The Repository Pattern Sucks

A couple of years ago I would have told you that if you are accessing the a database you need a repository pattern. This allows you to hide away your data access code behind a nice interface - then at least you can mock the data access layer and change your data access code if you ever wanted to do that.

So say we are in a web application and want to get a "customer" by a specific reference. This is how we would do it with the repository pattern...

How different my opinion is today. I can't help but feel that the pedal pushed repository pattern (in the C# .Net world) that is on the internet is fundamentally broken. The key thing for me is that everyone seems to want to create the following interface:
 public interface IRepository<TEntity, in TKey> where TEntity : class  
      IEnumerable<TEntity> GetAll();  
      TEntity GetBy(TKey id);  
      void Save(TEntity entity);       
      void Delete(TEntity entity);       

Not bad so far hey? This seems pretty "standard" apart from the fact you've broken single responsibility (SRP). How many responsibilities does the IRepository already have - 4 public methods which do very different things...

The other thing I've found is that things can & generally will get worse from here - then follows:
 public class Repository<TEntity, TKey> : IRepository<TEntity, TKey> where TEntity : class  
   private readonly IUnitOfWork _unitOfWork;  
   public Repository(IUnitOfWork unitOfWork)  
     _unitOfWork = unitOfWork;  
   public IEnumerable<TEntity> GetAll()  
     return _unitOfWork.Session.QueryOver<TEntity>().ToList();  
   public TEntity GetBy(TKey id)  
        return Session.QueryOver<TEntity>().FirstOrDefault(x => x.Id == id);  
   public void Save(TEntity entity)  
   public void Delete(TEntity entity)  

So now all you need to do to get access to something is as follows:

 public class CustomerRepository : Repository<Customer, int>, ICustomerRepository  
   public CustomerRepository(IUnitOfWork unitOfWork) : base(unitOfWork)  

Does this look familiar? It does to me - I've done this and I'm pretty sure its wrong!

Again there are a couple of things that appear wrong:
  1. SRP is broken
  2. What happens if you've got a complicated queryable object and you are using an ORM - you'll end up with lots of SELECT N+1.
  3. We've over complicated a simple task with - 3 interfaces, a base class (coupling), and 2 classes.
  4. If you want to read from one data store and write to another data store how would you do this?
We've introduced so many layers and complexity that it seems that even to do a simple thing it's been massively over complicated. I've forgot to show you the IUnitOfWork which is equally badly suggested and implemented on the internet...

Now for the alternative:
 var customer;  
 using(var transaction = Session.BeginTransaction())  
      customer = Session.QueryOver<Customer>().FirstOrDefault(x => x.Id == customerId) ?? new NullCustomer();  

Wow, is this simple or what? But what about testing? Well I tend to use NHibernate with an InMemoryDatabase so I don't have to mock the data access layer... I don't tend to use mocks but that's another story... The code is simple and clear.

Now if things advanced and I wanted to put it in a class then I'd put it behind something that only reads customer information:
 public class ReadCustomers  
      private ISession _session;  
      public ReadCustomers(ISession session)  
           _session = session;  
      public Customer GetBy(int customerId)  
           var customer;  
           using(var transaction = _session.BeginTransaction())  
                customer = _session.QueryOver<Customer>().FirstOrDefault(x => x.Id == customerId) ?? new NullCustomer();  
           return customer;  

Again with storing information if you really wanted to push it in a class then something like this:
 public class StoreCustomers  
      private ISession _session;  
      public StoreCustomers(ISession session)  
           _session = session;  
      public void Add(Customer customer)  
           using(var transaction = _session.BeginTransaction())  

The classes have clear responsibility & a single responsibility - you can even easily change where you read and write data from and too! Plus the code is much simpler and easier to optimise.

Tuesday, February 10, 2015


No this is not a post about music.  It's about software development and more importantly delivery.

Before I progress we are all agreed that we are mammals, therefore have an inbuilt rhythm - - this is usually a 24/25 hour period in which we all sleep, wake, eat and go to work or do something else!

So let's elaborate further and take this in to the realm of software delivery.  It appears that we (mammals) need rhythm - "Although circadian rhythms are endogenous ("built-in", self-sustained), they are adjusted (en-trained) to the local environment by external cues called zeitgebers, commonly the most important of which is daylight.".

This is where software delivery comes in, I'd like to take SCRUM as the example of setting up this rhythm.  Every 2 weeks we'll deliver software, have some planning and retrospect.  This sets the pace for the delivery - team members know when things are happening and therefore can get in to a rhythm.  The team understands when the start and end is - they have a focus too for a given time period.

Now I'm an advocate of delivering more often (every couple of days), but I believe that until you can do these things well which is walking, you can't sprint (no pun intended).

The other rhythm is that of the daily stand up - this occurs at the same place & time each day.

Other Agile practices seem to promote the same ideas - such as the TDD cycle, pair programming (if you pair well - i.e. swapping the keyboard!!).

It appears that with no rhythm you'll deliver software, but maybe you won't be going as quickly as you can.

Wednesday, February 04, 2015

From "Golden Master" to Unit Tests

So I'm currently working on some code which has been written but needs refactoring to add some additional business requirements - there are very few tests.  So I started by wrapping the code that I want to work with a "golden master"/characterisation test below is the pseudo code for this:

[Test, Ignore (Run this once at the begining)]
void GenerateGoldenMaster()
Call service
Serialize result to xml
Save to file

void CompareGoldenMaster()
Call service
Serialize result to xml
Save file
Compare with master & assert correctness

I save the file so that I can see the results side by side - sometimes this is helpful rather than assertion failure in the test runner.

If you've never heard of the "golden master"/characterisation test then it's a testing technique to ensure that as you refactor some legacy code the output is the same.  Anyhow, as I started the refactor the code to add the new functionality, I noticed a couple of bugs.  Now while you've got the "golden master" I would advise against fixing these bugs until a little later in the refatoring stage, I prefer to highlight and fix these bugs by using tests.  This is where you should be going - as I don't tend to like to run comparison tests in a CI environment.

A couple of things happened while running through the refactoring - the "golden master" didn't have enough test combinations in.  I therefore had to reconfigure the data in the database to give this additional test combinations.  I only did this once I had everything passing with the original "golden master" that I had created.  This gave me the confidence that the changes to the data would generate the necessary failing assertion.

Once I'd got to a point where I was happy that "most" of the combinations had been covered.  I then started to add the tests which will run in the CI environment.  Essentially, as this is a service with a db, I start by creating an in memory database, adding some data and asserting that the service response contains these bits of data.  I also use random data (guids) generated each test to add some testing invariance...

Now is when I want to highlight the bugs that I've found before as I now feel confident that the current refactored code is the same as it was before, but now I can add the tests to fix these bugs.  Insert the data as required then check that the output is incorrect - this will obviously create a red test which can now be fixed with confidence.

As the tests grow the "golden master" can be removed and deleted from the code base.

I've used the "golden master" a number of times within production code with no tests, if you find yourself in this position, then see if you can wrap the original code with a "golden master" and then start refactoring.  I think you'll be surprised how effect they are...

Friday, January 30, 2015

First Impressions Count

You hire someone, you've been through the interview process which can take up to 2 hours, you've invested time and money in getting that person to come and join your company. Then they join and you've not got the induction nailed. I've had this happen a few times to myself. Get to the company - no computer or a borrowed computer - sat alone on a bank of desks with none of my to be team members. Walking out for lunch by yourself to find the shops that supply food, which you can't find because the office is in a location you've not been to before. As an individual you've perceived this place to work as crap or not. I have positive work experiences after this has happened but it's taken time for this initial perception to be reversed.

Yet the solution to this is pretty simple - in my eyes anyhow. It just needs a bit of planning ahead, not much but just an idea of how you are going to get people bedded in to your team. Here is a rough guide for an induction that works:
  1. Get all the equipment they are going to be using ordered (fortunately when I was hiring, this was handled by my HR colleague) - it's got to arrive before and be ready for them to use on the day they get there. Keep pestering until you are happy it's going to on there desk when it arrives.
  2. Get a desk within the team area - ideally in the middle of where the team sits - if there is no room, make room - move yourselve if necessary.
  3. On the day of arrival get there before they do - i.e. if they are going to get there for 8:30 - you've got to be in for 8:15.
  4. Once you've been notified they are here - go get them straight away - they are your priority for the day.
  5. Buy them a coffee - have an informal chat about the weekend and travel in. I usually talk about working hours at this point - as people want to know this.
  6. We had a company induction the morning they would arrive - if possible get this out of the way as soon as possible.
  7. Get them to be dropped off in your team area after the induction.
  8. Go out for lunch as a team (you should be doing this every day so adding another person should be easy!).
  9. Once back from lunch - let them setup there kit which will be nicely boxed on there desk. Let them install whatever they need to get there job done (hopefully the PC is already imaged with key software). If there are problems then take them the people who can sort them out (i.e. infrastructure/desktop support people) free introduction done ;).
  10. Give them access to what they need to do there job (i.e. Git, TeamCity, Octopus).
So that's day 1. The next couple of days are pretty free form but generally I followed something as follows:
  1. Get them working on production code within the first 2 days of joining - I promote pairing so this was easy for them to drop in with a pair.
  2. Get them to release to production within the first week of being at the company - if you can't release this often then start working towards it.
  3. Walk them through the architecture of the solution they are working on.
  4. Walk them through the full architecture of the eco-system they are working with.
  5. Get them to setup a build pipeline within the first month of being at the company.
  6. Setup your 121's with them.
  7. Show them around the department - hopefully this should only take a couple of informal introductions - nothing to over the top required.
That's about it from memory - I know it looks like a lot. But once you've done it once it's a breeze. I've found that your other team members will end up helping with this as well! It really does matter making that first impression - people want to feel wanted and valued!

"in the first few milliseconds of our perceiving something we not only unconsciously comprehend what it is, but decide whether we like it or not."

Friday, August 16, 2013

Kanban board columns

Below is a pretty much "default" Kanban board & the relevant columns.  We still use CFD's to analyse & hopefully improve flow through our system.  So what I've been pondering is whether the below is right for us.

We have 2 main queues that I want to talk about today.  The Dev Done & the Ready for Release.  I'm going to focus on the right side of the board to start with as this is the most valuable side...

What we tend to see over our sprints is a build up towards the end of the sprint.  I'm now pondering the benefit of the Ready for Release column.  Over time stories/work builds up here and is not release.  Part of my limited WIP side comes in to play as well, as soon as I see more than 3/4 cards in play.  I'm probably thinking a little bit about the size of the release and the riskiness of the release too.

So this is starting to make me question:

  • Is this really a good column to have?  
  • Why are we completing stuff in test then not releasing it straight away?  Maybe the transaction cost to live is too much, if this is the case then this is the problem that needs to be highlighted and gone after.  
  • Are the stories not valuable to be released independently and maybe the relative value of them is not important?

I'm also questioning the benefit of any queues in the system - like the one between Dev Done & Test.  It seems as though this is a smell rather than a solution to flowing work through the system.  Queues are just a way to buffer another piece of the system from inconsistent flow of work, most of the work on the board can be controlled by the team.  We have cross functional dedicated teams, therefore they should be able to "dev & test" in a single flow.

Some may say that the start of the system is a queue, but in fact this is simply an input buffer to the system/team.  This helps to prevent variance within the system so that there is always enough work for the team to work on.  Therefore, my opinion is that this is "required".

Has anybody been able to eradicate these columns?

Maybe all I'm thinking about is Single Piece Flow really...