I recently visited a large and prestigious project. The project had already experiences massive overruns on initial estimates and was trying to grasp the situation by re-estimating the remaining work.
Re-estimating a project as such is not necessarily a bad idea, but in this case, the team were doing it for the fourth time in less than six months. The scope of the project was more or less the same as from the outset, but estimates had now grown to almost four times the initial estimates.
Feeling the need to estimate the same project so often is in my opinion a symptom of something else and “bad” estimates are not the real problem. Of cause it’s always easier to second-guess a project in hindsight, but I believe there is a couple of fundamental learning’s to be taken from the particular project.
First of all the team was working under a very tight (some would say impossible) timeline. To make the deadline they chose to staff the team with 30+ people almost from the start. This lead to a number of problem, including bad communication, bottlenecks and people wasting time because they didn’t know what to do or ended up doing the wrong things. An alternative approach would have been to start small and then scale. A smaller core team could have built a walking skeleton, before adding more people. By a walking skeleton I mean both the organization structure to support the final size, but also the fundamental architecture.
Teams were organized by role and not as self-organizing cross-functional units. This meant that developers were split into two teams located in one room with their own Scrum Master and requirements specialists, testers, etc. were placed in other teams in other rooms with their own team lead (Project Manager). This meant that communication between teams was limited and no individual team was able to produce any real value to the client on their own. Instead I believe the teams should have been kept small and cross-functional, with testers, requirements specialists, developers, architects, etc. represented in each team and they should have been allowed to organize their own work. The teams could then have used the Scrum of Scrum technique to coordinate across the different teams.
As mentioned above there was barriers to cross-team communication build into the project organization. To make matters worse the teams did little or no reflections on how they worked together as a team. Actually that’s an unfair accusation on my part. The teams started out with the best intentions and they did actually do a couple of reflection sessions. The team members had a lot on their mind and they came up with a lot of possible process improvements. Unfortunately they never took the time to prioritize the initiatives and therefore ended up trying to change too many things simultaneously. Instead the teams should have picked 1 to 3 initiatives to try out until the next reflection and made a joint commitment to prioritize the initiatives. To remind people it would have been a good idea to make the initiatives very visible and present on big posters in the project rooms were people worked.
When teams skip a demo it always makes my inner alarm go off – it’s like the CODE RED of Agile software development. In this case the teams found it very difficult to piece all the contributions from the different team together. They hadn’t had time to set-up automated build and deployment and they would have had to spend considerable time setting up a demo environment. Since they were behind the aggressive schedule almost from the start they skipped a few demos to save time. Unfortunately the teams paid the price by not having to face reality (their actually progress) and they never got real feedback from the client on how they were progressing. This goes back to starting small and building a walking skeleton before adding too many people to the project. Had they done a demo after the first sprint they would have been forced to face reality and initiate a dialog with the client on what to do. Telling a client you are not going to make a deadline is never fun, but postponing the inevitable rarely helps and often you also limit the number of possible solutions.
The teams were staffed with some of the best architects and developers I have worked with and everyone worked extremely hard to make the project a success. Unfortunately they got so caught up in trying to execute an unrealistic plan that they instead ended up going slower and slower as complexity grew out of control. Had they instead gone slow at the beginning, building a framework, established efficient communication, setting up environments and automation, working with the client on expectations and inspecting and adapting to the environment they were working in. I believe this would have had a much better change of delivering real value at a fair cost.
As I write this, the project described above is back on track. They have re-organized the teams by reducing the number of people involved to about half the initial size and by making the teams cross-functional and self-organizing. They have also negotiated a new less aggressive timeline, started demoing regularly and invested in automated test, build and deployment.
I hope this story inspires you to think about how you organize teams on large projects and I would love to hear your story if you have similar experiences.