Most Development teams now claim they are doing Agile and are usually doing some variant of Scrum. In this blog post I propose three reasons why teams can struggle with Scrum or any form Agile development. And conveniently they all begin with R.
In the pre-agile days, requirements where usually very detailed and well thought out. For example, when I worked in Ericsson, before we moved to the Agile methodology RUP, the projects were based on a classical Waterfall model that was a very high standard (usually around CMM level 3). The requirements came from System experts who had lots of experience in the field, were very technically skilled and had incredible domain knowledge. Their full time job was to tease things out knowing that they had effectively only one chance to get it right.
In the Agile world, with shorter iteration cycles and much more releases there are chances to make something right when you get it wrong. Functionality can be changed around much more easily. This is good. It is easier for customer collaboration and thus enable more opportunities to tweak, fine tune and get all stakeholders working together sculpting the best solution.
However, because the penalty for getting requirements wrong is not as great as it is in the Waterfall model it can mean that the level of detail and clarity in requirements can start becoming insufficient and before you know development time in the sprint gets wasted trying to figure what is actually required. The key is to get the balance right. Enough detail so that there can be clear agreement across the dev team, customer and any other stakeholders about what is coming next, but not so much detail that people are just getting bogged down in paralysis analysis and forgetting that they are supposed to be shipping regularly.
I suggest one process for helping get the balance between speed and detail for your requirements in this blog post.
Now, the simple maths tells us that a team with long release cycle (for whatever reason) will struggle releasing regularly and will thus struggle with any Agile methodology.
To mitigate this risk happening, it is vital that the team has superb CI with very high code coverage and works with a very strict CI culture. This includes:
By mastering automated testing, the release cycle will be greatly shortened. The dev team should be aiming towards a continuos delivery model where in theory any check could be released if the CI says it is green. Now, this all sounds simple, but it is not. In practise you need skilled developers to write good code and good tests. But the better you are at it, the easier you will be able to release, the easier you will be able to truly agile.
Note: One of the reasons why the micro-services architectural style has become popular is because it offers an opportunity to avoid big bang releases and instead only release what needs to be. That's true. However, most projects are not big enough or complex enough to need this approach. Don't just jump on a bandwagon, justify every major decision.
Of course you do. But this can be often forgotten. Scrum is a mechanism to help manage a project, it is not a mechanism to drive quality
engineering, quality architecture and minimise technical debt. Using Scrum is only one aspect of your process. While your Scrum Master might governs process they don't have to govern architecture or engineering. They may not have a clue about such matters. If all the techies are just doing their own stories trying to complete them before the next show and tell and no-one is looking at the big picture, the project will quickly turn into an unmanageable ball of spaghetti.
This is a classical mistake at the beginning of a project. Because at the beginning there is no Tech debt. There are also no features and no bugs and of course all of this is because there is no code! But, the key point is that there is no tech debt. Everyone starts firing away at stories and there is an illusion of rapid progress but if no-one is looking at the overall big picture, the architecture, the tech debt, the application of patterns or lack of, after a few happy sprints the software entropy will very quickly explode. Meaning that all that super high productivity in the first few weeks of the progress will quickly disappear.
To mitigate this, I think someone technical has to back away from the coding (especially the critical path) and focus on the architecture, the technical leading, enforcing code quality. This will help ensure good design and architecture decisions are made and non-functional targets of the system are not just well defined but are met. It is not always a glamorous job but if it ain't done, complexity will creep in and soon make everything from simple bug fixing, giving estimates, delivering new features all much harder than they should be.
In the Scrum world it is a fallacy to think every technical person must be doing a story and nothing else. It is the equivalent of saying that everyone building a house has to be laying a brick and then wondering why the house is never finished because when the bricks never seem to line up.
Someone has to stand back ensure that people's individual work is all coming together, the tech debt is kept at acceptable levels and any technical risks are quickly mitigated.
Until the next time, take care of yourselves.
Requirements
In the Agile world, with shorter iteration cycles and much more releases there are chances to make something right when you get it wrong. Functionality can be changed around much more easily. This is good. It is easier for customer collaboration and thus enable more opportunities to tweak, fine tune and get all stakeholders working together sculpting the best solution.
However, because the penalty for getting requirements wrong is not as great as it is in the Waterfall model it can mean that the level of detail and clarity in requirements can start becoming insufficient and before you know development time in the sprint gets wasted trying to figure what is actually required. The key is to get the balance right. Enough detail so that there can be clear agreement across the dev team, customer and any other stakeholders about what is coming next, but not so much detail that people are just getting bogged down in paralysis analysis and forgetting that they are supposed to be shipping regularly.
I suggest one process for helping get the balance between speed and detail for your requirements in this blog post.
Releases
In the Agile methodology Scrum you are supposed to release at the end of every sprint. This means instead of doing 1 - 4 releases a year you will be doing closer to 20 if not more. If your release is painful your team will struggle with any form of Agile. For example, say a full regression test, QA, bug fixing, build, deploy etc takes 10 days (including fixing any bugs found during smoke testing) it means that 20 * 10 = 200 man days are going to releases. Whereas in the old world, with 4 release it would just be 4 * 10 = 40 days. In case it's not obvious, that's a little bit regressive.Now, the simple maths tells us that a team with long release cycle (for whatever reason) will struggle releasing regularly and will thus struggle with any Agile methodology.
To mitigate this risk happening, it is vital that the team has superb CI with very high code coverage and works with a very strict CI culture. This includes:
- Ensuring developers are running full regressing tests on feature branches before merging into dev branches to minimise broken builds on dev branches
- Fix any broken build as a priority
- No checking into a broken build
- Tests are written to a high quality - they need to be as maintainable as your source code
- Coding culture where the code is written in a style so it is easily testable (I'll cover this in a separate blog post)
- CI needs to run fast. No point having 10,000 tests, if they take 18 hours to run. Run tests in parallel if you have to. If your project is so massive that it really does take 18 hours to run automated tests, you need to consider some decomposition. For example, a micro-service architecture where components are in smaller and more manageable parts that can be individually released and deployed in a lot less than 18 hours.
By mastering automated testing, the release cycle will be greatly shortened. The dev team should be aiming towards a continuos delivery model where in theory any check could be released if the CI says it is green. Now, this all sounds simple, but it is not. In practise you need skilled developers to write good code and good tests. But the better you are at it, the easier you will be able to release, the easier you will be able to truly agile.Note: One of the reasons why the micro-services architectural style has become popular is because it offers an opportunity to avoid big bang releases and instead only release what needs to be. That's true. However, most projects are not big enough or complex enough to need this approach. Don't just jump on a bandwagon, justify every major decision.
Roles
The most popular agile methodology Scrum only defines 3 Roles:
- Product Owner
- Scrum Master
- Dev Team.
Of course you do. But this can be often forgotten. Scrum is a mechanism to help manage a project, it is not a mechanism to drive quality
engineering, quality architecture and minimise technical debt. Using Scrum is only one aspect of your process. While your Scrum Master might governs process they don't have to govern architecture or engineering. They may not have a clue about such matters. If all the techies are just doing their own stories trying to complete them before the next show and tell and no-one is looking at the big picture, the project will quickly turn into an unmanageable ball of spaghetti.
This is a classical mistake at the beginning of a project. Because at the beginning there is no Tech debt. There are also no features and no bugs and of course all of this is because there is no code! But, the key point is that there is no tech debt. Everyone starts firing away at stories and there is an illusion of rapid progress but if no-one is looking at the overall big picture, the architecture, the tech debt, the application of patterns or lack of, after a few happy sprints the software entropy will very quickly explode. Meaning that all that super high productivity in the first few weeks of the progress will quickly disappear.
To mitigate this, I think someone technical has to back away from the coding (especially the critical path) and focus on the architecture, the technical leading, enforcing code quality. This will help ensure good design and architecture decisions are made and non-functional targets of the system are not just well defined but are met. It is not always a glamorous job but if it ain't done, complexity will creep in and soon make everything from simple bug fixing, giving estimates, delivering new features all much harder than they should be.
In the Scrum world it is a fallacy to think every technical person must be doing a story and nothing else. It is the equivalent of saying that everyone building a house has to be laying a brick and then wondering why the house is never finished because when the bricks never seem to line up.
Someone has to stand back ensure that people's individual work is all coming together, the tech debt is kept at acceptable levels and any technical risks are quickly mitigated.
Until the next time, take care of yourselves.
Hi Alex. This article is spot on. Would like to read a blog on knowledge sharing in teams, effective code review techniques, tooling, etc. I see this as another dysfunctional area of software development
ReplyDeleteThanks Ronan. Effective Code reviews: http://dublintech.blogspot.ie/2012/01/code-reviews-in-21st-century.html
ReplyDelete