Requirements in an Agile World

On a recent project, Stories with unclear requirements were ending up with developers. It was a pain point. Time was spent trying to clarify missing pieces meaning less time to write great code and meaning  developers ended up under even more pressure to get things done.

Now, there are many reasons why Development teams may struggle to adopt to a well structured agile process (whether it be Scrum, Kanban, XP or whatever).  I think one reason is when requirements start going into Sprints that aren't detailed enough. They lack sufficient specification.  This, of course, then creates further problems, including: bad estimations, drop in engineering quality and the correct functionality just not getting delivered. If this sounds in any way at all familiar, please read on.

In our case, we did have a development process which is best summarised by the Sprint board swim-lanes we were using:

• Ready for development - Story was in sprint but no-one had started it. 
• In Progress - developer working on story 
• Code Review - 2 people usually code reviewing. We were good at this. 
• UX review - UX or Product person would look at output to confirm everyone on same page.
• Done - Story merged, finished testing and ready to be released. 

The unclear requirements problem was happening because the process didn't vet stories sufficiently before they were passed over to Developers. I decided to attempt to fix this by introducing a new process for the team specifically for capturing requirements.

Some goals I had for this process:

• It should be JIRA centric. Developers, analysts, product owners, leads, should be able to get everything they needed from the one tool which in our case was JIRA.  Time to cut down on Excel, Word, random emails, meeting notes on someone's laptop as much as possible and put all the important information in one shared place.  It doesn't necessarily have to be JIRA, could be Asana, Trello or any Agile friendly project management tool that your team are using.
• The stories had to be vetted. Strict criteria had to be met before stories could be handed over to Developers. The vetting needed to be done by key technical people. It was fine someone with a non-technical background bringing forward an idea - they are usually the people to do this as they are closer to the customers. But whatever the idea it had to be technically feasible. And if it needed to be tweaked to make it so, it was better that happened as early as possible. 
• Key stakeholders had to come in at key stages in requirements capturing. For example, UX expertise for UX design. 
• Allow people work in an asynchronous manner. The reality was story evolution required different key people at different stages. The process needed to facilitate this. 

In the beginning...

So, not being a JIRA expert, I first created a JIRA project that I could play around without impacting the existing project. I then set about attempting to create a new workflow for the requirements capturing process.

JIRA statuses

Ok so every workflow involves moving an issue through various statuses.  I had a look at the existing JIRA issue statuses (that came with JIRA out of the box) both for inspiration and because I wanted to use existing industry norms as much as possible. So after having a quick look, I came up with some statuses I thought made sense:

• Idea - this would represent just a brain dump of someone's idea. Could be just a sentence, a paragraph or heck even just a picture! 
• Story Definition - at this state, the expectation would be that the acceptance criteria and the usual story stuff would be there. 
• UX Design - screen shots, wire frames etc would be added by someone with UX expertise which would guide the Developer on how things should look. 
• Technical Review - this step would be when the serious vetting was done. If there was not enough information Technical Review would fail and the issue would be sent back to Story Definition state for further work.  Two other important functions of this step:
1. It provided an opportunity to provide architectural advice if it was needed 
2. It prompted for an initial estimate for the work. Idea was to really capture if this was a small, medium or large piece of work.  The estimate could always be refined.
• Ready for development - this step would indicate that the technical review had passed and the story was ready to go into a sprint and be taken by a Developer.

The observant amongst you will note that the last step in the requirements capturing process is the first in the developer process. This was deliberate to emphasis the separation of flows and the sense of continuity when the issue ended up in a sprint with a Developer.

Workflow 

So next up was to put all those states into some sort of workflow. This is best explained by the JIRA workflow diagram below. 

Requirements workflow

With respect to the above diagram, the following are the key points: 

1. The general direction is: Idea -> Story Definition -> UX design -> Technical Review -> Ready for Development. 
2. It is possible for the issue to go forwards and backwards between several states. For example, it is possible to be in Technical Review and then go back to Story Definition, back to Technical Review and then back to Story Definition. The idea here is not only have some vetting before a story goes to Ready for development, but so that an analyst could gather some technical questions regarding feasibility and get them answered to help them evolve their story definition.  There might be an element of to and fro between key stakeholder and the workflow captured this.   It it worth re-emphasising that the Technical Review and Story Definition stages are usually handled by two different people; a technical expert for the former and a product expert for the latter.  The process must help them work together. 
3. I added an On Hold and a Closed stage. On Hold was for stories that just became less important but not irrelevant. The irrelevant were just Closed. The advantage of the Closed stage as opposed to just deleting them was that you had a record of the work that was done and the reason why you didn't proceed with the original idea. 

New fields

Next up was to add more fields. Some of these came from feedback from the team. This was good. The more people who contributed to the process the more buy in, the more likely the new process was going to work. 

• Business Driver - idea here was to capture why we were doing this JIRA? Was it an internal idea, coming from a customer or what? Important that everyone knew this, Analyst, Developer, Lead etc. 
• Acceptance criteria - a logical place to formally specify the acceptance criteria. 
• Technical review - a logical place to detail the technical review. This would include architectural advise on how the Developer should approach the story. 

Checkpoints

As stated, the plan for this process was to allow stories evolve from a simple idea into something that was formally agreed, that could just be implemented by a Developer. To facilitate this evolution, I made key fields compulsory out at key stages. 

• Acceptance Criteria had to filled out before something could be put in technical review 
• Story points had to be filled out before the story could leave Technical Review. 

To do this in JIRA: 

1. Select to edit the workflow
2. Select the relevant transition and then the view validators option.
3. Select validators tab
4. Select to add a new validator
5. Select "required fields".  Select the fields you want completed before the transition can happen

Could the right stakeholder please stand up?

As stated, a goal of this process was to bring in the right stakeholder at the right time.  More specifically:

• when the JIRA made it to UX Design it would be assigned to the team designer
• when the JIRA made it to Technical Review  - it was assigned to the technical lead. I played this role and then would either do the technical review or assign it to another senior technical person on the team. 
• If the issue didn't pass Technical Review and instead went back to Story Definition it was reassigned to the the issue reporter. 

To enable the auto assign in JIRA, I added a "post function" to specific steps to auto assign the JIRA issue to correct stakeholder.  This feature is available off any transition in the workflow.  Just pick the transition where you want to auto assign. 

Stories, New Features or Improvements?

JIRA, out of the box, gives the option to create many types of issues amongst them: Stories, New Features, Improvements. The problem is that there is an element of subjectivity here.  Implementing something like "Allow the user reset their password" could be a Story to one person, a New Feature to someone else and even an Improvement to the person sitting next to them.  I decided for this process, that something starts off as a "New Feature" or  an "Idea" only. Then, it must go through the new requirements process and when done will be moved to either a Story or Epic. 

This meant, that it made it easier to distinguish whether something was in Dev or still in a requirements capturing phase.  When someone is searching issues in JIRA, when they see a Story or an Epic, they immediately know that it has gone through the Requirements gathering process, it has been vetted  and can then be confident it is really ready for development.  This is really useful for backlog grooming which I think always works better when it is just a scheduling process rather than a debate about requirements that is open to the floor. A backlog grooming session that turns into a deep discussion about what should happen can just mean the entire dev team at a very long talk shop.

Boards 

As stated, we already had a Developer sprint board. This board worked well. We used it on the big screen at daily stand ups. I wanted something similar for the requirements capturing process. I decided against changing the existing dev board, for the following reasons: 

• It would would have way too many swimlanes (min 10) and difficult to even see on our massive widescreen. 
• The dev process and requirements process are really for two different sets of people. 
• The dev board was a scrum board. While our dev team was a mixture of Scrum and Kanban, we could use a scrum board, but there was no way the requirements team could.  At the end of the sprint, everything was in the right most swimlane and the release was ready to go.  It was nice to just have that board focussed on development with the end goal of releasing.  Rather, than cluttering it with requirements that could evolve at a different pace.  

So with that in mind, I created a new board specifically for the requirements gathering process.  With the following swimlanes:

Requirements swim lanes.

To avoid createing sprints for this board (they would be pointless), I made this a Kanban board. 

Filters

I added filters for key people on the Product side of the team. They were the individuals who would be driving most of the requirements.  This was to help them track their own work and show what they were doing to the rest of the team.

Priorities

I toyed with the idea of another field to give the JIRA issue a priority. However, why re-invent the wheel. JIRA already had a priority field which had 5 different levels (in order):

• Blocker
• Critical
• Major
• Minor 
• Trivial

So, thought, let's just use this and if we need a 6th we can add.  Five priority levels should really be enough though. To encourage people to use them, I colour coded the levels and made their colours display on the JIRA Board.

This meant any time there was a discussion about all the stories, the priorities were obvious. Things could be reprioritised easily.  The colour coding can be configured on the Agile board.   I got some great feedback on what the various colours should be from the team!  Naturally, I updated the existing Developer board to use the same scheme.

Meetings

Every process needs meetings.  The requirements capturing process has three types of meetings:

• Key Technical Leads and Product Analysts met once a week.  We would run through the entire requirements board, spending two mins on each issue.  I used this opportunity to make sure nothing was blocked, we were in agreement on priority and everyone was buying into the process.   It is important to emphasis this is not backlog grooming session.  All requirements must go through this process first before they are ever discussed in backlog grooming.   
• Product Analysts met amongst themselves.  This meeting didn't involve me but it gave the Product side a chance to delve more into stories,  exchange ideas and do important work without having to take away time from Technical people. 
• Product analyst and Developer for Story.  Usually, each issue ended up with one Product analyst and one Technical stakeholder who would end up developing the feature.  They could met 1-1 whenever they saw fit. This was to thrash out requirements, tease out edge cases, you name it.  The idea here was to give the developer a sense of ownership of the feature.  They were involved from an early stage and would responsible for delivery of the new feature.  How they met, how often they met was up to them. 

My Role

For this project I was a mixture between a Tech Architect, a Tech Lead and also coded.  To do the Tech Architect and Tech Lead work, I generally avoided stories on the critical path and didn't code as much as some of the other developers on the team.  I initially was going to do all the Technical Reviews.  I decided against this as it just doesn't scale and it would slow things down.  I also wanted to get developers involved early and give them a sense of ownership.  The team had strong developers who wanted full stack development.  The best way to achieve that is to give them a feature to own and then just provide the architectural advice when needed.  The architectural advice would be on the lines of:

• Discussion and agreement of changes to key interfaces, ReST endpoints or Database schema
• Patterns
• Best practises
• Technical risk assessment and mitigation

When the issues made it to Technical Review, they were auto assigned to me in JIRA and then I would delegate them out as much as possible and then just zoom in and out as appropriate.

Ah more advantages

The other advantage of this process was that I could see what might be coming up in the next 6 months.  Product Analysts would continuously add ideas for things that might or might not evolve into an actual requirement depending on customer needs etc.  This meant, that I could be confident that the existing architecture was safe with respect to what might be coming for next 6 months.  So for example, if I saw a lot of ideas coming in around different types of Searching, it would mean I would keep something like Elastic Search on the Tech Radar - as in this might become more relevant very quickly at some stage.   It would also help strike a good balance in technical decisions and avoid over engineering for scenarios that were extremely unlikely.  

Final words

Anyway, that's about it.   I believe no matter what you are doing, to get the best results you need a good structure.  In software, that means you need a good process. Everyone needs to know what they need to be doing, what the priority is and the structure should make it as easy as possible for the team (which is always going to be a mixture of skills and backgrounds) to work together to deliver a top quality product.