At Third & Grove, we have mostly adopted agile and scrum methodologies in our web and mobile development projects, and we’ve learned that this dramatically changes the way testing needs to be perceived and integrated throughout the project. We are still fine-tuning the process, but we’ve picked up some tips along the way — and can also share some pitfalls to avoid — when testing in an agile environment.
1. Encourage collaboration among the Product Owner, Developer, and Tester
Rather than a linear “handoff,” testing has to go “hand in hand” with programming tasks for agile testing to work. A cyclical relationship needs to exist among the product owner, programmer and tester to refine requirements before, during and after the actual development.
A tester can provide valuable input from the outset, not only on defects, but also on usability of features. The tester viewpoint is to think through all happy paths, unhappy and error paths, and then write the appropriate test cases.
Writing test cases is an involved process that requires various steps and inputs depending on the complexity of the feature:
- learning what it's about
- obtaining and/or devising testing instructions
- setting up the testing materials
- thinking through the various scenarios and edge cases
- ensuring usability
Whenever possible, it’s best to do this before development, so that the developer can build all paths at the same time, rather than as a defect after testing.
All three roles should keep in mind the mantra “How do we test it?” which is just as important as “How do we build it?”
2. Integrate testing in an agile environment early on
As can be seen from tip #1, testing is no longer a phase tagged on to the end of the development process as in a traditional waterfall environment. In an agile development environment, testing has to be integrated throughout the entire iteration cycle. We find that test phases need to occur in parallel with other tasks, as outlined in the following workflow.
Phase 1: Scrum master reviews backlog and schedules sprint. Product owner writes stories
Phase 2: Engineers start developing. QA starts building test cases.
Phase 3: Engineers continue developing. QA starts testing initial tickets and submit defects.
Phase 4: Engineers finalize development and resolve bug tickets. QA does integration and compatability testing.
Phase 5: Engineers continue resolving bug tickets. QA verifies bugs. UAT begins.
Phase 6: Engineers resolve UAT bugs. QA verifies bugs. Launch.
3. Promote various channels of communication
Excellent communication among the project manager, scrum master, product owner, programmer and tester is crucial to the success of the project (note: sometimes the project manager, scrum master and product owner roles may be collapsed into 1-2 people). From a tester’s perspective, we’ve found that various forms of communication help to keep testers in the loop, especially as our company is distributed over various time zones and physical locations.
- Beginnings and endings
- Hold a kickoff meeting at the beginning of each project/sprint describing the scope of the project/sprint and any risks/challenges
- Create a test brief detailing information such as testing budget/hour limitations, timeframes and milestones, types of testing to be done, risks, mockup repositories, testing URLs
- Announce beginning and end of sprints (via email, work chat tool, stand-upshttps://www.thirdandgrove.com/wp-admin/post.php?post=1969&action=edit&lang=en, etc.)
- Communicate release schedules ahead of time
- Be clear about due dates
- Ongoing
- Hold a daily stand-up during intense project periods (as possible given time zone differences)
- Have a weekly touch base when the project is phasing down or in a lull, or when a daily standup is not possible
- Even if your whole team is located in the same office, it’s still useful to have a team messaging app, such as HipChat and Slack, integrated into the workflow. These allow your teams to set up group and private channels around projects, topics or individual discussion. (We use Slack as our chat system – see more in our blog post about the Slack app.)
4. Work that agile board
We’re assuming that if you are in an agile environment, you are already use a shared workflow board, whether a physical wall with post-its or an online app such as JIRA. Agile boards help to make the workflow efficient, but only if everyone understands how it works and uses the same methods. Below we’ve outlined some tips that may help the trio of product owner, developer and tester speak the same language around story requirements and ticket workflow, as well as pitfalls to avoid.
The Workflow Enigma:
Once a workflow is implemented, train staff on how to use it. The scrum master has a crucial role here, but everyone should be clear about when a ticket should be pushed to testing, and who it should be assigned to. Adequate QA and UAT steps should be included. If a project goes through various development and staging servers, that should be reflected in the testing workflow as well. Keep a document on hand outlining the different stages to remind participants on the steps.
Confusion about workflow leads to tickets getting stuck and unattended.
The Test Trajectory Tripup:
Plan ahead and include the QA team early on in the process. Don’t wait until development is complete to show any signs of life to the QA team.
If the tester is first learning about a feature when assigned a story ticket to test, you have already failed!
The Instruction Deficiency:
If the feature requires testing a complex process, or especially a backend function, the product owner or engineer should add overall testing steps to the story ticket whenever possible. The tester will fill in the in-between steps.
Don’t waste time making the tester track the programmer down for instructions after the fact. Anticipate testing needs from the outset.
The Ticket Constipation Dilemma:
For the scrum master, be sure to keep tickets flowing and ensure QA knows when items are ready for testing.
Don’t keep tickets stacking up.
The Bug Resolution Relevancy:
As bugs come in, the scrum master should determine which are truly relevant to get passed in the sprint. (The others can wait for a future sprint or get put into the backlog.) Allow additional development and testing time to resolve and verify these bugs at the end of the same sprint. Testers need these components working for integration testing in future sprints.
Don’t leave relevant bug tickets unresolved until right before the project release, otherwise testers won't be able to pass story tickets in their same sprint and keep building on system testing in future sprints.
The Single Ticket Dependency:
Agile boards are great, but we can’t rely on just the tickets to sign off on testing. Single tickets are great for component testing, but we need to build time in for integration testing. Depending on your workflow, you might consider creating epic tickets that describe an end-to-end test.
Don't be ticket dependent — build time for end-to-end testing.
The Story–Bug Conundrum:
Keep your story tickets clean. When the work is done on that ticket, it is done. Any problems relating to the work should be addressed in separate bug tickets, no matter how small. We use the link tool to assign the bug tickets to its parent story so it’s easier to track when that feature is bug free and ready to pass on to UAT. When there are multiple testers, it makes it easier to glance at the board and see if defects have already been identified.
Don’t add bugs to a story ticket. This makes it difficult to track the progress of the defect and the true state of the story ticket, and engineers have a difficult time tracking their time against the work they do resolving the bug.
The Bug Ticket Inadequacy:
Anyone submitting a defect should write complete bug tickets, no matter how small an issue. Provide user data, URLs, screenshots and complete steps. We use a template that anyone filing a defect must use.
- Preconditions (user role, platform/browser, server, URLs/locations, any other state that must be in play)
- Steps
- Expected Results
- Actual Results
- Type of issue (cross-browser, functional, theming, etc.)
Don’t make the scrum master or engineer have to think through your process with skimpy details and contact you for clear steps.
The Bug Ticket Necessity:
Break out issues into bite-size morsels and create separate tickets, no matter how small. This makes it easier for an engineer to go through a task and close it out. We try to limit one functional defect to a ticket. For theming issues, we try to keep a few relevant ones grouped together (such as on the same page). In this case, try to keep it to 5-6 separate pieces in one ticket.
If there are too many defects in the same ticket, issues will be missed.
As there will always be changes in timelines and requirements mid-project, the workflow will never be 100% perfect. But having a solid system in place and an understanding of the role of testing in an agile environment will maximize efficiency and assure a better quality product.