While there is a lot of attractions towards continuous deployments, deploying to production 37 times a day etc, still the area of “Test Automation” seems to gain less attention that what it deserves. Most places adopt Agile could possibly suffer with lack of drive and ownership on Quality/Test automation during the transition, especially when your QAs transitioning to be an Agile team member. From what I’ve seen, those QAs who adopt the change are likely to get attreacted more towards product development and other tasks and team might get to a point where no one evangelizes quality. Especially, those places with legacy product in the market, it’s much harder. Reasons may be many, our PO wants new features, we got deal with tech debt, lot of energy goes into support and bug fix, our developers hate test automation;its a QA thing, we found a new tool and want to try that out, no proper tooling guidance, integration and so on..but the impact will hit our face sooner than later. I’m collaborating with atleast 100 developers in regular basis and could notice ~ 5% of them talking about practices like BDD and TDD. Remainder of the crowd not necessarily hate automating their verifications, rather their professional passion and interest is not in test automation. After all, AngularJs got better future for developers than Selenium webdriver automation. However, delivering high quality working software needs quality checks and verification. If we were to align with the faster delivery cadence, release once in 2 weeks, those verifications need to be automated mindfully.
Automation has been there since ages, tooling space have evolved so much and plenty of reasons why automation makes sense. But still those don’t seem to motivate agile teams. If you are one of those team members who would prefer to pass on the automation task to someone else, pls continue to read. IMHO, at the very least, automated tests would help in below situations
Pull Request shivering
Geographically distributed teams can’t be avoided in software development. Even within the same team, team members are spread across the globe, nowadays we develop a lot of small components, libraries and put it out for consumption. Great example NPM carries 163,000+ packages, libraries and reusables. Micro services is a buzz word amongst many. These situations are driving us towards small, self-contained repositories of code base and potentially a lot of run time dependencies between them. When more than one developer need to collaborate, pull request (PR) has become a must have a thing.
- does the pull request send shivers down your spine?
- when a PR comes in, how confident are you to accept that code change?
- how do you verify the code quality of the incoming PR? how would you verify whether it confirms to your coding standards and wouldn’t degrade your current code quality?
- how do you verify that the new PR wouldn’t break current functionality?
- naturally code base grows, multiple people contribute, is there anyway to know the current state of a given functionality?
If it’s a small product, being developed by a few co-located developers, all the team members might stay in sync and might follow the same routine steps to verify releases. As product complexity increases, this will become a repeated time-consuming activity. When product grows, more people contribute, it’ll demand for manual procedures/documentation to make sure everyone checks the mission critical functionality the same way to certify. While human verification is valuable, we could delegate some error prone, multi step complex processes to the machine
- is your post deployment verification process becoming a nightmare?
- are you spending a lot of time to verify the deployment?
- are you unsure that the new build didn’t break mission critical functionalities?
- are you unsure that the new build didn’t degrade performance?
- are you catching significant bugs after the release while they could have been fixed earlier?
- do you notice inconsistency in verification process? every team member follows different steps?
If current level of automation is so low, you are likely to suffer from these symptoms. For some reason, if you are in this situation, it’s a great opportunity to showcase your technical leadership to up level your team. recipe is not that hard, couple of easy recommendations
Sonar Analysis, quality gates, attach quality metrics with Pull Request
Minimum, I would suggest to use SonarQube. Sonar is a flagship open source software that will add tremendous value to understand current code quality metrics and improve towards your target goal. Sonar can be used to analyze the codebase and provide various code quality feedback, can be integrated to most of the CI build systems to run code analysis upon check in, can be integrated with source control and be an indicator of code quality along with pull request. Example Sonar + Stash PR integration, very handy for code reviews.
Consider writing some tests (unit and/or integration tests) that can be executed part of the CI process. Sonar offers Quality Gates that can help enforcing standards, fail the CI build when desired level of code quality is not met.
Smoke Test Suite
The level of automation such as unit testing, api, gui doesn’t matter. Patterns and practices such as BDD, TDD, ATDD doesn’t matter. It makes complete sense to identify mission critical flows of your application and start developing smoke testing suite. If your app is e-commerce, search is a must have. checkout is a must have. I would at least build a smoke testing suite and work towards integrating that with the delivery pipeline. Shoot for high confidence, faster, repeatable and consistent releases.
With increased modularity and libraries sharing eco system, these would also help the consumer to consume your library with high confidence. I generally don’t pull down any library from github unless I see tests passing.