QA Team + TripleA Release Process
Background: TripleA QA team has been operating for some time:
We have coordinated mostly via gitter channel: https://gitter.im/triplea-game/QA
We coordinate testing via a pretty simple google spreadsheet:
When there are multiple green bars lining up, meaning multiple people have reasonably checked as much of the game as they can, that version of the game can become the latest release.
Why the QA team?
The QA team is essentially a way to coordinate multiple people to test the software before it is released. The team was created in response to a number of problems.
- Development team had trouble tracking results and general confusion on "is it safe to release?". With a defined policy and tracking, we can now look at the testing results and know when a specific version has been confirmed to be okay for release.
- the TripleA code base is very difficult to modify and repeated releases kept introducing small but extremely significant bugs. Regression testing for any of these is a huge effort, and needs to be done repeatedly as the code base is constantly modified. It's too large of a task for just one or two developers, hence the QA team.
How will we know if the QA team is successful?
- The number of major bugs reported by users to github issues will go down and head towards zero
- The number of times a release is 'rolled back' will go down and head to zero
Where is the QA team heading?
- Still formalizing how we operate, but we're getting pretty close to a stable operating process
- Long term try to work with development team to reduce the number of items being hand tested
- Short/medium term, formalize testing a bit, a few things are getting through, perhaps a testing check list of all features and cases that need to be checked would be a good start