QA Team + TripleA Release Process


  • Admin

    Background: TripleA QA team has been operating for some time:
    https://forums.triplea-game.org/topic/268/volunteers-needed-early-release-testing-group

    We have coordinated mostly via gitter channel: https://gitter.im/triplea-game/QA

    We coordinate testing via a pretty simple google spreadsheet:
    https://docs.google.com/spreadsheets/d/1CSZ4s7wBLcvgue_IHwl-CbjQd95qj8hhR8m89ngCzbU/edit#gid=0

    When there are multiple green bars lining up, meaning multiple people have reasonably checked as much of the game as they can, that version of the game can become the latest release.


  • Admin

    Why the QA team?

    The QA team is essentially a way to coordinate multiple people to test the software before it is released. The team was created in response to a number of problems.

    • Development team had trouble tracking results and general confusion on "is it safe to release?". With a defined policy and tracking, we can now look at the testing results and know when a specific version has been confirmed to be okay for release.
    • the TripleA code base is very difficult to modify and repeated releases kept introducing small but extremely significant bugs. Regression testing for any of these is a huge effort, and needs to be done repeatedly as the code base is constantly modified. It's too large of a task for just one or two developers, hence the QA team.

    How will we know if the QA team is successful?

    • The number of major bugs reported by users to github issues will go down and head towards zero
    • The number of times a release is 'rolled back' will go down and head to zero

    Where is the QA team heading?

    • Still formalizing how we operate, but we're getting pretty close to a stable operating process
    • Long term try to work with development team to reduce the number of items being hand tested
    • Short/medium term, formalize testing a bit, a few things are getting through, perhaps a testing check list of all features and cases that need to be checked would be a good start

Log in to reply
 

Looks like your connection to TripleA Forum was lost, please wait while we try to reconnect.