As part of the tasks of the subject Case Studies II, of the Master on Free Software, we have to publish a series of posts with the aim of summarizing and analyzing the content of the submissions received in each of the sessions and are presented by different personalities of different free software projects.
The first presentation was giving by Hirliman Ludovic, who is responsible for Quality Assurance in Thunderbird and made a presentation on “Quality Assurance, Thunderbird & Mozilla”, which outlined the methodology used in Mozilla QA and experience leading the QA section in Thunderbird.
It is first necessary to understand the reasons for which are essential QA for software projects, and as they say a image is worth a thousand words (I found it very fun that Ludovic use precisely this picture, because in my work we have exactly the same pasted on a board, to lest we forget this in our projects):
QA is precisely one of the most important steps to secure and mitigate the changes and deviations produced by the different views of each of the groups that are involved in a project (software project or not), allowing to detect errors, redirecting the project to prevent future errors, strengthening it and extend it to the most faithful to the client’s needs, that after all is who demands it and that is the project ultimate goal.
The technical and formal definition of QA is as follows, according to Wikipedia:
“Quality assurance (QA) refers to the planned and systematic activities implemented in a quality system so that quality requirements for a product or service will be fulfilled. It is the systematic measurement, comparison with a standard, monitoring of processes and an associated feedback loop that confers error prevention”.
In the case of Mozilla QA process consists of the following steps:
- Writting test cases: key each time you are deploying any functionality in the project, with the aim of validating each of the functions developed for the program in isolation. These test cases will allow to tests each functionality individually, but also lay a foundation to test various features ensuring that it is error free.
- Maintaining Test cases: yet to have good test case is useless if not maintained and evolve in the same way that changes the source code and functionality, as these will allow us to perform a simple regression testing and ensuring quality in parts, in more static parts as those who suffer major changes. Dispose of obsolete test cases cause a very negative effect, not only because they ensure the quality of the functionality that we already have but because they can be misleading or give us false positives.
- Testing: of course do test cases and another tests that are not covered in these. This is extremely important for software projects in general, since in many cases the developers, because they are very involved in the product or for convenience, they just prove what they know to work and leave out a lot of tests that can cause failures in the program. In this part play an important role that test users without having much knowledge in the tool but can report lots of bugs to them to rectify the errors found in the program. In some subjects we have seen that one of the biggest problems with user testing is to get the correct information clearly to reproduce the error.
- Managing bug (bug triagge): after receiving reports of bugs is necessary to manage it appropriately, so that there is a group of people who are responsible to review and categorize them, in order to discard those that do not serve, merge duplicates, prioritizing relevant bugs, etc.
- Signing off: finally is very important to include the solution to the bugs in the next version of the product, provided the bug has passed the appropriate test cases and can ensure the good performance of the product. You can consult the upcoming beta and stable versions for Mozilla products on this website https://wiki.mozilla.org/Releases#.
To carry out all QA tasks set out above in an efficient and optimal is very important and necessary to use some tools such as:
- Test case manager.
- Bug report tool/database.
- Testing tools.
- Bugzilla: the bugtracking system.
- Buildbot + tbpl + tinderbox: automatic building the source.
- Litmus: test case management.
- Wikis: documentation.
- Mailing lists: with different purposes, for ads, for developers, for differents countries, etc.
- IRC: irc channel for online communication.
At the end of the presentation Ludovic told us his experience in the QA team of Thunderbird (team made by him alone) and supported by volunteers. Ludovic is responsible for reviewing and distributing test cases among the volunteers during the test phases. He stressed the importance of creating unit test cases for each module and uploaded to the repository implemented with the aim to execute when the application is built so you can see if any faults or not.
In my opinion it was a pretty interesting conversation, in my case work in a software development department and know the importance of thorough testing, not only unit tests for each of the modules developed, but also regression testing in the rest modules to ensure the operation of the program. It’s very interesting to see a small picture of how it works projects as large and used as Firefox or Thunderbird, and the tools used to ensure the quality of their products as simple and automated.
At the end of the session we were able to ask questions to Ludovic, in my case I asked if they used different tools or filters in Bugzilla for the treatment of the errors reported or was a manual task, as I was watching this bug-tracking and I found it tedious and complex. Ludovic said he has several forms, the first querys using advanced system that exists in Bugzilla, so it allows more detailed filtering of bugs, some of it by searching google and the other way is when you find something interesting to send a email, that way then look in your mailbox is quick and easy.
- QA in Thunderbird and Mozilla Part One (Video).
- QA in Thunderbird and Mozilla Part Two (Video).