Behind the scenes at TDF: LibreOffice QA, so much accomplished so far this year!
Many of you are familiar with LibreOffice or one of its predecessors (StarOffice, OpenOffice.org, etc..), but you may not be aware of the immense amount of work that goes into the production of the software and the careful testing of each release. Although there are many different teams within the LibreOffice community who each perform essential roles in the collaborative development process, I don’t have enough space to cover them all today, so I’ll focus on the LibreOffice QA Team, a group of volunteers and employees of various companies around the world who work tirelessly to identify issues with LibreOffice on all platforms, including issues of interoperability, process, accessibility, and ease of use.
The QA Team has accomplished much in the first quarter of this year, including significant reduction of UNCONFIRMED bug count, broad testing of our support for media on all major platforms, migration of our Bugzilla bug tracker to our own infrastructure, information and advice for our Annual Report, exhaustive work testing our LibreOffice Android port, and major improvements with our bibisect repositories. I’m sure I’ve omitted something from that list, but the sentence was getting long enough that I figured I should stop before I ran out of breath 😉
As of mid-December of last year, our UNCONFIRMED bug count was steadily dropping, but was still above 500. Throughout the holiday season and into January, the QA Team amazed everyone by lowering the number of bugs that needed triage to sub-400, then to 359, and even down below 280. Although our current count has stabilized around 350, we hope that with increased participation we can continue to chip away at the remaining pile.
Although most of the code for displaying images is cross-platform, LibreOffice uses different libraries on Win, Mac, and GNU/Linux for audio and video playback. With confusion about media playback a persistent issue, the QA Team created a set of wiki pages to help clarify the extent and quality of media support on all of our platforms, as well as provide user-friendly information about what codecs we recommend for use. Test support is ongoing, and we very much welcome additional test results or suggestions on how we can increase our playback support on proprietary platforms such as Windows and Mac OS X.
QA’s biggest project this year to date has been our Bugzilla Migration from Freedesktop.org to TDF (The Document Foundation) infrastructure. With careful planning and dry-runs tested over many weeks, our migration went very smoothly and had minimal interruption to the overall development of LibreOffice. With TDF control of Bugzilla in place, we’ve been able to make additional changes to the bug tracker, add new Components and Products for all of our current software projects, and even make small tweaks and add well-crafted messages to assist our users in reporting, updating, and interacting with our bugtracker, with the QA Team, and our developers. We will continue to make changes and improvements to Bugzilla throughout 2015, and are eager to hear from you about any bugs you’ve found in LibreOffice or suggestions you have for enhancement. Please file all bugs and enhancement requests at: https://bugs.documentfoundation.org/.
Throughout 2014, people continually asked me about running LibreOffice on Android, and by the beginning of 2015, I was excited to hear that our port was on its way! Upon the arrival of editing support (currently in beta) in the LibreOffice for Android app, QA has provided insightful feedback about various aspects of document use, feature support, and even special, edge-case problems that affect only a limited number of our users, but which we would like to eliminate (as with all bugs!) With volunteers stepping up to test on phones, tablets, and even laptops, the QA Team has provided extensive testing of our port of LibreOffice to Android, and has committed to keeping UNCONFIRMED bugs for Android in the single digits.
As in 2014, our bibisect repositories remain one of the best investigative tools of the QA Team to help identify the point at which regressions have been introduced into the codebase. With the ability to use binary search and pre-built binaries to quickly focus-in on a particular regression, we can use our time much more effectively, avoiding building and re-building the same versions. The more steps that QA can take to ferret-out the particular commit that caused a crash or changed program behavior, the faster that developers can create a fix and patch the code for the next version of LibreOffice.
Historically, our bibisect repositories have contained only a fraction of all commits to our mainline repository. Due to a confluence of factors, including server resources, software limitations, and hard disk sizes, it was most prudent for us to include only every 60th commit, allowing us to drastically narrow the search field to a much more manageable size. Armed with faster hardware and some careful optimization of non-relevant commits, superstar volunteer Matthew Francis has created a breed of “max” bibisect repositories that include all relevant commits in a single repository. These new repositories will give the QA Team the ability to delve much deeper into the particulars of a given regression, often identifying the particular developer who committed the change, allowing us much faster feedback and progressed towards a clean and consistent codebase.
Of course, QA works on many other tasks as well, including testing and triaging our Android Impress Remote, providing feedback about our websites and infra, helping the Document Liberation Project, and helping to identify new enhancements for LibreOffice. Although our primary focus is the LibreOffice suite, we do our best to keep track of everything else in the ecosystem, so that we can identify any potential problems before they affect many users.
With new platforms such as LibreOffice Online announced, and more interesting developments on the horizon, the members of the QA Team will definitely have more than enough work to keep them busy throughout 2015. If you’re interested in joining our efforts, or just curious about what’s involved in testing such a large project, drop by #libreoffice-qa on Freenode or say hello on our mailing list.
Cheers, Robinson