LibreOffice Quality Assurance: six months in statistics (part 1)

During the last six months (from 23 November 2016 to 21 May 2017), many things have happened in LibreOffice and in Bugzilla, its bug tracker, where bugs are reported by users, triaged by the quality assurance (QA) team and finally handled by developers, if needed.

New bugs

During this time, 3664 bugs were reported by 1621 users, of which 425 are tagged as enhancement requests.

Top 15 reporters:

  1. Telesto (285)
  2. Yousuf Philips (jay) (103)
  3. Xisco Faulí (97)
  4. Regina Henschel (61)
  5. Thomas Lendo (60)
  6. Áron Budea (51)
  7. Volga (45)
  8. Gábor Kelemen (44)
  9. Samuel Mehrbrodt (42)
  10. Heiko Tietze (31)
  11. andreas_k (28)
  12. Cor Nouws (27)
  13. Miklos Vajna (25)
  14. Mike Kaganski (25)
  15. Timur (23)

The following chart shows the total number of bugs reported each week during this period, week 48 of 2016 being the week with the highest number of bug reports (185) and week 52 with the lowest (94) – and with week 52 being the last week of the year, this comes as no surprise:

Writer (1168) is the component with the most bug reports, followed by Calc (647), LibreOffice in general (495), Impress (284) and the user interface (UI) (253):

Most of these bugs are reported for all operating systems (2194). However, some are specific to Windows (791), Linux (518) or macOS (142). This is an important part of the triage process, as it’s fundamental to determine whether a bug affects all operating systems or just some of them.

The same happens with the CPU architectures. Most of these bugs affect all architectures (2725), while some affect only x64 (790) or x86 (140) computers:

Onto bug statuses: freshly reported bugs start as UNCONFIRMED, and have to be independently confirmed to become NEW. If they can’t be confirmed, the QA team often asks further details from the reporter, and sets the status to NEEDINFO.

Once a bug has been confirmed, a developer can pick it up, and either have the bug ASSIGNED to him or her – or submit a fix and set status to RESOLVED FIXED right away. Once the issue has been fixed, the fix should be (and sometimes is) VERIFIED.

A couple of other statuses could indicate different reasons why the bug report was closed: the bug is not there anymore (RESOLVED WORKSFORME), it was already reported (RESOLVED DUPLICATE) or it was not a bug in the first place (RESOLVED NOTABUG) to name a few.

This is the current status of the reported bugs:

Here we can see that most of them are either set to RESOLVED (1534) or to NEW (1302) which represent 41.8% and 35.5% of the total respectively. 373 are set to NEEDINFO and 257 are still UNCONFIRMED.

In the following chart, we can see that most of the RESOLVED bugs created during the last six months are closed as DUPLICATE (550) or already FIXED (549). 205 are closed as WORKSFORME, and 201 as NOTABUG.

Unconfirmed bugs

Having a low number of unconfirmed bugs allows the QA team to have a quicker feedback loop between users and developers.

On 17 November 2016, the number of unconfirmed bugs was at 562 – and on 11 May 2017 it was at 450, which shows a downward trend over the months:

During this time, 2941 bugs were triaged and moved from UNCONFIRMED to some other status by 174 people.

Top 20 ‘Confirmers’:

  1. Xisco Faulí (727)
  2. Buovjaga (648)
  3. V Stuart Foote (139)
  4. tommy27 (109)
  5. Áron Budea (101)
  6. m.a.riosv (94)
  7. Julien Nabet (84)
  8. Alex Thurgood (83)
  9. Heiko Tietze (81)
  10. Jacques Guilleron (79)
  11. Yousuf Philips (jay) (77)
  12. Telesto (69)
  13. Cor Nouws (40)
  14. Samuel Mehrbrodt (34)
  15. carlos.deambroggio (33)
  16. Regina Henschel (30)
  17. Olivier Hallot (24)
  18. Joel Madero (23)
  19. Miklos Vajna (22)
  20. Adolfo Jayme (22)

You can check the current list of unconfirmed bugs here. More help is always welcome in keeping the project healthy and bug-free!

This concludes the first part of the statistics – we’ll post the second part soon, so keep an eye on the blog.

First Bug Hunting Session for LibreOffice 5.4

LibreOffice 5.4 is due to be announced at the end of July 2017, with many new features (those already implemented are summarized on the release notes page – https://wiki.documentfoundation.org/ReleaseNotes/5.4 – with much more to come).

In order to find, report and triage bugs, the QA team is organizing the first Bug Hunting Session on Friday, April 28, 2017. All details are available on the specific wiki page: https://wiki.documentfoundation.org/QA/BugHuntingSession/5.4.0Alpha.

Tests will be performed on the first Alpha version of LibreOffice 5.4, which will be available on the pre-releases server (http://dev-builds.libreoffice.org/pre-releases/) a few days before the event. Builds will be available for Linux (DEB and RPM), macOS and Windows – and they will run in parallel with the production version, so you don’t have to install the version of LibreOffice you’re using.

Mentors to help you report and confirm bugs will be available on April 28, 2017, from 8AM UTC to 10PM UTC. Of course, hunting bugs will be possible also on other days, as the builds of this particular Alpha release (LibreOffice 5.4.0 Alpha1) will be available until the middle of May.

During the day there will be two dedicated sessions: the first to chase bugs on the main LibreOffice modules between 3PM UTC and 5PM UTC, and the second to test a set of the top 7 features between 5PM UTC and 7PM UTC.

During the dedicated sessions, we will concentrate all efforts on chasing and reproducing bugs, in order to confirm and file them in a more comprehensive way. Of course, the more comprehensive will be the bug report, the easier will be for the developers to solve the bugs in time for the final release.

We hope to see you there, and if you take part, thanks for helping to make LibreOffice more robust for everyone!

Video interview: Xisco Fauli, QA engineer for LibreOffice

Xisco Fauli works for The Document Foundation as a quality assurance engineer, helping the QA community handle bug reports, triaging and bibisecting. We talked to him about projects he’s working on, and how everyone can get involved:

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

Third Bug Hunting Session for LibreOffice 5.3

bhs_5-3-0-beta2_largeLibreOffice 5.3 will be announced at the end of January 2017, with a large number of new features which are summarized on the release notes page: https://wiki.documentfoundation.org/ReleaseNotes/5.3. In order to find, report and triage bugs, the QA team is organizing a third bug hunting session on Friday, December 9, and Saturday, December 10, 2016.

Tests will be performed on LibreOffice 5.3 Beta 2, which will be available on the pre-releases server (http://dev-builds.libreoffice.org/pre-releases/) in time for the event. Builds will be available for Linux (DEB and RPM), MacOS and Windows, and will run in parallel with the production version.

Mentors will be available on December 9 and December 10, 2016, from 8AM UTC to 10PM UTC. Of course, hunting bugs will be possible also on other days, as the builds of LibreOffice 5.3.0 Beta 2 will be available until mid December.

We would like to see the new font layout engine tested on MacOS in particular. Please drag all your MacOS friends and relatives into the testing effort! All details of the third bug hunting session are available on the specific wiki page: https://wiki.documentfoundation.org/QA/BugHuntingSession/5.3.0Beta2.

During the dedicated sessions, we will concentrate all efforts to chase and reproduce the bugs, in order to confirm and file them in a more comprehensive way. Of course, the more comprehensive the bug report, the easier it will be for the developers to solve the bugs in time for the final release.

Second Bug Hunting Session for LibreOffice 5.3

noun_83830_ccLibreOffice 5.3 will be announced at the end of January 2017, with a large number of new features which are summarized on the release notes page: https://wiki.documentfoundation.org/ReleaseNotes/5.3. In order to find, report and triage bugs, the QA team is organizing a second bug hunting session on Friday, November 25, 2016. Tests will be performed on the Beta version of LibreOffice 5.3, which will be available on the pre-releases server (http://dev-builds.libreoffice.org/pre-releases/) a few days before the event. Builds will be available for Linux (DEB and RPM), MacOS and Windows, and will run in parallel with the production version.

Mentors will be available on November 25, 2016, from 8AM UTC to 10PM UTC. Of course, hunting bugs will be possible also on other days, as the builds of this particular Beta release (LibreOffice 5.3.0 Beta1) will be available until mid December.

During the day there will be two dedicated sessions: the first to chase bugs on the main LibreOffice modules between 3PM UTC and 5PM UTC, and the second to test a set of the top 7 features between 5PM UTC and 7PM UTC. All details of the second bug hunting session are available on the specific wiki page: https://wiki.documentfoundation.org/QA/BugHuntingSession/5.3.0Beta1.

During the dedicated sessions, we will concentrate all efforts to chase and reproduce the bugs, in order to confirm and file them in a more comprehensive way. Of course, the more comprehensive will be the bug report, the easier will be for the developers to solve the bugs in time for the final release.

FOSDEM Call for Papers: Open Document Editors DevRoom

fosdemFOSDEM is one of the largest gatherings of Free Software contributors in the world and happens each year in Brussels (Belgium) at the ULB Campus Solbosch. In 2017, it will be held on Saturday, February 4, and Sunday, February 5.

As usual, the Open Document Editors DevRoom will be jointly organized by Apache OpenOffice and LibreOffice, on Saturday, February 4, in room 4.401 in Building K (from 10:30AM to 6:30PM). The shared devroom gives every project in this area a chance to present ODF related developments and innovations.

We are now inviting proposals for talks about Open Document Editors or the ODF document format, on topics such as code, extensions, localization, QA, UX, tools and adoption related cases. This is a unique opportunity to show new ideas and developments to a wide technical audience.

Length of talks should be limited to a maximum of 30 minutes, as we would like to have questions after each presentation, and to fit as many presenters as possible in the schedule. Exceptions must be explicitly requested and justified. You may be assigned LESS time than you request.

All submissions have to be made in the Pentabarf event planning tool: https://penta.fosdem.org/submission/FOSDEM17.

While filing your proposal, please provide the title of your talk, a short abstract (one or two paragraphs), some information about yourself (name, bio and photo, but please do remember that your profile might be already stored at Pentabarf).

To submit your talk, click on “Create Event”, then make sure to select the “Open Document Editors” devroom as the “Track”. Otherwise, your talk will not be even considered for any devroom at all.

If you already have a Pentabarf account from a previous year, even if your talk was not accepted, please reuse it. Create an account if, and only if, you don’t have one from a previous year. If you have any issues with Pentabarf, please contact ode-devroom-manager@fosdem.org.

The deadline is Monday, December 5th, 2016. Accepted speakers will be notified by Sunday, December 11th, 2016. The DevRoom schedule will be published on the same day.

Recording Permission

The talks in the Open Document Editors DevRoom will be audio and video recorded, and possibly streamed live too.

In the “Submission notes” field, please indicate that you agree that your presentation will be licensed under the CC-BY-SA-4.0 or CC-BY-4.0 license and that you agree to have your presentation recorded. For example: “If my speech is accepted for FOSDEM, I hereby agree to license all recordings, slides, and other associated materials under the Creative Commons Attribution Share-Alike 4.0 International License. Sincerely, Name”.