LibreOffice Quality Assurance: six months in statistics (part 1)

During the last six months (from 23 November 2016 to 21 May 2017), many things have happened in LibreOffice and in Bugzilla, its bug tracker, where bugs are reported by users, triaged by the quality assurance (QA) team and finally handled by developers, if needed.

New bugs

During this time, 3664 bugs were reported by 1621 users, of which 425 are tagged as enhancement requests.

Top 15 reporters:

1. Telesto (285)
2. Yousuf Philips (jay) (103)
3. Xisco Faulí (97)
4. Regina Henschel (61)
5. Thomas Lendo (60)
6. Áron Budea (51)
7. Volga (45)
8. Gábor Kelemen (44)
9. Samuel Mehrbrodt (42)
10. Heiko Tietze (31)
11. andreas_k (28)
12. Cor Nouws (27)
13. Miklos Vajna (25)
14. Mike Kaganski (25)
15. Timur (23)

The following chart shows the total number of bugs reported each week during this period, week 48 of 2016 being the week with the highest number of bug reports (185) and week 52 with the lowest (94) – and with week 52 being the last week of the year, this comes as no surprise:

Writer (1168) is the component with the most bug reports, followed by Calc (647), LibreOffice in general (495), Impress (284) and the user interface (UI) (253):

Most of these bugs are reported for all operating systems (2194). However, some are specific to Windows (791), Linux (518) or macOS (142). This is an important part of the triage process, as it’s fundamental to determine whether a bug affects all operating systems or just some of them.

The same happens with the CPU architectures. Most of these bugs affect all architectures (2725), while some affect only x64 (790) or x86 (140) computers:

Onto bug statuses: freshly reported bugs start as UNCONFIRMED, and have to be independently confirmed to become NEW. If they can’t be confirmed, the QA team often asks further details from the reporter, and sets the status to NEEDINFO.

Once a bug has been confirmed, a developer can pick it up, and either have the bug ASSIGNED to him or her – or submit a fix and set status to RESOLVED FIXED right away. Once the issue has been fixed, the fix should be (and sometimes is) VERIFIED.

A couple of other statuses could indicate different reasons why the bug report was closed: the bug is not there anymore (RESOLVED WORKSFORME), it was already reported (RESOLVED DUPLICATE) or it was not a bug in the first place (RESOLVED NOTABUG) to name a few.

This is the current status of the reported bugs:

Here we can see that most of them are either set to RESOLVED (1534) or to NEW (1302) which represent 41.8% and 35.5% of the total respectively. 373 are set to NEEDINFO and 257 are still UNCONFIRMED.

In the following chart, we can see that most of the RESOLVED bugs created during the last six months are closed as DUPLICATE (550) or already FIXED (549). 205 are closed as WORKSFORME, and 201 as NOTABUG.

Unconfirmed bugs

Having a low number of unconfirmed bugs allows the QA team to have a quicker feedback loop between users and developers.

On 17 November 2016, the number of unconfirmed bugs was at 562 – and on 11 May 2017 it was at 450, which shows a downward trend over the months:

During this time, 2941 bugs were triaged and moved from UNCONFIRMED to some other status by 174 people.

Top 20 ‘Confirmers’:

1. Xisco Faulí (727)
2. Buovjaga (648)
3. V Stuart Foote (139)
4. tommy27 (109)
5. Áron Budea (101)
6. m.a.riosv (94)
7. Julien Nabet (84)
8. Alex Thurgood (83)
9. Heiko Tietze (81)
10. Jacques Guilleron (79)
11. Yousuf Philips (jay) (77)
12. Telesto (69)
13. Cor Nouws (40)
14. Samuel Mehrbrodt (34)
15. carlos.deambroggio (33)
16. Regina Henschel (30)
17. Olivier Hallot (24)
18. Joel Madero (23)
19. Miklos Vajna (22)
20. Adolfo Jayme (22)

You can check the current list of unconfirmed bugs here. More help is always welcome in keeping the project healthy and bug-free!

This concludes the first part of the statistics – we’ll post the second part soon, so keep an eye on the blog.