07 Jun 2017

LibreOffice Quality Assurance: six months in statistics (part 2)

Posted in LibreOffice, QA By Mike Saunders On June 7, 2017

This is the second part in our blog series about the LibreOffice QA (quality assurance) community – see here for the first part.

Regressions

During the six month period from 23 November 2016 to 21 May 2017, 553 bugs were identified as regressions by 61 people. This means a feature behaved correctly in the past – but a change in the code made it work incorrectly. This is an expected problem in any software project, especially when many changes to the code are done every day. For that reason it’s very important to determine whether a bug is a regression or not – and to identify regressions as soon as possible to maintain the quality of the software.

It’s also important to mention the tremendous amount of effort developers spend on adding new unit tests in order to avoid regressions from being introduced again. More information about that here.

Top 15 regression identifiers:

  1. Xisco Faulí (190)
  2. Buovjaga (81)
  3. Telesto (50)
  4. Áron Budea (35)
  5. Alex Thurgood (32)
  6. Justin L (11)
  7. Cor Nouws (11)
  8. tommy27 (11)
  9. Julien Nabet (9)
  10. Yousuf Philips (jay) (9)
  11. m.a.riosv (9)
  12. V Stuart Foote (7)
  13. Kevin Suo (6)
  14. Luke (6)
  15. Jacques Guilleron (6)

These are the statuses of the bugs identified as regressions:

More than half of these bugs have been RESOLVED (243), or resolved and VERIFIED (57) – 54.2% to be exact.

Bisected

Once we identify that a bug is a regression, we can use a tool called bibisect to identify at which point the regression was introduced, allowing us to speed up the process of fixing the bug.

During the last 6 months, 346 bugs have been bisected by 14 people.

Top 14 ‘bisecters’:

  1. Xisco Faulí (186)
  2. Áron Budea (77)
  3. raal (50)
  4. Justin L (10)
  5. Terrence Enger (6)
  6. Michael Stahl (4)
  7. Mike Kaganski (4)
  8. Miklos Vajna (3)
  9. Buovjaga (1)
  10. Kevin Suo (1)
  11. Luke (1)
  12. Jean-Baptiste Faure (1)
  13. Caolán McNamara (1)
  14. Katarina Behrens (1)

And these are the statuses of the bugs bisected:

Of the bisected bugs, 55.7% are either RESOLVED (148), or resolved and VERIFIED (45).

Backtraces

Another way to accelerate the fixing process is by providing a debug trace when there’s a crash, slowness or a freeze in the software.

During the last 6 months, a debug trace has been added to 140 bugs by 14 people.

Top 13 debug trace providers:

  1. Julien Nabet (48)
  2. Xisco Faulí (35)
  3. Buovjaga (28)
  4. fiftyigfuci_f_mi (10)
  5. Alex Thurgood (6)
  6. V Stuart Foote (4)
  7. Áron Budea (4)
  8. Telesto (3)
  9. Terrence Enger (3)
  10. Yousuf Philips (jay) (2)
  11. JoNi (2)
  12. Kevin Suo (1)
  13. Cor Nouws (1)

These are the statuses of the bugs with a debug trace:

More than two-thirds of these bugs have been RESOLVED (74), or resolved and VERIFIED (22) – 69.4% to be exact.

Duplicate bugs

Knowing that many bugs are reported everyday, it’s important to identify if a certain bug was reported previously and therefore whether it’s a duplicate or has never been reported before.
During the last six months, 757 bugs have been identified as duplicates by 91 people.

Top 20 duplicate identifiers:

  1. Xisco Faulí (142)
  2. Buovjaga (116)
  3. V Stuart Foote (79)
  4. Áron Budea (40)
  5. Telesto (31)
  6. Alex Thurgood (28)
  7. m.a.riosv (27)
  8. Julien Nabet (26)
  9. Heiko Tietze (20)
  10. Timur (19)
  11. Khaled Hosny (18)
  12. Yousuf Philips (jay) (15)
  13. Maxim Monastirsky (14)
  14. Adolfo Jayme (13)
  15. Gábor Kelemen (11)
  16. Mike Kaganski (10)
  17. Cor Nouws (10)
  18. Jacques Guilleron (8)
  19. Bartosz (8)
  20. Regina Henschel (6)

Resolved bugs

The cornerstone of the project is its development work. During the six month period, 955 bugs were set to RESOLVED FIXED by 147 people.

Top 20 fixers:

  1. Caolán McNamara (73)
  2. Miklos Vajna (53)
  3. Eike Rathke (41)
  4. Michael Stahl (39)
  5. Julien Nabet (38)
  6. Justin L (36)
  7. Adolfo Jayme (33)
  8. Heiko Tietze (29)
  9. Samuel Mehrbrodt (29)
  10. Xisco Faulí (26)
  11. Gábor Kelemen (25)
  12. Yousuf Philips (jay) (23)
  13. Maxim Monastirsky (22)
  14. Markus Mohrhard (22)
  15. V Stuart Foote (19)
  16. Zolnai Tamás (18)
  17. Mike Kaganski (18)
  18. Katarina Behrens (17)
  19. Khaled Hosny (16)
  20. Winfried Donkers (15)

Verified fixed bugs

Finally, once a bug has been fixed, it’s important to verify that it has indeed been fixed.

During the last six months, 212 fixes have been verified by 28 people.

Top 15 verifiers:

  1. Justin L (65)
  2. Xisco Faulí (24)
  3. Buovjaga (18)
  4. Timur (16)
  5. m.a.riosv (11)
  6. Julien Nabet (9)
  7. Heiko Tietze (7)
  8. Terrence Enger (7)
  9. Jean-Baptiste Faure (7)
  10. Thomas Lendo (6)
  11. Áron Budea (6)
  12. raal (4)
  13. Yousuf Philips (jay) (4)
  14. Cor Nouws (4)
  15. V Stuart Foote (3)
  16. Kevin Suo (3)
  17. Luke (3)
  18. Jacques Guilleron (3)
  19. Zineta (2)
  20. Alex Arnaud (2)

Get Involved!

So, you’ve seen what the QA team is doing across the LibreOffice project – why not get involved and help out? Even if you only have half an hour of spare time each week, by confirming bugs (and fixes) you can make LibreOffice better for millions of people around the world. And in addition, you build up valuable experience working with a large project and open source community – which could be very useful for a future career! Discover more about the QA team in our video interview with QA engineer Xisco Fauli.

(Notes about this blog post: raw data can be checked here. For more stats, visit the stats page in the QA wiki.)