Behind the scenes at TDF: LibreOffice major releases

tdf-roadtolo52LibreOffice has a time-based release schedule (https://wiki.documentfoundation.org/ReleasePlan), with a major announcement every six months: early February, and early August. A time-based release schedule is predictable, and as such makes it easier to plan large deployments and subsequent updates.

On the other hand, a major announcement every six months translates to a “project that never sleeps”. In fact, you start thinking about the next major release a few weeks after the previous one is over.

In advance of every release, we listen to end users about requested features and changes requested to support the individual workflow. We do not only listen but iteratively present proposals in order to create a user experience that is in alignment with our long-term vision of being the best free office suite, easy for beginners and powerful for experts.

noun_27464Let’s take the just-released LibreOffice 5.2 as an example of the many activities which have been carried out during five full months – from early March to early August – by several members of the team, and a very large number of community members in every geography.

LibreOffice is available in a large number of native languages (actually, it is the office suite available in the largest number of native languages), thanks to the localization activity carried out by volunteers in many geographies. This activity is extremely important, but is one of the less known, as many of the people involved are more interested in getting the work done than in raising their visibility inside the project.

In early March, the development team consolidates the release plan (https://wiki.documentfoundation.org/ReleasePlan/5.2), and the marketing team sets the release date according to industry events or major software announcements. Of course, the release date might need last minute changes, as in 2015, when the launch of LibreOffice 5.0 was shifted because of the announcement of Windows 10.

From that point on, developers work on finalizing new features, which are usually described in the release notes on the wiki (https://wiki.documentfoundation.org/ReleaseNotes/5.2). Based on that page, and on other sources of information within the project, the marketing team starts to brainstorm about the messaging.

LibreOffice 5.2 has been a rather nice example of successful messaging, as many significant articles on popular media underlined the enterprise focus based on a few specific features like document classification and new Calc financial forecasting functions.

noun_83830_ccAt the same time, the Quality Assurance team – with volunteers active in many geographies on many different language version of the office suite – start testing nightly builds to discover bugs and regressions over previous LibreOffice releases. As soon as Alphas and Betas are available, Quality Assurance volunteers organize global bug hunting sessions (https://wiki.documentfoundation.org/BugHunting_Session_5.2.0_Alpha).

Bug hunting sessions are one or two day “virtual” events, where Quality Assurance volunteers provide their expert support to less skilled users, to help them file reports about bugs and regressions. Although they may look boring on paper, they are usually rather engaging, as most people involved get the distinctive feeling of contributing in a significant way to the success of the new LibreOffice major release.

Bug hunting sessions are usually announced in advance on this blog. Stay tuned for the next one, to test the first Beta of LibreOffice 5.3.

Alphas and Betas are extremely important for the marketing team, too. In fact, they are used to test new features, and to understand their relevance for the upcoming announcement. In some cases, as for the classification feature, the team gathers additional information, to add a background document to the press kit.

When the first Release Candidate is announced, around six weeks before the launch date, most of the back office activities have been completed. At that time, the different teams start to concentrate on the final details, to be ready for the new major release.

Localizers have already completed their activity, and committed all translated strings into the Pootle translation system. Quality Assurance experts are focusing their attention on bugs and regressions which might impact on the release, with the help of developers who are refining new features while getting rid of the last “wrinkles” in the source code.

The marketing team defines the announcement strategy, and the media relations activities for the week prior to the announcement. According to the media angle, they develop a couple of different mailing lists: a smaller one for press briefings where recipients are sent individual messages, and a large one for press releases. In addition, country-specific mailing lists are used for press releases in native languages.

classificationAt the same time, the launch documents go from draft to final: a press release, a technical backgrounder for the classification feature, the image visually explaining classification levels, an infographics for the top feature list, and a slide deck which is going to be used for press briefings.

The last steps are coordinated by team members: release management prepares the builds and loads them on the mirrors, in time for the launch, while marketing hosts the phone press briefings with journalists (with the participation of board members, who provide statements about the new release).

On the launch date, release management switches the links on the download page, while marketing publishes the blog post and sends out the email to TDF members and announce mailing lists (https://blog.documentfoundation.org/blog/2016/08/03/libreoffice-5-2-fresh-released-for-windows-mac-os-and-gnulinux/). In addition, it handles the distribution of press releases in English, and in many other native language translations provided by community volunteers, by using the free open source PHPList.

Each LibreOffice major release is a real team effort, which is possible thanks to the contribution of many people around the world.

Last, but not least, we should not forget companies sitting in the Advisory Board (http://www.documentfoundation.org/governance/advisory-board/), which are supporting the global development effort by employing a large number of full time hackers. In addition, a couple of these companies provide a Long Term Supported version of LibreOffice (at the moment, based on the LibreOffice 5.1 family).

Improving LibreOffice User Experience (UX)

g4hDOEET

Effective from May 2016, Heiko Tietze has started working as a consultant to drive LibreOffice UX one step further.

Heiko has been one of the most active UX volunteers during the last few years, and has been instrumental in a rather large number of the user interface improvements since LibreOffice 4.4.

In addition, he has coordinated UX focused research, which has – amongst other results – led to the development of the human interface guidelines (HIG), and to incremental UI changes to menus and toolbars. Currently, the focus is on the improvement of the LibreOffice Draw user interface.

Most of Heiko’s and other UX volunteers’ activities are reflected in the TDF Design blog, where Heiko has extensively reported about the research and the most significant UX improvements, such as the reorganization of toolbars and the new application specific-menus launched in LibreOffice 5.1.

Heiko’s main objectives will be to work with developers to find out what is technically possible and makes sense to improve LibreOffice UX, to grow the UX and design contributor base, and help with UX-related bugs.

Heiko Tietze has a degree and a doctorate in psychology at the University of Jena, where he has also worked as researcher in general psychology. In his professional career, he has worked with neurophysiological methods like electroencephalogram (EEG) and eye tracking (EOG) on different topics, such as vigilance in traffic and the outcome of automation.

On top of his strong methodological and statistical background, he has developed the skills to write software code to deal with large bulks of data, and has specialized in computer science. In the last ten years, he has focused on usability for different customers, with projects from embedded systems to complex desktop software.

In addition to LibreOffice, Heiko is also a long time contributor to the KDE project and a member of the visual design group (VDG).

TDF Marketing Activity (and a Debian Bug Squashing Party!)

As we head towards the second half of the year – and an exciting new release of LibreOffice in August – I wanted to recap some work from the Marketing Team that I’ve been involved in over the last few months. But before that, I’d also like to say congratulations to the LiMux team for a great Debian Bug Squashing Party!

LiMux is a project by the City of Munich to transition away from proprietary software and closed document formats to free software and open standards. GNU/Linux and LibreOffice are key components in this migration, and on 27 – 29 May the LiMux offices hosted a Debian Bug Squashing Party. As the name suggests, this is a mini hackfest where developers work to fix bugs in the Debian GNU/Linux distribution (and related software).

LiMux is based on Kubuntu, which in turn builds on Debian’s mighty foundations; other developers associated with KDE, Kolab and LibreOffice were present as well. Lots of hacking was done and bugs fixed – but it was also a great opportunity just to chat about the state of LiMux, make new contacts, and try great food! The Indian dishes on the first evening were especially good…

Marketing updates

Moving on to Marketing at The Document Foundation: we’ve had a busy few months, attending the CeBIT show in Hannover with new flyers and other materials. Thanks to the ever-industrious community, we received audio translations for our LibreOffice 5.1 New Features videos – so there are now versions in French, German and Spanish. Following this, we started work on the 2015 Annual Report, a summary of last year’s news and activity inside The Document Foundation and LibreOffice. It will be online soon, so stay tuned to this blog!

The LibreOffice main website and The Document Foundation wiki are packed with useful information, but some of it is duplicated or slightly dated, so we’ve been going through both sites to perform cleanups and updates. In addition, we added a new page to track migrations to LibreOffice, a First Steps guide for potential contributors to the marketing project, along with a style guide. On the main website we updated the screenshots and added a new “Who uses LibreOffice?” page.

The Document Liberation Project (DLP) is another effort supported by The Document Foundation, and we’ve been working to raise awareness about its mission and spread the word. After talking with the teams at Inkscape and Scribus, both of those projects now feature DLP logos on their sites, while the DLP front page has been redesigned to more succinctly express what it does. On top of that, we made a short video summarising the DLP’s goals.

Month of LibreOffice

Most recently, we had our Month of LibreOffice campaign, highlighting activity across all areas of the software. Many people think that contributing to an open source project is all about code – but we’ve tried to show that it’s possible to help in other ways including translations, confirming bugs, writing documentation and simply spreading the word. We awarded over 330 badges, so thanks to everyone who got one, and we plan to do another campaign like this later in the year.

The next few months are going to be busy as well: LibreOffice 5.2 is due to be released in August with many new features, and then we’ll have our yearly conference in September. We hope to see you there!

Behind the scenes at TDF: NLP/L10n

_SDS5526In the first quarter of 2016 a lot of work has been carried out by the community. On the events side, FOSDEM gathered several LibreOffice members together, and a face-to-face meeting with the Pootle team took place to discuss further fixes and enhancements needed by our localization team. In the meantime, the board has approved the budget to implement those features and work is being carried out.

The German community went to the DIDACTA trade fair (and wrote a nice report in several languages) and also attended the full week at CeBIT.

Guess what? Our Japanese team organized 14 events since the beginning of the year – kudos to them for their energy, enthusiasm and dedication! The Indian team also organized four LibreOffice meetups since January where localisation (l10n) for Garhwali and Angika began during one of them. One of the meetups was a hackathon dedicated to l10n QA. Don’t hesitate to subscribe to the LibreIndia Facebook page, as each event is announced and reported there.

And there is much more to report on what’s happening this quarter, like the hackfests in Ankara (Turkey) and the LibreOffice summit in the city of Las Palmas (Spain). I’m so happy to see the local communities invested in so many LibreOffice events.

Do not forget to check regularly the LibreOffice conference site; the local Czech team is working hard already to prepare everything, and a lot of information will be added to the site in the coming days. There is also the call for papers – submit your idea before July 15th!

On the l10n activities, LibreOffice 5.1 brought a bunch of modifications and several new strings to translate. But there will be more for the next release, 5.2, where a lot of help articles have been added or updated. A new project has been added to Pootle: now there is also the LibreOffice Online project, the newborn baby who has few new strings of its own.

Behind the Scenes at TDF: Marketing in Q1 2016

Before going into the details of the marketing activities, The Document Foundation (TDF) is proud to announce that Mike Saunders was hired as Marketing Assistant in mid-February. Mike is known for the famous feature videos he made for LibreOffice 5.1, and he has kicked off the Month of LibreOffice Contribution in May, amongst many other projects. Welcome on board, Mike!

italo-cecchignolaParaphrasing a famous song, marketing at The Document Foundation is “the activity that never sleeps…”. The first quarter is one of the busiest, as we have FOSDEM and one major release happening between the end of January and mid February, followed by several minor releases – to keep up momentum – and a few events, including CeBIT in Germany.

FOSDEM 2016 was a large success for LibreOffice, thanks to the strategic positioning of the booth in the main lobby of building K, where most of the large projects are represented. It was the first time for TDF in the “kernel” of the exhibition, and it showed up. On Saturday, January 30, the project registered more visitors than during the entire FOSDEM 2015 (and almost as many on Sunday, January 31).

On Saturday, January 30, TDF developers were also on stage during the Open Document Editors DevRoom, with over 20 different talks about the upcoming LibreOffice 5.1 release, and related topics.

During FOSDEM, the certification committee also hosted several sessions, mostly via remote connections, to evaluate five candidates: one from Spain: Pasqual Milvaques, one from Taiwan: Frankling Weng, and 3 from Italy: Paolo Dongilli, Antonio Faccioli and Diego Maniacco.

One week after FOSDEM, the marketing team coordinated the announcement of LibreOffice 5.1 on Wednesday, February 10. During Monday and Tuesday, I hosted five different conference calls, with journalists from Europe and the United States. Thanks to this effort, the media coverage has been excellent, with hundreds of positive articles worldwide.

One year later, the improvements to the distribution of press releases based on specific features of phpList are showing up, with an average hit rate often higher than 30%. Thanks to phpList for offering us a free account.

In March, the marketing team has focused on two different projects: CeBIT in Hannover, the largest high tech trade show in Germany, and TDF 2015 Annual Report, which is the most important document produced by the project during the year.

CeBIT is the largest German trade show, and most visitors already know the office suite or even use it. Even if the interest has been lower than in the past, several project members – Thorsten Behrens, Thomas Krumbein and myself – have had the opportunity to present LibreOffice at the open source area, thanks to the collaboration with CIB.

TDF’s 2015 Annual Report will be released during the second quarter, and will provide an overview of what has been achieved by the project at large – i.e. including native language projects at local level – in each area.

Starting from May, we will be organizing a monthly PR call, to involve native language communities in marketing activities. In addition, starting from July, we will also organize online webinars to share competences in marketing and communications with local communities. During these webinars, we will also cover the LibreOffice migration protocol.

Behind the scenes at TDF: infrastructure

alex-infraThe year 2015 brought some challenging and exciting developments regarding the ongoing restructuring of our infrastructure. At the beginning of the year, the migration of our existing virtual machines and bare metal machines was ongoing after an extensive test phase of the new virtualization platform.

This virtualization platform consists of three servers, each with 256GB RAM, 64 CPU cores and quite a lot of hard drive space. One of the machines is meant to be used exclusively by developers for crash testing. These machines are all hosted by manitu in St. Wendel, Germany, and are currently undergoing migration onto our own dedicated 42U rack – including the flexibility to set up a private network between these machines and others that we house there.

After some problems with the software previously chosen for our virtualization platform, much work went into setting up virtual machines where services run isolated from each other, based on plain KVM. This already led to the transition of the hosted blog to one of our own machines, which give us more control over installed plugins, and also provides more flexible control over the WordPress setup that we use.

During the Hackfest at the University of Gran Canaria, work went into making the used Salt States more easy to hack on by people who want to get involved in our infrastructure. This also resulted in a tutorial video on how to create a development environment for our infrastructure.

Monthly infra calls were also set up, taking place every last Wednesday of the month at 17:00 UTC. They resulted in the creation of a weekly maintenance window for server upgrades, reboots and major configuration changes, every Monday between 03:00 and 05:00 UTC.

Operating system upgrades

noun_215124_ccDuring the calls the community decided to upgrade the base operating system to Debian 8 over the next few months. This was already carried out on one of our virtualization hosts during the newly set up maintenance window, in order to check for any problems that may occur during the update. During the upgrade, some obstacles were identified and workarounds were set in place to allow smooth upgrades.

We have also invested in hardware from vendor Thomas Krenn which will allow us to set up two additional Windows buildbots with powerful dual CPUs and high speed SSDs, along with two more Linux buildbots with the same specs. These buildbots will also be housed in St. Wendel and connect to our growing intranet there. Two more servers will be used for backup space. We plan to connect all TDF-owned hardware with a VPN, forming a world-wide intranet.

In the second half of the year, more machines were migrated to Debian 8, including the two hypervisors still running Wheezy (Debian 7). Due to the huge success of the new build bots, two more were ordered and now extend the intranet, with a high performance cloud core router from Mikrotik becoming the central connection point of our intranet. The cloud core router also serves as a VPN provider for TDF members at areas with restricted internet access – such as the LibreOffice conference in Aarhus, Denmark in October.

As the number of new servers grew, we decided to migrate our monitoring platform to TKmon, running on a high-availability virtual machine that is separated from the rest of our infrastructure. TKmon integrates with the hardware vendor’s support and notifies them of hardware failures automatically. TKmon is open source software and uses tools such as icinga and pnp4nagios.

To be more flexible with the monitoring notifications, I wrote a tool called TMB that provides a bot for the Telegram chat service and sends notifications to admins. Development happened with PyCharm, a Python IDE.

Our server fleet

noun_203179_ccThe current state of the infrastructure consists of three rented hypervisors, each with four CPUs, 256 GB RAM, eight HDDs and partially SSDs. Additional rented servers include one backup server and one website stand-in host that was needed after the virtualization problems occurred at the beginning of the year, and that will be decommissioned soon. Nine housed servers with Intel SSDs and powerful dual CPUs are only reachable in the intranet, with access to them being controlled by the core router.

On the hypervisors, there are currently 31 VMs, providing services such as AskBot, WordPress, Gerrit, Bugzilla, Jenkins, MozTrap and much more. At Hetzner there are currently four servers: one that contains the Wiki, MirrorBrain and our public mailing lists, one that is for internal services, and two backup hosts – including one that provides storage capacity of over 17TB and is currently being set up.

Much of our documentation and many of our Salt States are published now at https://github.com/tdf/salt-states-base, while the compiled documentation can be found at http://salt-states-base.readthedocs.org/en/latest/. The Salt States are now tested with Travis and the build results are at https://travis-ci.org/tdf/salt-states-base. It is therefore now very easy to contribute to development and improve the documentation. Just fork the repository and create a pull request – then the results will automatically be tested in Travis. If you want to contribute to the infrastructure of our projects, you are invited to join our monthly infra calls, the next taking place on … or introduce yourself to the infra team in #tdf-infra on Freenode.