What is the Open Document Format (ODF), and how is it developed?

Regina Henschel is a long-time member of the LibreOffice community, and has worked on ODF, the native file format of the suite. At our recent German community meetup, we talked to her about how ODF is developed, and how users can help to improve it…

Tell us a bit about ODF…

Open Document Format is LibreOffice’s native file format. (If you have a file with a .odt, .ods, .odp or .odg extension, then it’s an Open Document Text, Spreadsheet or Presentation file or Graphic respectively.)

ODF is developed by OASIS, then submitted to ISO (the International Organization for Standardization), and then adopted as a standard. There is also a working group at ISO, which by the way also works on OOXML – which can then ask questions about development, and so on.

For ODF we are now working on version 1.3. We had a “feature freeze” last summer. We have come so far that everything we wanted to have in it is available in the “editor version”. Now we’re going to fine-tune it, then we’ll be back in summer – so that was a whole year. Then comes the coordination process at OASIS, so it usually takes two years until a new version of the standard is ready.

How do you decide which new features to add?

The feature has to be well defined, and that depends a little on how good the proposal you get is. There are some suggestions like: “We want to have a feature for Fourier transformations” – but nothing else! And then we say: as long as there is no implementation that actively handles it, we’ll postpone it for later. It’s not worth it, for a standards body to deal with a proposal like this, if there are no applications that are actually interested in it.

ODF exists as a “strict” standard, for features that are fully standardized, and then there is an “extended mode” in which you can try out new features in applications. They then run under their own namespace, and if that works, organizations can make a suggestion for OASIS, eg: “This feature works well for us, so please include this in the strict standard.”

Then others can say: “Yes, we want to do something similar, but it would be better for us if we didn’t call it X, but Y instead”. Or: “We need an additional attribute to make it work for us”, for example. In situations like that, the proposals are discussed.

For example, in LibreOffice, it’s now possible to specify the size of charts, independent of their labels. In the old standard, you could only set the size including the label. This means that when data changed and the label changed, the effect was that the size of the chart changed. So that’s been fixed in ODF 1.3.

There are also bugfixes, where there are things that are simply wrong, or that are not clear enough for implementation. For example, one thing that was really incorrect was to put the units “at” and “atm” on the same level – that was definitely wrong. And one time, a matrix form was wrong as well. So we work on fixes for these.

Can normal end-users help out?

Yes, you can be a “normal person”, so to speak, without being on any committee – you can still send something to the Technical Committee. There is a mailing list, and also a homepage, where you can get informed about the work of the committee. There you can also find a link to the mailing list – but you have to register, because of spam. Messages on this list are then read and answered by the Committee.

The second way to get involved is when the Committee is about to make a new version of ODF. This will then be made available for voting – and then there will be an official time for comments. There the regulations are a bit stricter; this is not voluntary, but the committee has to actually work on these comments. That should then give a qualified answer.

The next step where official participation takes place is at ISO. They have the same disclosure process again, where comments are collected accordingly. Then the Technical Committee gets a long list from ISO: “We have received these comments, please discuss them.”

So this is a process that is very open. That’s one of the reasons why I decided to join the Committee. It’s not that something happens in secret, and then suddenly there’s a standard – but rather, the process is totally comprehensible.

The other thing that interested people can see is the archives of the mailing lists – and there are weekly teleconferences too.

Thanks to Regina for all her help over the years! ODF – and other open standards – are incredibly important, especially for long-term data storage. Indeed, the UK government recommends using ODF, and has guidelines for using the format in organisations and companies. Also check out this presentation from Regina at FOSDEM 2018.

LibreOffice 6.3 on Linux, a statement

Following the availability of LibreOffice 6.3 Beta, there have been speculations about 32-bit compatibility based on a the missing 32-bit binaries for Linux.

We have prepared a short and a long statement to clarify the situation.

TL;DR

  1. The Document Foundation is ending the provision of 32-bit binaries, and NOT 32-bit compatibility as a whole.

  2. Distro vendors or anyone running a more current 32-bit Linux system can still create 32-bit versions of LibreOffice, as developers have not in any way removed 32-bit compatibility from the source code. Additionally, we are not removing any 32-bit builds that were previously created.

  3. Most Linux users are sourcing LibreOffice from their distro repositories, which are usually compiled against the distro’s version of the various external libraries. We do not anticipate distros dropping 32-bit LibreOffice packages.

  4. TDF does not anticipate the same decision happening for LibreOffice 32-bit binaries for Windows any time soon.

LONG

  1. During the last two years, the number of downloads of the 32-bit Linux distribution-neutral binaries provided by The Document Foundation have decreased to a very low number. Today, the time needed to compile, test, maintain and distribute those binaries is not worth the effort, based on current download numbers. So, TDF is ending the provision of 32-bit binaries, and NOT 32-bit compatibility as a whole.

  2. Today, 32-bit packages are very much the domain of specific Linux distros rather than a general user need. So, we are leaving them to distros, who will upstream fixes. Indeed, distro vendors or anyone running a more current 32-bit Linux system can still create 32-bit versions of LibreOffice, as developers have not in any way removed 32-bit compatibility from the source code. Additionally, TDF is not removing any 32-bit binaries that were previously created.

  3. Most Linux users are sourcing LibreOffice from their distro repositories, which are usually compiled against the distro’s version of the various external libraries. LibreOffice by itself ships a number of external components to avoid dependencies, while distros link against the versions of those components which are part of the distro anyway. TDF does not anticipate distros dropping 32-bit LibreOffice packages.

  4. The Document Foundation does not anticipate the same decision happening for Windows 32-bit binaries any time soon. Of course, if downloads of Windows 32-bit binaries from TDF mirror servers drop to the same very low number as Linux 32-bit packages, TDF will reconsider the situation.

OASIS announces the ODF Advocacy Open Project

The ODF Advocacy Open Project we have pre-announced at FOSDEM is now a reality. Yesterday, OASIS has released the following press release, which is just the first step of a new sustained activity focused on supporting the adoption of ODF – the only true standard document format available on the market – by governments, public administrations and enterprises worldwide, to increase interoperability (and thus knowledge sharing), reduce hidden costs associated to document management, and get rid of vendor lock-in.

OASIS Introduces Open Projects Program to Bridge Open Source and Standards Development

AirBus, CIB, Fujitsu, IBM, Red Hat, Siemens, Software AG, The Document Foundation, and others sponsor Open Projects

Boston, May 8, 2019 – OASIS, a global nonprofit consortium, today announced the launch of Open Projects, the first-of-its-kind program that creates a more transparent and collaborative future for open source and standards development. Open Projects gives communities the power to develop what they choose–APIs, code, specifications, reference implementations, guidelines– in one place, under open source licenses, with a path to recognition in global policy and procurement.

The lines between open source and open standards have been blurring for some time, and communities in both arenas have been calling for more flexibility and options for collaboration. Open Projects is a new approach that addresses the need for change in everything from handling IP to governance and decision-making, from funding to establishing trust and assuring quality.

“With Open Projects, we’re building a movement to transform the open source and standards world,” said Gershon Janssen, Chairman, OASIS Board of Directors. “We want to dissolve the barriers that separate communities. We want to empower groups with more control and streamlined governance. We want to support projects by giving them all the process they need–and not a bit more–so they can accomplish great things fast.”

Open Projects builds on the OASIS experience and reputation for producing quality work that’s been trusted and supported by governments and industries worldwide for more than 25 years.

“For many, open source has become a means of establishing de facto software standards. However, de facto standards are not recognized by many governments and institutions,” said Chris Ferris, IBM Fellow and CTO Open Tech for IBM. “OASIS Open Projects provides an important new opportunity to leverage the rapid innovation of open source in the process of developing open standards. The potential to achieve ISO, IEC, or ITU standards approval is a huge value for many important open source initiatives.” Ferris, who also holds a leadership position on the Hyperledger Fabric project, played an instrumental role in defining the OASIS Open Projects program and now serves on its Advisory Council.

The Open Projects program is being advanced by some of the most accomplished, regarded minds in open source today.

As part of the program announcement, OASIS is launching the first two Open Projects – Open Services for Lifecycle Collaboration (OSLC) and OpenDocument Format (ODF) Advocacy.

The OSLC Open Project advances a suite of standard REST APIs to connect data and achieve the digital thread across domains, applications, and organizations. It is sponsored by AirBus, Austrian Institute of Technology, Bank of America, Boeing, Dassault, Fujitsu, IBM, Koneksys, KTH Royal Institute of Technology, Red Hat, Siemens, Software AG, and Tasktop.

“OSLC helps create standard REST APIs that solve industry integration challenges,” said Andrew Berezovskyi of the KTH Royal Institute of Technology. “Open Projects enables the OSLC community to produce deliverables that meet requirements from various stakeholders without being hindered by the weight of foundation bureaucracy or the baggage that comes with financial, legal, technical, and marketing administration.”

The ODF Advocacy Open Project promotes the world’s leading document standard. After being approved as an OASIS Standard, ODF was recognized by ISO/IEC and endorsed by governments around the world as a way to ensure permanent access to data and eliminate the risk of vendor lock-in. The ODF Advocacy Open Project is sponsored by CIB and The Document Foundation.

“ODF guarantees perennial access to data that can be transferred in a transparent way between different apps, computers and operating systems, getting rid of hidden interoperability costs, vendor lock-in issues and license fees,” said Italo Vignoli, Co-Founder, The Document Foundation.

Additional Open Projects for blockchain and other areas will be announced in the coming months. Further details about OASIS Open Projects are available here or email info@oasis-open-projects.org.

MITRE names The Document Foundation as a CVE Numbering Authority (CNA)

Berlin, March 15, 2019 – MITRE announced that The Document Foundation, the home of LibreOffice, has been approved as CVE Numbering Authority (CNA). The Document Foundation is at the center of one of the largest free open source software ecosystems, where enterprise sponsored developers and contributors work side by side with volunteers coming from every continent. The nomination is the result of significant investments in security provided by the LibreOffice Red Hat team under Caolán McNamara leadership.

What is CVE?

Common Vulnerabilities and Exposures (CVE) is a reference list of public cybersecurity vulnerabilities, with entries that describe those vulnerabilities and provide references for them. These references are often used as the vulnerability names, especially in security updates. To date, LibreOffice has a track record of rapid response to all reported threats.

What is a CVE Numbering Authority (CNA)?

A CNA is an organization that can assign and announce CVE entries within a particular scope. Some CNAs are organizations providing CVEs for their products such as The Document Foundation.

How will The Document Foundation assign CVEs?

The Document Foundation Security Team provides a forum for all of the vendors and individuals who contribute to LibreOffice development to co-ordinate the work of protecting our users from threats related to the application.

As a CNA, The Document Foundation Security Team now has the ability to assign CVE IDs to vulnerabilities affecting our products, the ability to control the disclosure of vulnerability information without pre-publishing, and notification of vulnerabilities in our products by researchers who request a CVE ID from us.

LibreOffice Conference 2018: More presentation videos

The next batch of videos from our conference in Tirana is online. (Use headphones for the best audio.)

First, Simon Phipps talks about the 20th anniversary of the Open Source Initiative:

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

Then Heiko Tietze provides some news from the LibreOffice design team:

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

Finally, Caolan McNamara describes some updates to the native GTK UI:

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

Many more videos still to come!

Interview: Guilhem Moulin on LibreOffice infrastructure and services

A large free and open source software project like LibreOffice requires a lot of infrastructure, to support our users, developers and worldwide community. Today we speak to Guilhem Moulin, who is in charge of TDF’s infrastructure and services, about new developments and how others can get involved…

To start, please give us a quick overview of TDF’s public infrastructure.

The public infrastructure is powered by about 50 Kernel-based Virtual Machines (KVM) spread across 4 hypervisors plugged to an internal 10Gbps switch and hosted at Manitu in St. Wendel (Germany), and managed with libvirt and its KVM/QEMU driver. The virtual disk images are typically stored in GlusterFS volumes — distributed across the hypervisors — except for some transient disks (such as cache) where the IOPS need is higher and the redundancy less important.

All our public VMs run Debian GNU/Linux (currently a mix of Jessie — which are to be upgraded — and Stretch), each typically hosting a single service for better isolation. The rest of the stack is fairly usual: systemd as PID 1 & service manager, a mix of MySQL and PostgreSQL as RDMS, and nginx as SSL/TLS endpoint & reverse HTTP proxy. All of this is orchestrated and managed using saltstack.

About half of our Virtual Machines host public-facing websites; the other half are used for test instances, various production backends and internal services, as well as for tinderboxes and other hacking VMs. Some of these websites are mostly useful for developers, such our Bugzilla or gerrit instances — an overview of the development-focused sites can be found at https://devcentral.libreoffice.org. The remaining sites include the main LibreOffice website, the download page, the Wiki, Askbot, and of course the blog.

Beside these VMs, we also operate a handful of other machines for backups, monitoring, and mail systems, which are hosted offsite for obvious reasons.

What have been the most significant infra developments in the last six months?

Single Sign On (SSO) is probably what’s been the most visible to the community. Traditionally each frontend (Wiki, Bugzilla, Askbot, etc.) has its own private authentication backend, so once someone sign in to multiple services, they would have to remember multiple sets of credentials, which is cumbersome and makes password & email rotation difficult.

We now have a central authentication system (which uses an LDAP DIT as backend), but aren’t pointing individual services to it, as it would 1/ expose the shared credentials to all services hence increase the attack surface; and 2/ doesn’t solve the fact that users would have to enter their password to each service individually. Instead we’re deploying a solution using the SAML 2.0 protocol: unauthenticated users are redirected to an authentication portal against which they can authenticate, and they are redirected to the protected page afterwards.

Not all services have been migrated to SSO yet. An issue is that we have to unify accounts (people use different usernames in different services); and while we want a “critical mass” of active user accounts in LDAP before migrating a service, it’s been rather difficult to reach out to people — even among TDF officials! — and convince them to create an account in the new system. Fortunately since we migrated the authentication system of our wiki, more and more people (among whom a lot of dormant accounts, probably spammers unfortunately) started using the new system.

While it’s only visible to infra team members, we also replaced our Graphite (+ Carbon + Icinga2) based monitoring system with Prometheus (+ data exporters + alert manager). Furthermore, still on the monitoring front but public this time, we just deployed a new service, CachetHQ, to show a quick overview of TDF’s infra status:
https://status.documentfoundation.org.

Last but not least, earlier this spring we were fairly busy with GDPR compliance.

What are you working on at the moment, and what are your plans for the next six months?

Aside from daily maintenance and occasional emergencies (system crashes, hardware hiccups, performances issues, etc.), infra team members still spends quite a lot of time on the above, as this is not completely finished yet. Projects for next year include working on a better backup solution, in particular regarding database snapshots. The data collection system for download metrics needs some improvement, too.

Finally, what cool things can new volunteer admins do to get involved and help the project?

We have a wide variety of systems, ranging for highly sensitive (election, internal mail, LDAP DIT, whitebox monitoring) to pretty much fully public beside the access logs (bitergia dashboard, blackbox monitoring). We can’t give upfront access to the sensitive side of the spectrum to everyone, but there are things to help with on the other side too (developer-focused services are typically less sensitive, since development is open anyway).

Sometimes we also start fresh and replace a service with something equivalent on a brand new box; in that case there is no sensitive data at stake, and it’s a great way for new volunteer admins to gain trust. I mentioned the monitoring migration earlier; we could also imagine replacing our ageing MirrorBrain deployment with a more modern solution like Mirrorbits, for instance.

Thanks to Guilhem for his time and help. If you’re interested in joining our infra community and gaining valuable experience in a large FOSS project, see here to get started!