Tender to implement Curl based HTTP/WebDAV UCP (#202104-01)

Note: for questions asked about this tender and their respective answers, please see the bottom of this page


We are extending the application deadline.

The deadline for questions stays as in the original tender: June 15, 2021
The deadline for applications has been extended to: June 24, 2021


The Document Foundation (TDF) is the charitable entity behind the world’s leading free/libre/open source (FLOSS) office suite LibreOffice.

We are looking for an individual or company to implement Curl based HTTP/WebDAV UCP.

The work has to be developed on LibreOffice master, so that it will be released in the next major version.

The task consists of addressing two problems. All of the mentioned features and requirements are a mandatory part of this tender and therefore have to be part of the bid. This tender does not contain any optional items.

Problem description #1 – Currently we need to bundle crypto libraries

  • TDF releases of LibreOffice bundle both OpenSSL and NSS, but both libraries have a high number of security issues.
  • On macOS and Windows, neither OpenSSL nor NSS integrate with the crytographic APIs supplied by the operating system, so they will use a bundled hard-coded set of trusted certificate authorities (CAs), that is different from what the operating system itself would trust. This hard-coded set of trusted certificates is also not user-modifiable.
  • Additionally, OpenSSL cannot ever be used from the system because it has no application binary interface (ABI).

Problem description #2 – Currently we duplicate and use different HTTP/WebDAV UCPs

  • One which is used by everybody, including TDF releases, using a bundled Neon WebDAV library. This requires OpenSSL and cannot be used on a hypothetical Apple iOS port.
  • Another (for a hypothetical Apple iOS port) using a bundled Serf library. This requires OpenSSL.
    • Serf does not actually support WebDAV directly, only HTTP, so the UCP itself implements the additional WebDAV protocol features.
    • It is complicated to build, as it drags in two other bundled external libraries.
    • Additionally, this cannot be upgraded to a current version without introducing a new build dependency on the “scons” tool.
  • TDF releases also bundle the Curl library.
    • This can can do HTTP, likely similar to Serf.
    • Also, it can use native operating system cryptographic APIs and trusted certificate authorities (CAs) on Windows, macOS and Linux.
    • It can be used on Apple iOS without problems. (iOS deliverables are not part of this tender.)

The solution we seek, and as such the scope of this tender, is to implement a HTTP/WebDAV UCP with Curl, possibly based on code from the Serf UCP, to solve these issues, by getting rid of four bundled external libraries and one hard OpenSSL dependency. Besides addressing the above issues, the new Curl-based implementation needs to be at least as functionally complete as the existing Neon-based one.

All technology standards of relevance, as well as their targeted versions for this tender should be declared or defined in the offer’s description of implementation (e.g. name and version of the cryptographic API on the respective operating systems).

A key item of the deliverables for this tender, and therefore also a decision criteria – besides qualification, references, price, and completeness of fullfilment – is extensive documentation about the approach chosen to implement the above items, covering more than just the pure implementation. We expect bidders to provide documentation on both the code and the non-code parts of this tender, e.g. methodology, structure and technical aspects.

The Document Foundation will publish this under a free and open source license and make it available to the general public. Another criteria for the evaluation of the bids will be the description of the required test activities and the delivery of (automated) tests supporting work items for the described tender implementation or feature specification.

Required skills

  • Extensive knowledge of C++
    Experience working on the LibreOffice source code

Other skills

  • English (conversationally fluent in order to coordinate and plan with members of TDF)

We use free, libre and open source (FLOSS) software for development wherever possible, and the resulting work must be licensed under the Mozilla Public License v2.0.

TDF welcomes applications from all suitably qualified persons regardless of their race, sex, disability, religion/belief, sexual orientation or age.

Bidders will get a preference for including a partner or independent developer who has not been involved in a successful tender before. For such developers, who have not yet been part of a successful tender bid, we aim on a best-effort basis, but without any guarantees whatsoever, to provide some mentoring in understanding the code base and the process in contributing to the code. We expect that time and efforts on the bidder’s side for this should not be part of the paid work for this tender. Please mention such need of LibreOffice development mentoring in your offer.

As always, TDF will give some preference to individuals who have previously shown a commitment to TDF, including but not limited to certified developers and/or members of TDF. Not being a member, or never having contributed before, does not exclude any applicants from consideration.

The task offered is a project-based one-off, with no immediate plans to a mid- or long-term contractual relationship. It is offered on a freelance, project basis. Individuals and companies applying can be located anywhere in the world.

When budgeting, we anticipated that this project (all items combined) to take in the region of 40 days of work. Should bidders’ assessment result in a significantly different number, please reach out to us before sending your bid, so we can clarify upfront.

TDF is looking forward to receiving your applications for the aforementioned tasks, your offer in form of a fixed-time, fixed-budget approach, and the duration period for the implementation in calendar weeks after the final awarding of the tender, via e-mail to a committee at tender20210401@documentfoundation.org no later than May 31, 2021.

Applicants who have not received feedback by June 30, 2021 should consider that their application, after careful review, was not accepted.

All bidders are invited to ask their questions on this tender until June 15, 2021. Questions can be sent informally to the above e-mail address, and answers will be made public in a collected and anonymized form.


We received the following question:

“getting rid of four bundled external libraries” – Could you please clarify what these four libraries are?

Answer: the text only indirectly mentions that serf “is complicated to build, as it drags in two other bundled external libraries”, which are apr and its dependency apr_util (both in external/apr/). So the “four bundled external libraries” would be apr, apr_util, neon, and serf. (There are still other dependencies on openssl, like ExternalProject_python3 and ExternalProject_xmlsec, so we will not get rid of it through this tender.)


We received the following question:

Could you please expand a bit on:

1. the meaning of ‘extensive documentation’, e.g. number of pages or words, and level of detail (classes, functions, line-by-line) that would meet the acceptance criteria?

2. the target audience (e.g. certified LibreOffice developers, or more general?) and purpose of the documentation (e.g. teaching, reference, new implementations, maintenance), as well as form (document format) and acceptable publishing location (e.g. inline, separate-but-within-the-code-base, wiki, or elsewhere)?

Answer: There is no fixed criteria for the documentation. Our goal is to share the knowledge about the approach chosen to address the problem and/or implement the feature, in order to make that information available to the general public.

The target audience is a suitably skilled developer. As such, industry-standard inline documentation in the code, targetting experienced and/or certified LibreOffice developers, plus documenting any non-obvious design choices in an accompanying README file, would be sufficient.

LibreOffice Macro Team: progress report

Macros help users to automate common tasks in LibreOffice. In September 2019 we announced a new team in our community to work on macro support. The last progress report was published in April 2020, so it is high time to look into what has happened since then.

If you are interested in contributing to the macro team (development, testing or documentation), we’d love to hear from you – please send an email to ilmari.lauhakangas@libreoffice.org and we’ll get in touch.

ScriptForge Libraries

The biggest single event was the introduction of ScriptForge Libraries in LibreOffice 7.1. ScriptForge and its documentation is a collaboration betwen Jean-Pierre Ledure, Alain Romedenne and Rafael Lima. You can read more about it in the January 2021 blog post and the work-in-progress Help content.

Wiki docs

Nathan Ullberg continued working on Impress macro articles.

Celia Palacios improved the Python guide and added new macro tutorials, such as populating spreadsheets with data from an SQL database.

Alain Romedenne continued adding syntax diagrams and improved and expanded the Python guide and macro articles.

Mauricio Baeza improved and expanded articles and added new ones, such as Insert a comment with custom presets, Copy content cell from Spreadsheet to other and Charts in Calc.

Steve Fanning added several new examples of Calc macros.

Code contributions from macro team members

Alain Romedenne:

Andreas Heinisch:

George Bateman:

Tomoyuki Kubota:

Code contributions from honorary associate members

Compatibility fixes for Python 3.8 to 3.12 done by David Ostrovsky, Dante Doménech, Noel Grandin (Collabora) and Stephan Bergmann (Red Hat).

Anshu Khare:

Arnaud Versini:

  • Many cleanups and optimisations in Basic handling code

Arpit Bandejiya:

Caolán McNamara (Red Hat):

John Turpish:

Maxim Monastirsky:

Michael Stahl (allotropia):

Mike Kaganski (Collabora):

Noel Grandin (Collabora):

Serge Krot (CIB):

Shubham Jain:

Stephan Bergmann (Red Hat):

Tushar Kumar Rai:

Xisco Fauli (TDF):

Help content

Improved by Alain Romedenne:

Added by Alain Romedenne:

Improved by Rafael Lima:

Improved by Olivier Hallot (TDF):

Added by Olivier Hallot (TDF):

LibreOffice in the Google Summer of Code 2021

New features in LibreOffice are made by volunteers, certified developers, and – during the summer – participants in the Google Summer of Code programme. This is focused on introducing students to open source software development, and last year LibreOffice received a bunch of new features thanks to the work of several students.

Well, we’re happy to announce that LibreOffice is part of this year’s Summer of Code (GSoC). If you’re a student, want to improve your programming skills and receive a financial stipend to implement new features in LibreOffice, take a look. Get in contact with us, show us that you’ve learnt the basics by working on an Easy Hack, and then propose your project(s). We look forward to meeting you!

Click here to get started

And to learn more about GSoC, check out this interview with Gautam Prajapati, who was part of the programme a few years ago:

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

Update on tender for a built-in UNO object inspection tool in LibreOffice

In July last year, we launched a tender to implement a dedicated, built-in UNO object inspection tool in LibreOffice. UNO refers to Unified Network Objects, the component model used by the software.

Tomaž Vajngerl was assigned to work on the tender, and has blogged about his progress. He discusses the point-and-click functionality to inspect selected objects in the document, and his next steps.

If you’re interested in the technology “under the hood” in LibreOffice, check it out!

[White Paper] LibreOffice Technology, the only software platform for personal productivity on the desktop, mobile and cloud

The most mature and capable code-base, outside of Microsoft, to interoperate with Microsoft’s proprietary document formats

LibreOffice Technology is the result of ten years of intensive activity on the software’s open source code, coordinated by the Engineering Steering Committee and carried out by developers, software engineers, security experts, and interface and user experience specialists of many affiliations.

The goal of this evolutionary process was to create a single software platform for individual productivity on desktop, mobile and the cloud: the only approach able to offer users the interoperability features that enable transparent sharing of all content, and independence from single commercial vendors and vendor lock-in strategies.

This is the opposite approach to all other proprietary and open core office suites, which have developed different versions for each platform trying to replicate the functionality, but only succeeding in part, so that – for example – the internal structure of documents (which is not visible to users) is different for each application.

The evolutionary process from product to platform

The source code inherited from OpenOffice – with a heritage stretching back decades – was too complex for the average developer and had a build environment that was difficult to create and manage. It was therefore essential to mentor new developers, to reduce the learning curve and speed up the process.

To help newcomers, senior developers invented “Easy Hacks”: simple tasks for people with little or no knowledge of the source code, with the twofold objective of easing the learning curve, and having a number of problems solved without spending their precious time.

Thanks to Easy Hacks, the project managed to attract new code contributors, create a strong base of new developers, increase the number of core developers, and also involve several new “star” contributors capable of working on challenging code cleaning and refactoring tasks.

In a short time, under the stewardship of LibreOffice’s Engineering Steering Committee, developers managed to reduce the software footprint, undertake a long awaited source code renovation with the removal of tens of thousands of lines of dead code (and a large number of deprecated libraries), simplify the build process to the point where it could be automated and translate the majority of German comments into English.

To support the process, the development infrastructure was enhanced with the integration of several components and the update or replacement of existing ones. Today, the process is based on automation – as much as possible – including tests, to free developers’ time for more demanding tasks.

The result is a source code base characterised by one of the lowest defect densities in the industry, according to static analysis tools such as Coverity Scan [1]. The density inherited from OpenOffice – 1.1 errors per 1000 lines of code – has dropped to 0.001, against an average of 0.71 for projects of the same size.

Overall performance was also significantly improved, especially when reading and writing large files in the ODF native document format and in DOCX, XLSX and PPTX proprietary file formats.

Once the bulk of the source code renovation process was completed, developers and UI designers worked on improving the user experience. They reorganized several menus, created new icon sets for Windows and macOS, improved existing icon sets, and extended the user interface options by adding a NotebookBar, with Tabbed, Grouped and Contextual variants.

Last but not least: LibreOffice – thanks to a huge community of volunteers active in localization – is the software for personal productivity available in more native language versions than any other application. Today, the software is released in 119 language versions, with 26 more in development.

New LibreOffice APIs and Scripting Libraries

LibreOffice’s APIs (Application Programming Interfaces) are known for their steep learning curve, which was considered an obstacle for the development of macros within LibreOffice, and for integration projects on mobile and cloud platforms. To solve this issue, developers have created the more accessible LibreOffice Kit APIs and ScriptForge scripting libraries.

LibreOfficeKit is a thin API that provides access to LibreOffice functionality via C/C++ or writing external apps, without having to compile or link to LibreOffice, or learn UNO – the office suite component model – or other complex languages. The Kit allows fast rendering of documents for any application.

ScriptForge is an extensible and robust collection of macro scripting resources for LibreOffice, which can be invoked from user Basic or Python scripts. They help to overcome the LibreOffice API’s challenges, as they offer easy access to and management of windows and documents, automation of Calc sheets, cells and ranges of cells, management of dialogs and their controls, as well as access to data contained in databases.

LibreOffice, a champion of interoperability

LibreOffice is the most versatile office suite in the market for interoperability, thanks to its outstanding level of compatibility with the OOXML Transitional proprietary file format – the majority of DOCX, XLSX and PPTX files – and to the large number of import and export filters for legacy file formats released by the Document Liberation Project [2].

Given the large number of proprietary Microsoft Office files generated by users, they are managed by teams of developers with specific skills.

Microsoft Office file are analyzed to spot issues and “roundtripped”: opened and edited with LibreOffice, and saved back in the original format to check for consistency, until all issues are solved and the result is the same, independent of the software used (the image shows the process for XLSX files, but the same applies to DOCX and PPTX).

Interoperability with proprietary Microsoft Office formats is further improved with every new LibreOffice release, to ensure transparent handling of documents independent to the format: standard ODF or proprietary OOXML.

Open Document Format, the true document standard

LibreOffice Technology is also the best platform for true interoperability thanks to the ISO-standard ODF (Open Document Format), the native file format inherited from OpenOffice. ODF is maintained and further developed thanks to The Document Foundation, which has established the independent COSM [3] project – the Community of ODF Specification Maintainers – to hold funds and retain editors for contributing to standardisation of ODF 1.3 and other future versions.

The ODF standard document format was designed in a vendor neutral manner from the ground-up, using existing standards wherever possible, to achieve a level of interoperability which is still unrivaled. ODF is robust and solid, and is consistent and predictable (i.e. the underlying XML code of the document will always be the same, independent to the program version, the hardware platform and the operating system, and this will make the document easy to visualize and manage on any device).

Products based on LibreOffice Technology

Today, there is a large family of products based on LibreOffice Technology, for the desktop (with community and enterprise-optimized versions), for the cloud, for Android and iOS, and for Chrome OS. LibreOffice Technology is also used within larger products to convert and process document formats. These products are released by different organizations and have different brand names, but share the same common engine to offer users the same unique advantages in term of interoperability, resiliency, robustness and security.

[1] https://scan.coverity.com/
[2] https://www.documentliberation.org/
[3] https://blog.documentfoundation.org/blog/2019/07/02/the-cosm-project/

LibreOffice Technology White Paper is also available as PDF: Download File

Next set of videos from the openSUSE + LibreOffice Conference 2020

Note: these videos are also available on PeerTube

It’s time for another batch of presentations and workshops from the recent openSUSE + LibreOffice Conference 2020! You can see them in the YouTube playlist, and here are the individual videos (apologies for the not-perfect audio in some places):

Marketing and social media in LibreOffice (Mike Saunders):

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

 

openSUSE + LibreOffice Conference 2020: Recruiting for LibreOffice globally and locally through volunteer platforms (Ilmari Lauhakangas):

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

 

The history and pre-history of LibreOffice (Michael Meeks):

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

 

LibreOffice oss-fuzz, crashtesting, coverity (Caolán McNamara):

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

 

Faster Jail Creation with Bind-Mount (Ashod Nakashian):

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

 

The ODF TC GitHub (Svante Schubert):

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

 

LibreOffice in your browser – WebAssembly & other neat hacks to make that happen (Thorsten Behrens):

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

 

The ODF Toolkit (Svante Schubert):

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

 

Implementation Detail (Stephan Bergmann):

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.