Update on tender for a built-in UNO object inspection tool in LibreOffice

In July last year, we launched a tender to implement a dedicated, built-in UNO object inspection tool in LibreOffice. UNO refers to Unified Network Objects, the component model used by the software.

Tomaž Vajngerl was assigned to work on the tender, and has blogged about his progress. He discusses the point-and-click functionality to inspect selected objects in the document, and his next steps.

If you’re interested in the technology “under the hood” in LibreOffice, check it out!

[White Paper] LibreOffice Technology, the only software platform for personal productivity on the desktop, mobile and cloud

The most mature and capable code-base, outside of Microsoft, to interoperate with Microsoft’s proprietary document formats

LibreOffice Technology is the result of ten years of intensive activity on the software’s open source code, coordinated by the Engineering Steering Committee and carried out by developers, software engineers, security experts, and interface and user experience specialists of many affiliations.

The goal of this evolutionary process was to create a single software platform for individual productivity on desktop, mobile and the cloud: the only approach able to offer users the interoperability features that enable transparent sharing of all content, and independence from single commercial vendors and vendor lock-in strategies.

This is the opposite approach to all other proprietary and open core office suites, which have developed different versions for each platform trying to replicate the functionality, but only succeeding in part, so that – for example – the internal structure of documents (which is not visible to users) is different for each application.

The evolutionary process from product to platform

The source code inherited from OpenOffice – with a heritage stretching back decades – was too complex for the average developer and had a build environment that was difficult to create and manage. It was therefore essential to mentor new developers, to reduce the learning curve and speed up the process.

To help newcomers, senior developers invented “Easy Hacks”: simple tasks for people with little or no knowledge of the source code, with the twofold objective of easing the learning curve, and having a number of problems solved without spending their precious time.

Thanks to Easy Hacks, the project managed to attract new code contributors, create a strong base of new developers, increase the number of core developers, and also involve several new “star” contributors capable of working on challenging code cleaning and refactoring tasks.

In a short time, under the stewardship of LibreOffice’s Engineering Steering Committee, developers managed to reduce the software footprint, undertake a long awaited source code renovation with the removal of tens of thousands of lines of dead code (and a large number of deprecated libraries), simplify the build process to the point where it could be automated and translate the majority of German comments into English.

To support the process, the development infrastructure was enhanced with the integration of several components and the update or replacement of existing ones. Today, the process is based on automation – as much as possible – including tests, to free developers’ time for more demanding tasks.

The result is a source code base characterised by one of the lowest defect densities in the industry, according to static analysis tools such as Coverity Scan [1]. The density inherited from OpenOffice – 1.1 errors per 1000 lines of code – has dropped to 0.001, against an average of 0.71 for projects of the same size.

Overall performance was also significantly improved, especially when reading and writing large files in the ODF native document format and in DOCX, XLSX and PPTX proprietary file formats.

Once the bulk of the source code renovation process was completed, developers and UI designers worked on improving the user experience. They reorganized several menus, created new icon sets for Windows and macOS, improved existing icon sets, and extended the user interface options by adding a NotebookBar, with Tabbed, Grouped and Contextual variants.

Last but not least: LibreOffice – thanks to a huge community of volunteers active in localization – is the software for personal productivity available in more native language versions than any other application. Today, the software is released in 119 language versions, with 26 more in development.

New LibreOffice APIs and Scripting Libraries

LibreOffice’s APIs (Application Programming Interfaces) are known for their steep learning curve, which was considered an obstacle for the development of macros within LibreOffice, and for integration projects on mobile and cloud platforms. To solve this issue, developers have created the more accessible LibreOffice Kit APIs and ScriptForge scripting libraries.

LibreOfficeKit is a thin API that provides access to LibreOffice functionality via C/C++ or writing external apps, without having to compile or link to LibreOffice, or learn UNO – the office suite component model – or other complex languages. The Kit allows fast rendering of documents for any application.

ScriptForge is an extensible and robust collection of macro scripting resources for LibreOffice, which can be invoked from user Basic or Python scripts. They help to overcome the LibreOffice API’s challenges, as they offer easy access to and management of windows and documents, automation of Calc sheets, cells and ranges of cells, management of dialogs and their controls, as well as access to data contained in databases.

LibreOffice, a champion of interoperability

LibreOffice is the most versatile office suite in the market for interoperability, thanks to its outstanding level of compatibility with the OOXML Transitional proprietary file format – the majority of DOCX, XLSX and PPTX files – and to the large number of import and export filters for legacy file formats released by the Document Liberation Project [2].

Given the large number of proprietary Microsoft Office files generated by users, they are managed by teams of developers with specific skills.

Microsoft Office file are analyzed to spot issues and “roundtripped”: opened and edited with LibreOffice, and saved back in the original format to check for consistency, until all issues are solved and the result is the same, independent of the software used (the image shows the process for XLSX files, but the same applies to DOCX and PPTX).

Interoperability with proprietary Microsoft Office formats is further improved with every new LibreOffice release, to ensure transparent handling of documents independent to the format: standard ODF or proprietary OOXML.

Open Document Format, the true document standard

LibreOffice Technology is also the best platform for true interoperability thanks to the ISO-standard ODF (Open Document Format), the native file format inherited from OpenOffice. ODF is maintained and further developed thanks to The Document Foundation, which has established the independent COSM [3] project – the Community of ODF Specification Maintainers – to hold funds and retain editors for contributing to standardisation of ODF 1.3 and other future versions.

The ODF standard document format was designed in a vendor neutral manner from the ground-up, using existing standards wherever possible, to achieve a level of interoperability which is still unrivaled. ODF is robust and solid, and is consistent and predictable (i.e. the underlying XML code of the document will always be the same, independent to the program version, the hardware platform and the operating system, and this will make the document easy to visualize and manage on any device).

Products based on LibreOffice Technology

Today, there is a large family of products based on LibreOffice Technology, for the desktop (with community and enterprise-optimized versions), for the cloud, for Android and iOS, and for Chrome OS. LibreOffice Technology is also used within larger products to convert and process document formats. These products are released by different organizations and have different brand names, but share the same common engine to offer users the same unique advantages in term of interoperability, resiliency, robustness and security.

[1] https://scan.coverity.com/
[2] https://www.documentliberation.org/
[3] https://blog.documentfoundation.org/blog/2019/07/02/the-cosm-project/

LibreOffice Technology White Paper is also available as PDF: Download File

Next set of videos from the openSUSE + LibreOffice Conference 2020

Note: these videos are also available on PeerTube

It’s time for another batch of presentations and workshops from the recent openSUSE + LibreOffice Conference 2020! You can see them in the YouTube playlist, and here are the individual videos (apologies for the not-perfect audio in some places):

Marketing and social media in LibreOffice (Mike Saunders):

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

 

openSUSE + LibreOffice Conference 2020: Recruiting for LibreOffice globally and locally through volunteer platforms (Ilmari Lauhakangas):

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

 

The history and pre-history of LibreOffice (Michael Meeks):

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

 

LibreOffice oss-fuzz, crashtesting, coverity (Caolán McNamara):

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

 

Faster Jail Creation with Bind-Mount (Ashod Nakashian):

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

 

The ODF TC GitHub (Svante Schubert):

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

 

LibreOffice in your browser – WebAssembly & other neat hacks to make that happen (Thorsten Behrens):

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

 

The ODF Toolkit (Svante Schubert):

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

 

Implementation Detail (Stephan Bergmann):

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

Tender to finish transition of LibreOffice to ODF 1.3 (ODF 1.3 delta) (#202010-01)

The Document Foundation (TDF) is the charitable entity behind the world’s leading free/libre open source (FLOSS) office suite LibreOffice.

We are looking for an individual or company to finish transition of LibreOffice to ODF 1.3 (ODF 1.3 delta).

This tender builds on the previous ODF 1.3 tender and aims to implement additional features.

The work has to be developed on LibreOffice master, so that it will be released in the next major version.


The following required features (section A) need to be implemented:

  • chart:data-label-series. Missing feature. It is needed for import from Excel.
    • Relevant bugs in TDF’s Bugzilla: #94235, #133176
    • OASIS reference: OFFICE-2117
  • chart:regression-moving-type. Implementation of types “center” and “average-abscissa” is missing. It is needed for interoperability with Gnumeric.
    • For this feature, there is existing code that can be extended.
    • Relevant bug in TDF’s Bugzilla: #133423
    • OASIS reference: OFFICE-3959
  • <text:index-entry-link-start> and <text:index-entry-link-end> in user-index. The link marks exist, but the function itself is not implemented.
    • For this feature, there is existing code that can be extended.
    • Relevant bug in TDF’s Bugzilla: #121842
    • OASIS reference: OFFICE-3941

The following are desirable features (section B):

  • draw:fill for background of pages. Attribute draw:background-size specifies whether a background fill covers the entire page or only the content area of the page. It belongs to element <style:drawing-page-properties>. ODF 1.3 has extended its use to all kind of pages. Some related bug reports have been set to “fixed”, but the problem is not completely solved, details in the bug report.
    • The respective attribute seems to get written by LibreOffice already. All Writer documents are now the entire page.
    • Relevant bug in TDF’s Bugzilla: #134734
    • OASIS reference: OFFICE-3937
  • draw:z-index more precise with increase from back to front. Problem needs to be solved too for cases when converting from docx to odt.
    • Relevant bug in TDF’s Bugzilla: #133487
    • OASIS reference: OFFICE-2122

The following features are nice to have (section C):

  • Wrong icon of master document template in Windows Explorer
    • Relevant bug in TDF’s Bugzilla: #133285
    • OASIS reference: OFFICE-2580
  • pivot table based on named range with local scope
    • The previous implementation didn’t work out. This is a rather complex task.
    • Relevant bug in TDF’s Bugzilla: #37268
    • OASIS reference: OFFICE-3665

The following features are purely optional (section D):

  • Adapt function wizard to the fact, that the second parameter of DCOUNT and DCOUNTA may be empty (i.e. optional)
    • This can be qualified as an “EasyHack”.
    • OASIS reference: OFFICE-3906
  • chart:coordinate-region. There is no help about this topic. There exists no option to not use this kind of position and size reference.
    • That feature was handled differently until OpenOffice.org 3.0, where the coordinates included the description, now they are without, i.e. the reference changed.
    • OASIS reference: OFFICE-3928
  • fo:min-height as attribute of draw:text-box. LibreOffice does not use that attribute and ignores it on file open. Missing feature.
    • That element seems to be written by Word, but likely not yet by LibreOffice.
    • OASIS reference: OFFICE-3735

The following feature is purely optional, and not trivial from a UX point of view (section E):

  • svg:stroke-linecap at object style vs draw:style in element. Implementation of draw:style is incomplete. Especially there is no UI to define a line style with round dashes.
    • Relevant bugs in TDF’s Bugzilla: #133499 (Implementation has error), #127509, #127348, #127266, #123349, #53276, #127207
    • OASIS reference: OFFICE-3742

Required skills

  • Extensive knowledge of C++
  • Experience working on the LibreOffice source code
  • Knowledge of the OpenDocument Format standard, particularly in version 1.3.

Other skills

  • English (conversationally fluent in order to coordinate and plan with members of TDF)

We use free, libre and open source (FLOSS) software for development wherever possible, and the resulting work must be licensed under the Mozilla Public License v2.0.

TDF welcomes applications from all suitably qualified persons regardless of their race, sex, disability, religion/belief, sexual orientation or age.

Bidders will get a preference for including a partner or independent developer who has not been involved in a successful tender before.

As always, TDF will give some preference to individuals who have previously shown a commitment to TDF, including but not limited to certified developers and/or members of TDF. Not being a member, or never having contributed before, does not exclude any applicants from consideration.

The task offered is a project-based one-off, with no plans to a mid- or long-term contractual relationship. It is offered on a freelance, project basis. Individuals and companies applying can be located anywhere in the world.

When budgeting, we anticipated that this project (sections A-E) to take in the region of 32 days of work.

TDF is looking forward to receiving your applications for one or more of the aforementioned tasks, your financial expectations and the earliest date of your availability, via e-mail to a committee at tender20201001@documentfoundation.org no later than November 10, 2020.

Applicants who have not received feedback by December 8, 2020 should consider that their application, after careful review, was not accepted.

LibreOffice and Google Summer of Code 2020: The results

Google Summer of Code logo

This year, LibreOffice was once again a mentoring organization in the Google Summer of Code (GSoC) a global programme focused on bringing more student developers into free and open source software development. We ran six projects – and all were finished successfully. Students and mentors enjoyed the time, and here we present some of the achievements, which should make their way into LibreOffice 7.1 in early February 2021!

You can experiment with the new functionality by using daily builds and report any problems in our bug tracker.


Styles Inspector for Writer by Shivam Kumar Singh

Mentors: Tomaž Vajngerl, Mikhail Kaganskiy (Collabora)

Dealing with styles and formatting in complex documents can become tedious, especially when you are working on something you did not create yourself. The Styles Inspector implemented by Shivam conveniently displays all the properties of the elements making up a document. It will surely become an essential tool for Writer experts.

Learn more about the Styles Inspector in the final report.

Styles Inspector


Additions – Tight integration of extensions by Yusuf Keten

Mentor: Muhammet Kara (Collabora)

Thanks to the work of Yusuf, users are now able to fetch extensions, templates and other resources as well as discover guide books without ever leaving LibreOffice.

Learn more about Additions in the final report.

Additions


Extending the UI testing framework by Ahmed ElShreif

Mentor: Markus Mohrhard

A domain-specific language (DSL) for LibreOffice’s Python-based UI testing framework was originally implemented by Saurav Chirania in 2018. Ahmed ElShreif continued the work in 2019 and now tackled further improvements in the DSL and in the testing framework itself. This means that the automated quality assurance system is better at preventing bugs from slipping into LibreOffice.

Learn more about the UI testing project in the final report.


Impress shape animations with a real physics engine by Sarper Akdemir

Mentor: Thorsten Behrens (CIB)

Sarper added the ability to enrich presentations with animations powered by the physics simulator engine Box2D.

Please confirm that you want to play a YouTube video. By accepting, you will be accessing content from YouTube, a service provided by an external third party.

YouTube privacy policy

If you accept this notice, your choice will be saved and the page will refresh.

Learn more about the physics engine project in the final report.


Moving the gallery code to a friendly format by Aditya Sahu

Mentors: Tomaž Vajngerl, Michael Meeks (Collabora)

It was not simple to work on galleries due to them being stored in a custom binary format. Now Aditya got us out of this unfortunate situation and designers will have a much easier time.

Learn more about the gallery project in the final report.


Blurry shadows by Ahmad Ganzouri

Mentors: Tomaž Vajngerl, Miklos Vajna (Collabora)

Shapes and objects in LibreOffice used to only support hard shadows. Now Ahmad implemented proper blurriness for the shadows, supporting both ODF and OOXML formats.

Blurry shadows

Learn more about blurry shadows in the final report.


Wrapping up

Many thanks to all students who spent their summer time improving LibreOffice. You are awesome! And special thanks also to the mentors who always put so much love and energy into these tasks. That‘s what makes LibreOffice rock.

Now we are looking forward to next year’s GsoC. If you are interested, why not prepare early? Learn more at out wiki page where some ideas are listed. Participating in GSoC is a great way to build your skills, and show future employers what you’re capable of!

Tender for implementing support for a dedicated, built-in UNO object inspection tool in LibreOffice (#202007-02)

The Document Foundation (TDF) is the charitable entity behind the world’s leading free/libre open source (FLOSS) office suite LibreOffice.

We are looking for an individual or company to implement support for a dedicated, built-in UNO object inspection tool in LibreOffice, to start work as soon as possible.

In order to make working with UNO objects easier and to avoid the need to always install extensions before debugging, it is necessary to be able to inspect UNO objects in a running LibreOffice instance effectively.

This task involves reading the existing Basic IDE Watch code, evaluating how it can be improved based on ideas implemented in external tools like xray and MRI and extending the Watch code to be a first-class inspector that allows focusing the relevant part of the UNO API for opened documents and also based on your current selection (similar to what is possible in web browsers).

A good part of the features are implemented already. Work carried out under this tender will therefore mostly consist in making the features more accessible and more stable, adjusting the UI and refactoring things.

The work has to be developed on LibreOffice master, so it can get released in the next major version.


The following required features need to be implemented:

  • Dockable toolbar that can appear at the bottom of a document frame, similar to find-bar.
  • Left-hand side of the toolbar exposes a snapshot of a useful subset of the DOM as a tree view: Writer paragraphs, Calc sheets, Impress slides
    • This tree widget should populate its content on-demand whenever possible in order to ensure performance.
  • Point&click inspect mode (similar to F12 in Chrome): combine Help -> What’s this (point on something & then an action) and normal selection, so it’s possible to point on something (e.g. an image), make it the current selection and automatically launch DevTools on it. Perhaps rename ‘DevTools’ to something else.

    Note that this is mostly UI work, ThisComponent.CurrentController.Selection in Basic already gives access to the current selection in a not-so-convenient way.

  • Right-hand side: show details in a table about the current selected item in the “DOM tree-view”, which is implemented as part of the watch window in the Basic IDE:
    • object’s UNO properties
    • object methods
    • supported services and interfaces
  • User documentation for the new dialog is mandatory.
  • Brief developer documentation for the newly introduced classes is required.
  • Whenever adding new functionality, it should be considered if it’s possible to test the functionality with automated tests with reasonable amount of effort.

The following are optional features:

  • Configuration support: remember which tabpage was open last time (properties, methods, etc.)
    • Remember sorting settings (prioritize paragraphs/sheets/slides and other relevant properties or sort alphabetically)
  • Click on value for details: primitive types
    • This is useful if the user selected lots of text for inspection, we can’t show all the content in a table cell, but can if a multi-line edit replaces the table widget.
  • Click on value for details: re-launch DevTools on a sub-object on the right-hand side. This is already handled to a large degree in the existing watch variable code, which represents the object already as a tree.
    • This allows recursion: double-click on a value with a complex type, which has its own properties, methods, etc -> inspect it.
  • Show implementation name of object
    • If a consistent name is provided by the object, this can allow jumping to the relevant C++ code from DevTools easily.
  • Copy&paste support:
    • Normally content of a table in a widget is not easily copy&paste-able. It can help debugging if add explicit support to copy the table content still, e.g. all property names and their primitive values.
  • Improved presentation of the DOM for Writer/Calc/Impress:
    • The inspector tool could be just a generic presenter for any UNO object, but in practice macro and extension authors are interested in a subset of the extremely rich and generic API we provide.
    • The idea is to select a few key properties for each component, so it’s trivial for the user to see how to access the most important details of a document:
      • Writer: style families, paragraph list, text portion list, etc.
      • Calc: sheets, columns, cells, named ranges, etc.
      • Impress: slides, shapes, etc.

Required skills

  • Extensive knowledge of C++
  • Experience working on the LibreOffice source code
  • Knowledge of UNO

Other skills

  • English (conversationally fluent in order to coordinate and plan with members of TDF)

We use free, libre and open source (FLOSS) software for development wherever possible, and the resulting work must be licensed under the Mozilla Public License v2.0.

TDF welcomes applications from all suitably qualified persons regardless of their race, sex, disability, religion/belief, sexual orientation or age.

Bidders will get a preference for including a partner or independent developer who has not been involved in a successful tender before.

As always, TDF will give some preference to individuals who have previously shown a commitment to TDF, including but not limited to certified developers and/or members of TDF. Not being a member, or never having contributed before, does not exclude any applicants from consideration.
Further discussions on this tender took place on the public board discussion mailing list of The Document Foundation.

The task offered is a project-based one-off, with no immediate plans to a mid- or long-term contractual relationship. It is offered on a freelance, project basis. Individuals and companies applying can be located anywhere in the world.

When budgeting, we anticipated that this project (incl. optional tasks) to take in the region of 60 days of work. Bidders are free to bid on the required features only. Bidders who bid on both the required and optional features are asked to provide a breakdown in terms of costs for each.

TDF is looking forward to receiving your applications for one or more of the aforementioned tasks, your financial expectations and the earliest date of your availability, via e-mail to a committee at tender20200702@documentfoundation.org no later than September 1, 2020.

Applicants who have not received feedback by October 15, 2020 should consider that their application, after careful review, was not accepted.