First steps with Sphinx

First steps with Sphinx and localization

Operating system: MacOS
Required software: Installed and working Homebrew package manager, installed and working Python 3.0
Required skill level: Comfortable using the command line, basic Python knowledge, strong Google Fu

Contents:

  1. Install and configure Sphinx
  2. Steps to localize documentation by using PoEdit or a Text Editor
  3. Possible troubles to encounter and how to solve them

Note: This posting isn’t entirely technical. When learning about a field you’ve never worked in before, including its entire range of applications, the learning curve is steep and strewn with error messages. The most important thing is, if you don’t know how to go on, ask someone who might:

  • Ask Google. You can copy entire lines of output from your Terminal window, and paste them in the search field.
  • Ask StackOverflow. It’s extremely likely that you find a posting about the issue you encounter, or at least relating to it.
  • Ask the application-specific forums. Be it Discourse, Slack, a traditional forum - there’s channels you can use to get support.

My intended usage for Sphinx was contributing to Open Source. The benefits:

  • You have to use git a lot.
  • You need to familiarize yourself with requirements the maintainers ask for contributions, e.g. existing style guides.

I followed the Official Sphinx Guide as well as the guide by Read The Docs. At the time of writing, both guides varied slightly. However, Sphinx have revised their docs since then. Your experience might vary.

Install and configure Sphinx

To begin with, I forked the git repository containing the documentation to translate. Existing documentation is not a requirement for working with Sphinx, though. You can of course start from scratch and create your own. To install Sphinx, you can either use Homebrew or Python. Via Homebrew: Open a Terminal window and enter brew install sphinx-doc
Via Python: Open a Terminal window and enter pip3 install sphinx
Note: Some guides still use pip install sphinx. Using this command with Python 3 will fail.

If you have existing docs to work on, you now switch to your docs' directory. Enter in Terminal cd path/to/your/docs where you replace the path/to/your/docs with the actual path of your docs. To create a new directory, enter in terminal
mkdir path/to/your/docs/ and again replace path/to/your/docs with your desired directory.

Sphinx comes with a script to ease basic configuration. Execute the script by entering in Terminal sphinx-quickstart, and answer the questions displayed in your Terminal window.
The values being created during quickstart are stored in Sphinx’s configuration file, conf.py. The file is located in your docs' directory, also referred to as your docs' root directory.
Before proceeding, I recommend checking your conf.py file for correct configuration. Specifically the following three variables and their value:

locale_dirs = ['locale/']  
language = 'de'  
html_search_language = 'de'

Setting locale_dirs to the above value ensures that Sphinx will pick up the files for translation automatically.
Note that the value of language and html_search_language should be set according to your target language. In the above example, de stands for German. Setting this variables will avoid error messages when building your translated documents (for more information, see section “Possible troubles to encounter and how to solve them”) You’re now ready to work with Sphinx.

Steps to localize documentation by using PoEdit or a Text Editor

Localization consists of several steps. Following is an overview:

  1. Sphinx searches your docs for text strings (also known as “messages”).
  2. The messages get collected in “catalog templates”.
  3. Catalog templates are then converted to “message catalogs”.
  4. Message catalogs are edited by the translator.
  5. The translated message catalogs are re-built in HTML.

Open a Terminal window and change to the directory your docs are located in. To begin searching your docs for translatable messages, enter in Terminal make-gettext
This generates the catalog templates, saved in *.pot format. The templates are stored under _build/gettext/. Next, transform the catalog templates into message catalogs you can edit. Enter in Terminal sphinx-intl update -p _build/gettext -l -de
Note that the -de part in above command again stands for German. Replace -de with your desired language code.
The message catalogs are stored as *.mo and *.po files, under the path /locale/de/LC_MESSAGES/, where /de/ will vary with your language code.
The LC_MESSAGES folder now contains subfolders for each section of your docs, according to the Table of Contents (or in short “ToC”). This means, every section’s pages have their own *.po file. An example folder structure could look like this:

locale
├── de_DE
│   └── LC_MESSAGES
│       └── index.po
          └── Installation
            └── how_to_install.po
          └── Usage
            └── usage.po
            └── troubleshooting.po

You can translate the *.po files in two different ways.
PoEdit is one way. It is convenient to use, because it displays only gettext’s extracted text strings on the left side of PoEdit’s interface, and your translations on the right side. PoEdit also suggests translations and style guide tips, which it draws from databases on the Internet. While this is very useful, the free version offers only a limited number of translation suggestions. Considering the size of my translation project, I didn’t want to invest in the Pro version. But if you do a lot of translation work, consider to purchase the Pro version of PoEdit.

Another way to translate is by using a simple text editor. Unlike PoEdit, the editor displays all content of a *.po file. This means, you need to look for the mapped text strings inside the *.po file yourself. Identify the mapped text strings by lines beginning with msgid "" and
msgstr "", like in the following example:

msgid ""
"Text strings in source language are located here."
"Please translate me."
msgstr ""
"Write your translated text strings here."
"Way to go."

Important: Do not omit the additional quotation marks at the beginning of every text string. Keep the structure as in the example above, as this enables Sphinx to pick up your translations automatically. This saves you a lot of work if you plan to add or change translations in the future.

Once the translation is done, re-build your docs from the *.po files. In Terminal, enter sphinx-build -b html . _build/html/de_DE, where you replace /de_DE in the path with your target target language. You find your re-built translated docs under _build/html/your_targetlanguage.

Possible troubles and how to solve them

When I executed make gettext the first time, it returned the following error:

Extension error:
Could not import extension sphinx_tabs.tabs (exception: No module named 'sphinx_tabs')
make: *** [gettext] Error 1

The extension “sphinx_tabs” was obviously missing. I added it by entering in Terminal pip3 install sphinx_tabs and executed make gettext again. This time, an exception occured:

Exception occurred:
  File "/Users/jas/tw_portfolio/jas-alnath.github.io/jas-alnath/docs/vlc-user/conf.py",  
    line 85, in setup
    app.add_javascript("js/version_switch.js")
AttributeError: 'Sphinx' object has no attribute 'add_javascript'

Some Googling later, I found the solution. In conf.py, line 85, the following entry causes the exception: app_add_javascript(„js/version_switch.js“)
I solved it by changing the line to app_add_js(„js/version_switch.js“)
This time, executing make gettext worked.

Rebuilding the docs also failed upon first execution, with the following error:

WARNING: while setting up extension conf.py: The app.add_stylesheet() is deprecated.  
Please use app.add_css_file() instead.
WARNING: The config value `release' has type `NoneType',  
defaults to `str'.
Exception occurred:
  File "/usr/local/lib/python3.9/site-packages/sphinx/builders/html/__init__.py",  
    line 502, in prepare_writing
    'release': return_codes_re.sub('', self.config.release),
TypeError: expected string or bytes-like object

A Google search revealed an issue opened at Sphinx’s official repo. Sphinx expects the release value to be of type String and no longer accepts type float. A workaround solution to this problem was to check the conf.py file for according entries. I found two corresponding variables, release and language set to a value of None. Changing both values to be of type String solved the error, and the build process worked.