Developing a «TODO» extension¶
This section is intended as a walkthrough for the creation of custom extensions. It covers the basics of writing and activating an extension, as well as commonly used features of extensions.
As an example, we will cover a «todo» extension that adds capabilities to include todo entries in the documentation, and to collect these in a central place. (A similar «todo» extension is distributed with Sphinx.)
Extension Design¶
Nota
To understand the design this extension, refer to Important objects and Build Phases.
We want the extension to add the following to Sphinx:
- A «todo» directive, containing some content that is marked with «TODO», and only shown in the output if a new config value is set. (Todo entries should not be in the output by default.)
- A «todolist» directive that creates a list of all todo entries throughout the documentation.
For that, we will need to add the following elements to Sphinx:
- New directives, called
todo
andtodolist
. - New document tree nodes to represent these directives, conventionally also
called
todo
andtodolist
. We wouldn’t need new nodes if the new directives only produced some content representable by existing nodes. - A new config value
todo_include_todos
(config value names should start with the extension name, in order to stay unique) that controls whether todo entries make it into the output. - New event handlers: one for the
doctree-resolved
event, to replace the todo and todolist nodes, and one forenv-purge-doc
(the reason for that will be covered later).
The Setup Function¶
The new elements are added in the extension’s setup function. Let us create a
new Python module called todo.py
and add the setup function:
def setup(app):
app.add_config_value('todo_include_todos', False, 'html')
app.add_node(todolist)
app.add_node(todo,
html=(visit_todo_node, depart_todo_node),
latex=(visit_todo_node, depart_todo_node),
text=(visit_todo_node, depart_todo_node))
app.add_directive('todo', TodoDirective)
app.add_directive('todolist', TodolistDirective)
app.connect('doctree-resolved', process_todo_nodes)
app.connect('env-purge-doc', purge_todos)
return {'version': '0.1'} # identifies the version of our extension
The calls in this function refer to classes and functions not yet written. What the individual calls do is the following:
add_config_value()
lets Sphinx know that it should recognize the new config valuetodo_include_todos
, whose default value should beFalse
(this also tells Sphinx that it is a boolean value).If the third argument was
'html'
, HTML documents would be full rebuild if the config value changed its value. This is needed for config values that influence reading (build phase 1).add_node()
adds a new node class to the build system. It also can specify visitor functions for each supported output format. These visitor functions are needed when the new nodes stay until phase 4 – since thetodolist
node is always replaced in phase 3, it doesn’t need any.We need to create the two node classes
todo
andtodolist
later.add_directive()
adds a new directive, given by name and class.The handler functions are created later.
Finally,
connect()
adds an event handler to the event whose name is given by the first argument. The event handler function is called with several arguments which are documented with the event.
The Node Classes¶
Let’s start with the node classes:
from docutils import nodes
class todo(nodes.Admonition, nodes.Element):
pass
class todolist(nodes.General, nodes.Element):
pass
def visit_todo_node(self, node):
self.visit_admonition(node)
def depart_todo_node(self, node):
self.depart_admonition(node)
Node classes usually don’t have to do anything except inherit from the standard
docutils classes defined in docutils.nodes
. todo
inherits from
Admonition
because it should be handled like a note or warning, todolist
is just a «general» node.
The Directive Classes¶
A directive class is a class deriving usually from
docutils.parsers.rst.Directive
. The directive interface is also
covered in detail in the docutils documentation; the important thing is that
the class should have attributes that configure the allowed markup,
and a run
method that returns a list of nodes.
The todolist
directive is quite simple:
from docutils.parsers.rst import Directive
class TodolistDirective(Directive):
def run(self):
return [todolist('')]
An instance of our todolist
node class is created and returned. The
todolist directive has neither content nor arguments that need to be handled.
The todo
directive function looks like this:
from sphinx.locale import _
class TodoDirective(Directive):
# this enables content in the directive
has_content = True
def run(self):
env = self.state.document.settings.env
targetid = "todo-%d" % env.new_serialno('todo')
targetnode = nodes.target('', '', ids=[targetid])
todo_node = todo('\n'.join(self.content))
todo_node += nodes.title(_('Todo'), _('Todo'))
self.state.nested_parse(self.content, self.content_offset, todo_node)
if not hasattr(env, 'todo_all_todos'):
env.todo_all_todos = []
env.todo_all_todos.append({
'docname': env.docname,
'lineno': self.lineno,
'todo': todo_node.deepcopy(),
'target': targetnode,
})
return [targetnode, todo_node]
Several important things are covered here. First, as you can see, you can refer
to the build environment instance using self.state.document.settings.env
.
Then, to act as a link target (from the todolist), the todo directive needs to
return a target node in addition to the todo node. The target ID (in HTML, this
will be the anchor name) is generated by using env.new_serialno
which
returns a new unique integer on each call and therefore leads to unique target
names. The target node is instantiated without any text (the first two
arguments).
On creating admonition node, the content body of the directive are parsed using
self.state.nested_parse
. The first argument gives the content body, and
the second one gives content offset. The third argument gives the parent node
of parsed result, in our case the todo
node.
Then, the todo node is added to the environment. This is needed to be able to
create a list of all todo entries throughout the documentation, in the place
where the author puts a todolist
directive. For this case, the environment
attribute todo_all_todos
is used (again, the name should be unique, so it is
prefixed by the extension name). It does not exist when a new environment is
created, so the directive must check and create it if necessary. Various
information about the todo entry’s location are stored along with a copy of the
node.
In the last line, the nodes that should be put into the doctree are returned: the target node and the admonition node.
The node structure that the directive returns looks like this:
+--------------------+
| target node |
+--------------------+
+--------------------+
| todo node |
+--------------------+
\__+--------------------+
| admonition title |
+--------------------+
| paragraph |
+--------------------+
| ... |
+--------------------+
The Event Handlers¶
Finally, let’s look at the event handlers. First, the one for the
env-purge-doc
event:
def purge_todos(app, env, docname):
if not hasattr(env, 'todo_all_todos'):
return
env.todo_all_todos = [todo for todo in env.todo_all_todos
if todo['docname'] != docname]
Since we store information from source files in the environment, which is
persistent, it may become out of date when the source file changes. Therefore,
before each source file is read, the environment’s records of it are cleared,
and the env-purge-doc
event gives extensions a chance to do the same.
Here we clear out all todos whose docname matches the given one from the
todo_all_todos
list. If there are todos left in the document, they will be
added again during parsing.
The other handler belongs to the doctree-resolved
event. This event is
emitted at the end of phase 3 and allows custom resolving
to be done:
def process_todo_nodes(app, doctree, fromdocname):
if not app.config.todo_include_todos:
for node in doctree.traverse(todo):
node.parent.remove(node)
# Replace all todolist nodes with a list of the collected todos.
# Augment each todo with a backlink to the original location.
env = app.builder.env
for node in doctree.traverse(todolist):
if not app.config.todo_include_todos:
node.replace_self([])
continue
content = []
for todo_info in env.todo_all_todos:
para = nodes.paragraph()
filename = env.doc2path(todo_info['docname'], base=None)
description = (
_('(The original entry is located in %s, line %d and can be found ') %
(filename, todo_info['lineno']))
para += nodes.Text(description, description)
# Create a reference
newnode = nodes.reference('', '')
innernode = nodes.emphasis(_('here'), _('here'))
newnode['refdocname'] = todo_info['docname']
newnode['refuri'] = app.builder.get_relative_uri(
fromdocname, todo_info['docname'])
newnode['refuri'] += '#' + todo_info['target']['refid']
newnode.append(innernode)
para += newnode
para += nodes.Text('.)', '.)')
# Insert into the todolist
content.append(todo_info['todo'])
content.append(para)
node.replace_self(content)
It is a bit more involved. If our new «todo_include_todos» config value is false, all todo and todolist nodes are removed from the documents.
If not, todo nodes just stay where and how they are. Todolist nodes are
replaced by a list of todo entries, complete with backlinks to the location
where they come from. The list items are composed of the nodes from the todo
entry and docutils nodes created on the fly: a paragraph for each entry,
containing text that gives the location, and a link (reference node containing
an italic node) with the backreference. The reference URI is built by
app.builder.get_relative_uri
which creates a suitable URI depending on the
used builder, and appending the todo node’s (the target’s) ID as the anchor
name.