61A Documentation¶

This app generates documentation for all 61A apps. If you need any help or have questions at any point during the process, please message Vanshaj!

Setup¶

Clone the repo and switch into the docs directory.
Create a new branch for your project using git checkout -b docs/app-name, where app-name is the name of the app you’re documenting (use hyphens if the name is multiple words). If someone has already created this branch, omit the -b flag.
Set up the Python environment using python3 -m venv env and env/bin/pip install -r requirements.txt, or simply sicp venv if you have sicp installed.
Run the Sphinx autobuilder using env/bin/sphinx-autobuild -b dirhtml .. _build
Visit http://localhost:8000 to see the docs.

Alternatively, to compile all docs once, run env/bin/sphinx-build -b dirhtml .. _build. This will generate an output folder _build containing the compiled documentation. Useful for when you don’t want to run the file watcher.

Writing Documentation¶

To write documentation for an app, we use a combination of reStructuredText and MyST. MyST allows us to write Sphinx documentation in a markdown equivalent of reStructuredText. The documentation for MyST should help you write the index and README files, but we’ll cover some examples and tips below anyway. The documentation for reStructuredText will help you write the inline Python documentation.

Text Editor Setup¶

First thing you should do is set up your text editor’s vertical ruler to let you know when you’re hitting an 80-character limit (this is a standard, but not a rule). In Visual Studio Code, this can be achieved by adding the following to your settings.json:

"editor.rulers": [
    {
        "column": 80,
        "color": "#555"
    },
]

Creating the README¶

Note

We will use MyST for the README.

Then, for whichever app you want to document, create a README.md under the directory for that app, and set it up like so:

# App Name

A brief description of what the app is meant to do.

## Setup

Include some steps to tell people how to develop this app locally.

## Other Sections

Include details that could help people develop or use the app.

Creating the Index¶

Note

We will use MyST for the index.

In order to place this app on the navbar, create an index.md under the same directory, and set it up like so:

```{include} README.md
```

## Code Segment 1

```{eval-rst}
.. automodule:: app_directory.module_name
    :members:
```

Where app_directory.module_name is the path to the file you’re including, such as common.db.

Code segments like the one at the end of the example will auto-include the code documentation for the various components of the app.

Documenting Code¶

Note

We will use reStructuredText for inline Python documentation.

To document a method in a Python file, format it like so:

def function(param1: str, param2: int = 2):
    """Description of the function.

    Can span multiple lines or paragraphs, if needed!

    :param param1: one line about the first parameter
    :type param1: str
    :param param2: one line about the second parameter
    :type param2: int

    :return: one line about the return value (including type)
    """
    # function body is here

Where str and int should be replaced with the actual type of the parameter.

Note

The type annotations go in two places: the function dignature, as well as the body of the docstring.

This will result in the following rendered documentation:

function(param1: str, param2: int = 2)¶

Description of the function.

Can span multiple lines or paragraphs, if needed!

Parameters

param1 (str) – one line about the first parameter
param2 (int) – one line about the second parameter

Returns

one line about the return value (including type)

Documenting `rpc` Methods¶

You do not have to write documentation for methods that are bound to RPC. For example, if you’re documenting howamidoing, when you get to upload_grades, you can simply write the following (see emphasized line):

@rpc_upload_grades.bind(app)
@only("grade-display", allow_staging=True)
def upload_grades(data: str):
    """See :func:`~common.rpc.howamidoing.upload_grades`."""
    with transaction_db() as db:
        set_grades(data, get_course(), db)

Linking to Other Documentation¶

If you mention another function, method, or class, please include a link to the documentation for such. If this is in a MyST file, you can do this as follows:

{func}`common.shell_utils.sh`
{meth}`common.hash_utils.HashState.update`
{class}`~subprocess.Popen`

This will appear as common.shell_utils.sh(), common.hash_utils.HashState.update(), and Popen.

If this is in a Python file (using rST), you can do this as follows:

:func:`common.shell_utils.sh`
:meth:`~common.hash_utils.HashState.update`
:class:`subprocess.Popen`

This will appear as common.shell_utils.sh(), update(), and subprocess.Popen.

Note

Per the examples above, if you insert a ~ before the path to the documentation, Sphinx will render the link using only the name of the object itself, and will drop the path itself. This is desirable for cleanliness, so use this whenever linking to a document.

If you want to refer to something that is documented outside of this project, the Python docs, and the Flask docs, message Vanshaj with what you’re trying to document, as well as a link to the documentation for the relevant project. He will then add it to intersphinx_mapping dictionary in the configuration, so that you can link to it as you would anything else. As an example, to link to Flask’s redirect function, you can use {func}`~flask.redirect` . This will render as redirect().

Full Example¶

Sample README¶

Here’s what the common README file looks like:

# `common`

This is a collection of utilities used by multiple apps in our ecosystem.

Sample Index¶

Here’s what the common index file looks like:

```{include} README.md
```

## CLI Utilities

This file contains miscellaneous utilities for command-line interfaces.

```{eval-rst}
.. automodule:: common.cli_utils
    :members:
```

## Course Configuration

This file contains various methods that can help identify a course as well as
determine the level of access a user (logged in with Okpy) has to the course.

```{eval-rst}
.. automodule:: common.course_config
    :members:
```

## Database Interface

This file contains the logic for interfacing with the 61A database backend,
or for interfacing with a local database if you don't have access to the
production database (or are developing something volatile).

By default, this code assumes that you are running locally without access to
the production database. As such, the default action is to create a local
`app.db` if an application requests to use a database. In production, the
environment variable `DATABASE_URL` is used to connect to the production
database.

To develop on the production database, use `ENV=DEV_ON_PROD` and pass in
`DATABASE_PW=password`, where `password` is the SQL password for the
`buildserver` user. Ask a Head of Software if you need access to this.

```{note}
Developing on production requires the Cloud SQL Proxy. Follow the instructions
at https://cloud.google.com/sql/docs/mysql/sql-proxy to install the proxy
in the root directory of the repository.
```

```{eval-rst}
.. automodule:: common.db
    :members:
```

## Hash Utilities

This file contains some utilities for hashing data.

```{eval-rst}
.. automodule:: common.hash_utils
    :members:
```

## HTML Helpers

This file contains some helpful HTML formatting tools for a standard frontend.

```{caution}
Do **not** use this library for student-facing apps, as it is vulnerable to XSS.
Only use it for quick staff-only frontends.
```

```{eval-rst}
.. automodule:: common.html
    :members:
```

## Job Routing

This file contains a decorator utility to add URL rules for recurring actions.

```{eval-rst}
.. automodule:: common.jobs
    :members:
```

## OAuth Client

This file contains some utilities for Okpy OAuth communication.

```{eval-rst}
.. automodule:: common.oauth_client
    :members:
```

## Secrets

This file contains some utilities to create/get secrets for an app.

```{eval-rst}
.. automodule:: common.secrets
    :members:
```

## Shell Utilities

This file contains some utilities to communicate with a shell.

```{eval-rst}
.. automodule:: common.shell_utils
    :members:
```

## `url_for`

This file creates a new `url_for` method to improve upon the default
{func}`~flask.url_for`.

```{eval-rst}
.. automodule:: common.url_for
    :members:
```

```{toctree}
:hidden:
:maxdepth: 3

rpc/index
```

Sample Code Documentation¶

Here’s what the common.db.connect_db() docs look like:

@contextmanager
def connect_db(*, retries=3):
    """Create a context that uses a connection to the current database.

    :param retries: the number of times to try connecting to the database
    :type retries: int

    :yields: a function with parameters ``(query: str, args: List[str] = [])``,
        where the ``query_str`` should use ``%s`` to represent sequential
        arguments in the ``args_list``

    :example usage:
    .. code-block:: python

        with connect_db() as db:
            db("INSERT INTO animals VALUES %s", ["cat"])
            output = db("SELECT * FROM animals")
    """
    if is_sphinx:

        def no_op(*args, **kwargs):
            class NoOp:
                fetchone = lambda *args: None
                fetchall = lambda *args: None

            return NoOp()

        yield no_op
    else:
        for i in range(retries):
            try:
                conn = engine.connect()
                break
            except:
                sleep(3)
                continue
        else:
            raise

        with conn:

            def db(query: str, args: List[str] = []):
                if use_devdb:
                    query = query.replace("%s", "?")
                return conn.execute(query, args)

            yield db

Here’s what the common.shell_utils.sh() docs look like:

def sh(
    *args,
    env={},
    capture_output=False,
    stream_output=False,
    quiet=False,
    shell=False,
    cwd=None,
    inherit_env=True,
):
    """Run a command on the command-line and optionally return output.

    :param args: a variable number of arguments representing the command to
        pass into :class:`~subprocess.Popen` or :func:`~subprocess.run`
    :type args: *str

    :param env: environment variables to set up the command environment with
    :type env: dict

    :param capture_output: a flag to return output from the command; uses
        :class:`~subprocess.Popen`
    :type capture_output: bool

    :param stream_output: a flag to stream output from the command; uses
        :func:`~subprocess.run`
    :type stream_output: bool

    :param quiet: a flag to run the command quietly; suppressed printed output
    :type quiet: bool

    :param shell: a flag to run the command in a full shell environment
    :type shell: bool

    :param cwd: the working directory to run the command in; current directory
        is used if omitted
    :type cwd: str

    :param inherit_env: a flag to include :obj:`os.environ` in the environment;
        ``True`` by default
    :type inherit_env: bool

    .. warning::
        Only one of ``capture_output`` and ``stream_output`` can be ``True``.
    """
    assert not (
        capture_output and stream_output
    ), "Cannot both capture and stream output"

    if shell:
        args = [" ".join(args)]

    if inherit_env:
        env = {**os.environ, **env, "ENV": "dev"}

    if stream_output:
        out = subprocess.Popen(
            args,
            env=env,
            stdout=subprocess.PIPE,
            stderr=subprocess.STDOUT,
            universal_newlines=True,
            bufsize=1,
            shell=shell,
            cwd=cwd,
        )

        def generator():
            while True:
                line = out.stdout.readline()
                yield line
                returncode = out.poll()
                if returncode is not None:
                    if returncode != 0:
                        # This exception will not be passed to the RPC handler,
                        # so we need to handle it ourselves
                        raise subprocess.CalledProcessError(returncode, args, "", "")
                    else:
                        return ""

        return generator()
    elif capture_output:
        out = subprocess.run(
            args, env=env, capture_output=capture_output, shell=shell, cwd=cwd
        )
    else:
        out = subprocess.run(
            args, env=env, stdout=subprocess.PIPE, shell=shell, cwd=cwd
        )
    if capture_output and not quiet:
        print(out.stdout.decode("utf-8"), file=sys.stdout)
        print(out.stderr.decode("utf-8"), file=sys.stderr)
    out.check_returncode()
    return out.stdout

61A Documentation¶

Setup¶

Writing Documentation¶

Text Editor Setup¶

Creating the README¶

Creating the Index¶

Documenting Code¶

Documenting rpc Methods¶

Linking to Other Documentation¶

Full Example¶

Sample README¶

Sample Index¶

Sample Code Documentation¶

Documenting `rpc` Methods¶