Grading Container Image#

When you use Otter Generate to create the configuration zip file for Gradescope, Otter includes the following software and packages for installation. The zip archive, when unzipped, has the following contents:

autograder
├── environment.yml
├── files/
├── otter_config.json
├── requirements.r
├── run_autograder
├── run_otter.py
├── setup.sh
└── tests/

Note that for pure-Python assignments, requirements.r is not included and all of the R-pertinent portions of setup.sh are removed. Below are descriptions of each of the items listed above and the Jinja2 templates used to create them (if applicable).

`setup.sh`#

This file, required by Gradescope, performs the installation of necessary software for the autograder to run.

The template for Python assignments is:

#!/usr/bin/env bash

export DEBIAN_FRONTEND=noninteractive
apt-get clean
apt-get update
apt-get install -y wget texlive-xetex texlive-fonts-recommended texlive-plain-generic \
    build-essential libcurl4-gnutls-dev libxml2-dev libssl-dev libgit2-dev texlive-lang-chinese

# install pandoc
wget -nv https://github.com/jgm/pandoc/releases/download/3.1.11.1/pandoc-3.1.11.1-1-amd64.deb \
    -O /tmp/pandoc.deb
dpkg -i /tmp/pandoc.deb

# install mamba
if [ $(uname -p) = "arm" ] || [ $(uname -p) = "aarch64" ] ; \
    then wget -nv https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-aarch64.sh \
        -O /autograder/source/mamba_install.sh ; \
    else wget -nv https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh \
        -O /autograder/source/mamba_install.sh ; \
fi
chmod +x /autograder/source/mamba_install.sh
/autograder/source/mamba_install.sh -b
echo "export PATH=/root/miniforge3/bin:\$PATH" >> /root/.bashrc

export PATH=/root/miniforge3/bin:$PATH
export TAR="/bin/tar"

# install dependencies with mamba
mamba env create -f /autograder/source/environment.yml
mamba install -n otter-env -c conda-forge nb_conda_kernels
mamba run -n otter-env bash -c "playwright install-deps && playwright install chromium"

# set mamba shell
mamba shell init --shell bash

And the template for R assignments is:

#!/usr/bin/env bash

export DEBIAN_FRONTEND=noninteractive
apt-get clean
apt-get update
apt-get install -y wget texlive-xetex texlive-fonts-recommended texlive-plain-generic \
    build-essential libcurl4-gnutls-dev libxml2-dev libssl-dev libgit2-dev texlive-lang-chinese
apt-get install -y libnlopt-dev cmake libfreetype6-dev libpng-dev libtiff5-dev libjpeg-dev \
    apt-utils libpoppler-cpp-dev libavfilter-dev  libharfbuzz-dev libfribidi-dev imagemagick \
    libmagick++-dev texlive-xetex texlive-fonts-recommended texlive-plain-generic \
    build-essential libcurl4-gnutls-dev libxml2-dev libssl-dev libgit2-dev texlive-lang-chinese \
    libxft-dev

# install pandoc
wget -nv https://github.com/jgm/pandoc/releases/download/3.1.11.1/pandoc-3.1.11.1-1-amd64.deb \
    -O /tmp/pandoc.deb
dpkg -i /tmp/pandoc.deb

# install mamba
if [ $(uname -p) = "arm" ] || [ $(uname -p) = "aarch64" ] ; \
    then wget -nv https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-aarch64.sh \
        -O /autograder/source/mamba_install.sh ; \
    else wget -nv https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh \
        -O /autograder/source/mamba_install.sh ; \
fi
chmod +x /autograder/source/mamba_install.sh
/autograder/source/mamba_install.sh -b
echo "export PATH=/root/miniforge3/bin:\$PATH" >> /root/.bashrc

export PATH=/root/miniforge3/bin:$PATH
export TAR="/bin/tar"

# install dependencies with mamba
mamba env create -f /autograder/source/environment.yml
mamba install -n otter-env -c conda-forge nb_conda_kernels
mamba run -n otter-env bash -c "playwright install-deps && playwright install chromium"
mamba run -n otter-env Rscript /autograder/source/requirements.r

# set mamba shell
mamba shell init --shell bash

Note that the line mamba run -n otter-env Rscript /autograder/source/requirements.r is only included if you have provided an R requirements file.

`environment.yml`#

This file specifies the conda environment that Otter creates in setup.sh. By default, it uses Python 3.9, but this can be changed using the --python-version flag to Otter Generate.

name: otter-env
channels:
  - defaults
  - conda-forge
dependencies:
  - python=3.9
  - pip
  - jupyter_server
  - pip:
      - otter-grader[grading,plugins]==6.1.3

If you’re grading a Python assignment, any dependencies in your requirements.txt will be added to the pip list in this file. If you pass --overwrite-requirements, your requirements.txt contents will be in the pip list instead of what’s above.

If you’re grading an R assignment, the environment.yml has additional depdencies:

name: otter-env
channels:
  - defaults
  - conda-forge
  - r
dependencies:
  - python=3.9
  - pip
  - jupyter_server
  - gcc_linux-64
  - gxx_linux-64
  - libgit2
  - libgomp
  - liblzma-devel
  - zlib-devel-amzn2-aarch64
  - zlib
  - r-base>=4.0.0
  - r-devtools
  - r-essentials
  - r-gert
  - r-rmarkdown
  - r-startup
  - r-stringi
  - r-testthat
  - r-usethis
  - r-ottr==1.5.0
  - pip:
      - otter-grader[grading,plugins,r]==6.1.3

`requirements.r`#

If you’re grading an R assignment, this file will be included if you specify it in the --requirements argument, and is just a copy of the file you provide. It should use functions like install.packages to install any additional dependencies your assignment requires. (Alternatively, if the depdendencies are available via conda, you can specify them in an environment.yml.)

`run_autograder`#

This is the file that Gradescope uses to actually run the autograder when a student submits. Otter provides this file as an executable that activates the conda environment and then calls /autograder/source/run_otter.py:

#!/bin/bash
export PATH=/root/miniforge3/bin:$PATH
eval "$(mamba shell hook --shell bash)"
mamba activate {{ otter_env_name }}
python {{ autograder_dir }}/source/run_otter.py

`run_otter.py`#

This file contains the logic to start the grading process by importing and running otter.run.run_autograder.main:

"""Runs Otter-Grader's autograding process"""

try:
    from otter.run.run_autograder import main as run_autograder
except ModuleNotFoundError as e:
    if "'otter'" in str(e):
        raise RuntimeError(
            "The 'otter' module could not be imported. This is usually caused by errors while building the "
            "grading image, so check the image build logs and include them when requesting "
            "support."
        )
    raise e


if __name__ == "__main__":
    run_autograder("{{ autograder_dir }}")

`otter_config.json`#

This file contains any user configurations for grading. It has no template but is populated with the any non-default values you specify for these configurations. When debugging grading via SSH on Gradescope, a helpful tip is to set the debug key of this JSON file to true; this will stop the autograding from ignoring errors when running students’ code, and can be helpful in debugging specific submission issues.

`tests`#

This is a directory containing the test files that you provide. All .py (or .R) files in the tests directory path that you provide are copied into this directory and are made available to submissions when the autograder runs.

`files`#

This directory, not present in all autograder zip files, contains any support files that you provide to be made available to submissions.

Grading Container Image

Contents

Grading Container Image#

setup.sh#

environment.yml#

requirements.r#

run_autograder#

run_otter.py#

otter_config.json#

tests#

files#