Grading Container Image¶

When you use Otter Generate to create the configuration zip file for Gradescope, Otter includes the following software and packages for installation. The zip archive, when unzipped, has the following contents:

autograder
├── environment.yml
├── files/
├── otter_config.json
├── requirements.r
├── requirements.txt
├── run_autograder
├── run_otter.py
├── setup.sh
└── tests/

Note that for pure-Python assignments, requirements.r is not included and all of the R-pertinent portions of setup.sh are removed. Below are descriptions of each of the items listed above and the Jinja2 templates used to create them (if applicable).

`setup.sh`¶

This file, required by Gradescope, performs the installation of necessary software for the autograder to run. The script template looks like this:

#!/usr/bin/env bash

if [ "${BASE_IMAGE}" != "ucbdsinfra/otter-grader" ]; then
    apt-get clean
    apt-get update
    apt-get install -y pandoc texlive-xetex texlive-fonts-recommended texlive-plain-generic build-essential libcurl4-gnutls-dev libxml2-dev libssl-dev libgit2-dev texlive-lang-chinese
    apt-get install -y libnlopt-dev cmake libfreetype6-dev libpng-dev libtiff5-dev libjpeg-dev apt-utils libpoppler-cpp-dev libavfilter-dev  libharfbuzz-dev libfribidi-dev imagemagick libmagick++-dev pandoc texlive-xetex texlive-fonts-recommended texlive-plain-generic build-essential libcurl4-gnutls-dev libxml2-dev libssl-dev libgit2-dev texlive-lang-chinese libxft-dev

    # install wkhtmltopdf
    wget --quiet -O /tmp/libssl1.1.deb http://archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1-1ubuntu2.1~18.04.20_amd64.deb
    apt-get install -y /tmp/libssl1.1.deb
    wget --quiet -O /tmp/wkhtmltopdf.deb https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6-1/wkhtmltox_0.12.6-1.bionic_amd64.deb
    apt-get install -y /tmp/wkhtmltopdf.deb

    # try to set up R
    apt-get clean
    apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9
    add-apt-repository 'deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran40/'

    # install conda
    wget -nv -O {{ autograder_dir }}/source/miniconda_install.sh "{{ miniconda_install_url }}"
    chmod +x {{ autograder_dir }}/source/miniconda_install.sh
    {{ autograder_dir }}/source/miniconda_install.sh -b
    echo "export PATH=/root/miniconda3/bin:\$PATH" >> /root/.bashrc

    export PATH=/root/miniconda3/bin:$PATH
    export TAR="/bin/tar"
fi

# install dependencies with conda
{% if channel_priority_strict %}conda config --set channel_priority strict
{% endif %}conda env create -f {{ autograder_dir }}/source/environment.yml
conda run -n {{ otter_env_name }} Rscript {{ autograder_dir }}/source/requirements.r

# set conda shell
conda init --all

# install ottr; not sure why it needs to happen twice but whatever
git clone --single-branch -b {{ ottr_branch }} https://github.com/ucbds-infra/ottr.git {{ autograder_dir }}/source/ottr
cd {{ autograder_dir }}/source/ottr 
conda run -n {{ otter_env_name }} Rscript -e "devtools::install\\(\\)"
conda run -n {{ otter_env_name }} Rscript -e "devtools::install\\(\\)"

This script does the following:

Install Python 3.7 and pip
Install pandoc and LaTeX
Install wkhtmltopdf
Set Python 3.7 to the default python version
Install apt-get dependencies for R
Install Miniconda
Install R, r-essentials, and r-devtools from conda
Install Python requirements from requirements.txt (this step installs Otter itself)
Install R requires from requirements.r
Install Otter’s R autograding package ottr

Currently this script is not customizable, but you can unzip the provided zip file to edit it manually.

`environment.yml`¶

This file specifies the conda environment that Otter creates in setup.sh. By default, it uses Python 3.7, but this can be changed using the --python-version flag to Otter Generate.

{% if not other_environment %}name: {{ otter_env_name }}
channels:
    - defaults
    - conda-forge
dependencies:
    - python={{ python_version }}
    - pip
    - nb_conda_kernels
    - r-base>=4.0.0
    - r-essentials
    - r-devtools
    - libgit2
    - libgomp
    - pip:
        - -r requirements.txt{% else %}{{ other_environment }}{% endif %}

`requirements.txt`¶

This file contains the Python dependencies required to execute submissions. It is also the file that your requirements are appended to when you provided your own requirements file for a Python assignment (unless --overwrite-requirements is passed). The template requirements for Python are:

{% if not overwrite_requirements %}datascience
jupyter_client
ipykernel
matplotlib
pandas
ipywidgets
scipy
seaborn
scikit-learn
jinja2
nbconvert
nbformat
dill
numpy
gspread
otter-grader==4.3.2
{% endif %}{% if other_requirements %}
{{ other_requirements }}{% endif %}

If you are using R, there will be no additional requirements sent to this template, and rpy2 will be added. Note that this installs the exact version of Otter that you are working from. This is important to ensure that the configurations you set locally are correctly processed and processable by the version on the Gradescope container. If you choose to overwrite the requirements, make sure to do the same.

`requirements.r`¶

This file uses R functions like install.packages and devtools::install_github to install R packages. If you are creating an R assignment, it is this file (rather than requirements.txt) that your requirements and overwrite-requirements options are applied to. The template file contains:

{% if not overwrite_requirements %}
install.packages(c(
    "gert",
    "usethis",
    "testthat",
    "startup",
    "rmarkdown"
), dependencies=TRUE, repos="http://cran.us.r-project.org")

install.packages(
    "stringi", 
    configure.args='--disable-pkg-config --with-extra-cxxflags="--std=c++11" --disable-cxx11', 
    repos="http://cran.us.r-project.org"
)
{% endif %}{% if other_requirements %}
{{ other_requirements }}
{% endif %}

`run_autograder`¶

This is the file that Gradescope uses to actually run the autograder when a student submits. Otter provides this file as an executable that activates the conda environment and then calls /autograder/source/run_otter.py:

#!/usr/bin/env bash
if [ "${BASE_IMAGE}" != "ucbdsinfra/otter-grader" ]; then
    export PATH="/root/miniconda3/bin:$PATH"
    source /root/miniconda3/etc/profile.d/conda.sh
else
    export PATH="/opt/conda/bin:$PATH"
    source /opt/conda/etc/profile.d/conda.sh
fi
conda activate {{ otter_env_name }}
python {{ autograder_dir }}/source/run_otter.py

`run_otter.py`¶

This file contains the logic to start the grading process by importing and running otter.run.run_autograder.main:

"""
Runs Otter on Gradescope
"""

import os
import subprocess

from otter.run.run_autograder import main as run_autograder

if __name__ == "__main__":
    run_autograder('{{ autograder_dir }}')

`otter_config.json`¶

This file contains any user configurations for grading. It has no template but is populated with the default values and any updates to those values that a user specifies. When debugging grading via SSH on Gradescope, a helpful tip is to set the debug key of this JSON file to true; this will stop the autograding from ignoring errors when running students’ code, and can be helpful in debugging specific submission issues.

`tests`¶

This is a directory containing the test files that you provide. All .py (or .R) files in the tests directory path that you provide are copied into this directory and are made available to submissions when the autograder runs.

`files`¶

This directory, not present in all autograder zip files, contains any support files that you provide to be made available to submissions.

Otter-Grader documentation

Grading Container Image¶

setup.sh¶

environment.yml¶

requirements.txt¶

requirements.r¶

run_autograder¶

run_otter.py¶

otter_config.json¶

tests¶

files¶