Otter Grader is a light-weight, modular open-source autograder developed by the Data Science Education Program at UC Berkeley. It is designed to grade Python and R assignments for classes at any scale by abstracting away the autograding internals in a way that is compatible with any instructor’s assignment distribution and collection pipeline. Otter supports local grading through parallel Docker containers, grading using the autograding platforms of 3rd-party learning management systems (LMSs), non-containerized grading on an instructor’s machine, and a client package that allows students to check and instructors to grade assignments their own machines. Otter is designed to grade Pyhon and R executables, Jupyter Notebooks, and RMarkdown documents and is compatible with a few different LMSs, including Canvas and Gradescope.
The core abstraction of Otter, as compared to other autograders like nbgrader and OkPy, is this: you provide the compute, and Otter takes care of the rest. All a instructor needs to do in order to autograde is find a place to run Otter (a server, a JupyterHub, their laptop, etc.) and Otter will take care of generating assignments and tests, creating and managing grading environents, and grading submissions. Otter is platform-agnostic, allowing you to put and grade your assignments anywhere you want.
Otter is organized into six components based on the different stages of the assignment pipeline, each with a command-line interface:
Otter Assign is an assignment development and distribution tool that allows instructors to create assignments with prompts, solutions, and tests in a simple notebook format that it then converts into santized versions for distribution to students and autograders.
Otter Generate creates the necessary setup files so that instructors can autograde assignments.
Otter Check allows students to run publically distributed tests written by instructors against their solutions as they work through assignments to verify their thought processes and implementations.
Otter Export generates PDFs with optional filtering of Jupyter Notebooks for manually grading portions of assignments.
Otter Run grades students’ assignments locally on the instructor’s machine without containerization and supports grading on a JupyterHub account.
Otter Grade grades students’ assignments locally on the instructor’s machine in parallel Docker containers, returning grade breakdowns as a CSV file.
Otter is a Python package that is compatible with Python 3.6+. The PDF export internals require either LaTeX and Pandoc or wkhtmltopdf to be installed. Docker is also required to grade assignments locally with containerization, and Postgres only if you’re using Otter Service. Otter’s Python package can be installed using pip. To install the current stable version, install with
pip install otter-grader
If you are going to be autograding R, you must also install the R package using
Installing the Python package will install the
otter binary so that Otter can be called from the
command line. You can also call Otter as a Python module with
python3 -m otter.
Otter uses Docker to create containers in which to run the students’ submissions. Docker and our Docker image are only required if using Otter Grade. Please make sure that you install Docker and pull our Docker image, which is used to grade the notebooks. To get the Docker image, run
docker pull ucbdsinfra/otter-grader