.. _workflow_executing_submissions_otter_grade: Grading Locally =============== The command line interface allows instructors to grade notebooks locally by launching Docker containers on the instructor's machine that grade notebooks and return a CSV of grades and (optionally) PDF versions of student submissions for manually graded questions. Configuration Files ------------------- Otter grades students submissions in individual Docker containers that are based on a Docker image generated through the use of a configuration zip file. Before grading assignments locally, an instructor should create such a zip file by using a tool such as :ref:`Otter Assign ` or :ref:`Otter Generate `. This file will be used in the construction of a Docker image tagged ``otter-grader:{zip file hash}``. This Docker image will then have containers spawned from it for each submission that is graded. Otter's Docker images can be pruned with ``otter grade --prune``. Using the CLI ------------- Before using the command line utility, you should have * written tests for the assignment, * generated a configuration zip file from those tests, and * downloaded submissions into a directory The grading interface, encapsulated in the ``otter grade`` command, runs the local grading process and defines the options that instructors can set when grading. A comprehensive list of flags is provided in the :ref:`cli_reference`. Basic Usage +++++++++++ The simplest usage of the Otter Grade is when we have a directory structure as below (and we have change directories into ``grading`` in the command line) and we don't require PDFs or additional requirements. .. code-block:: grading ├── autograder.zip ├── nb0.ipynb ├── nb1.ipynb ├── nb2.ipynb # etc. └── tests ├── q1.py ├── q2.py └── q3.py # etc. In the case above, our otter command would be, very simply, .. code-block:: console otter grade Because the submissions are on the current working directory (``grading``), our configuration file is at ``./autograder.zip``, and we don't mind output to ``./``, we can use the defualt values of the ``-a`` and ``-o`` flags. After grader, our directory will look like this: .. code-block:: grading ├── autograder.zip ├── final_grades.csv ├── nb0.ipynb ├── nb1.ipynb ├── nb2.ipynb # etc. └── tests ├── q1.py ├── q2.py └── q3.py # etc. and the grades for each submission will be in ``final_grades.csv``. If we wanted to generate PDFs for manual grading, we would specify this when making the configuration file and add the ``--pdfs`` flag to tell Otter to copy the PDFs out of the containers: .. code-block:: otter grade --pdfs and at the end of grading we would have .. code-block:: grading ├── autograder.zip ├── final_grades.csv ├── nb0.ipynb ├── nb1.ipynb ├── nb2.ipynb # etc. ├── submission_pdfs │ ├── nb0.pdf │ ├── nb1.pdf │ └── nb2.pdf # etc. └── tests ├── q1.py ├── q2.py └── q3.py # etc. To grade a single notebook(as opposed to all the notebooks in a directory), you can pass the path to the notebook file using the flag ``-p``. For example, using the directory structure above, the call looks like this: .. code-block:: console otter grade -p "./nb0.ipynb" The usual information for this graded notebook is written to ``final_grades.csv`` but the percent correct is returned to the command-line from the call as well. To grade submissions that aren't notebook files, use the ``--ext`` flag, which accepts the file extension to search for submissions with. For example, if we had the same example as above but with Rmd files: .. code-block:: console otter grade --ext Rmd If you're grading submission export zip files (those generated by ``otter.Notebook.export`` or ``ottr::export``), your ``otter_config.json`` should have its ``zips`` key set to true, and you should pass the ``-z`` flag to Otter Grade. You also won't need to use the ``--ext`` flag regardless of the type of file being graded inside the zip file. Requirements ++++++++++++ The Docker image used for grading will be built as described in the :ref:`Otter Generatte ` section. If you require any packages not listed there, or among the dependencies of any packages above, you should create a requirements.txt file *containing only those packages* and use it when running your configuration generator. Support Files +++++++++++++ Some notebooks require support files to run (e.g. data files). If your notebooks require any such files, you should generate your configuration zip file with those files. Intercell Seeding +++++++++++++++++ Otter Grade also supports :ref:`intercell seeding `. This behavior should be configured as a part of your configuration zip file.