Submission Execution
Contents
Submission Execution#
This section of the documentation describes the process by which submissions are executed in the autograder. Regardless of the method by which Otter is used, the autograding internals are all the same, although the execution differs slightly based on the file type of the submission.
Python#
All Python submissions, regardless of type, are converted to notebooks and graded with nbformat’s
ExecutePreprocessor
. For Python scripts, they are converted by creating a notebook with a
single code cell that contains the script’s contents.
The notebooks are executed as follows:
The
before_execution
plugin event is run.Cells tagged with
otter_ignore
or withotter[ignore]
set totrue
in the metadata are removed from the notebook.(If grading from an Otter log) all lines of code which are not import statements are removed from the submission and replaced with code to unpack the serialized environments from the log.
Additional checks indicated in the cell metadata (
otter[tests]
) are added those cells.(If intercell seeding is being used) a cell importing
numpy
andrandom
is added to the top of the notebook and each code cell is prepended with the seeding code.The current working directory is added to
sys.path
.Cells to initialize Otter and export grading results are added at the beginning and end of the notebook, respectively.
The notebook is executed with the
ExecutePreprocessor
. If running in debug mode, errors are not ignored.The results exported by Otter from the notebook are loaded.
The
after_grading
plugin event is run.