Debugging Submissions#
To help debug submissions that are not passing tests or which are behaving oddly, it can be helpful to use Otter’s debug mode. The main use for debug mode is to examine errors that are being thrown in the submission and which Otter swallows during normal execution. This is a guide for setting up an environment where Otter’s debug mode is enabled and for using it.
Setting Up a Debugging Environment#
The first step is to set up a debugging environment. This step depends on how you’re executing the
submissions. If you’re using containerized local grading (otter grade
) or Gradescope, follow the
steps in this section. If you’re using non-containerized grading (otter run
), follow the steps
in Debugging with Otter Run.
Otter Grade Setup#
To create a container for debugging a submission, use the Docker image that was created for your
assignment when you ran otter grade
. The image’s tag will use the assignment name (the value of
the -n
or --name
flag). If you passed -n hw01
, the image name will be
otter-grade:hw01
. Create a container using this image and run bash:
docker run -it otter-grade:<assignment name> bash
This should create the container and drop you into a bash prompt inside of it. Now, in a separate terminal, copy the submission file into the container you’ve just created:
docker cp path/to/subm.ipynb <container id>:/autograder/submission/subm.ipynb
You can get the container ID either from the prompt in the container’s bash shell or from the
docker container ls
command.
Now that you’ve set up the grading environment, continue with the debugging steps below. Once you’re finished debugging, remember to stop the container and remove it.
Gradescope Setup#
Find the submission you would like to debug on Gradescope and use Gradescope’s Debug via SSH feature to create a container with that submission and SSH into it, then continue with the debugging steps below.
Debugging in a Container#
In the container, cd
into the /autograder/source
directory. There should be an
otter_config.json
file in this directory containing Otter’s autograder configuration. (If there
isn’t one present, make one.) Edit this file to set "debug"
to true
(this enables debug
mode).
Now cd
into /autograder
and run
./run_autograder
This shell script runs Otter and its results should be printed to the console. If an error is thrown while executing the notebook, it will be shown and the autograder will stop. If you’re running on Gradescope, no changes or autograder runs made here will affect the submission or its grade.
If you need to edit the submission file(s), they are located in the /autograder/submission
directory.
Debugging with Otter Run#
Read the previous section first. Because Otter Run relies on an
autograder zip file for its configuration intsead of a Docker container, you will need to manually
edit the otter_config.json
file in your autograder zip file to set "debug"
to true
.
Then, re-zip the zip file’s contents and use this new autograder zip file for otter run
.