Creating Plugins
Contents
Creating Plugins#
Plugins can be created by exposing an importable class that inherits from
otter.plugins.AbstractOtterPlugin
. This class defines the base implementations of the plugin
event methods and sets up the other information needed to run the plugin.
Plugin Configurations#
When instantiated, plugins will receive three arguments: submission_path
, the absolute path to
the submission being executed, submission_metadata
, the information about the submission as read
in from submission_metadata.json
, and plugin_config
, the parsed configurations from the
user-specified list of plugins in the plugins
key of otter_config.json
. The default
implementation of __init__
, which should have the signature shown below, sets these two values to the instance variables
self.submission_metadata
and self.plugin_config
, respectively. Note that some events are
executed in contexts in which some or all of these variables are unavailable, and so these events
should not rely on the use of these instance variables. In cases in which these are unavailable,
they will be set for the falsey version of their type, e.g. {}
for self.submission_metadata
.
Plugin Events#
Plugins currently support five events. Most events are expected to modify the state of the
autograder and should not return anything, unless otherwise noted. Any plugins not supported should
raise a otter.plugins.PluginEventNotSupportedException
, as the default implementations in
AbstractOtterPlugin
do. Note that all methods should have the signatures described below.
during_assign
#
The during_assign
method will be called when Otter Assign is run. It will receive the
otter.assign.assignment.Assignment
instance with configurations for this assignment. The plugin
is run after output directories are written.
during_generate
#
The during_generate
method will be called when Otter Generate is run. It will receive the parsed
dictionary of configurations from otter_config.json
as its argument. Any changes made to the
configurations will be written to the otter_config.json
that will be stored in the configuration
zip file generated by Otter Generate.
from_notebook
#
The from_notebook
method will be called when a student makes a call to
otter.Notebook.run_plugin
with the plugin name. Any additional args and kwargs passed to
Notebook.run_plugin
will be passed to the plugin method. This method should not return anything
but can modify state or provide feedback to students. Note that because this plugin is not run in
the grading environment, it will *not* be passed the submission path, submission metadata, or a
plugin config. All information needed to run this plugin event must be gathered from the arguments
passed to it.
notebook_export
#
The notebook_export
method will be called when a student makes a call to
otter.Notebook.add_plugin_files
with the plugin name. Any additional args and kwargs passed to
Notebook.add_plugin_files
will be passed to the plugin mtehod. This method should return a list
of strings that correspond to file paths that should be included in the zip file generated by
otter.Notebook.export
when it is run. The Notebook
instance tracks these files in a list.
Note that because this plugin is not run in the grading environment, it will *not* be passed the
submission path, submission metadata, or a plugin config. All information needed to run this
plugin event must be gathered from the arguments passed to it.
before_grading
#
The before_grading
method will be called before grading occurs, just after the plugins are
instantiated, and will be passed an instance of the
otter.run.run_autograder.autograder_config.AutograderConfig
class, a fica configurations class.
Any changes made to the options will be used during grading.
before_execution
#
The before_execution
method will be called before execution occurs and will be passed the
student’s submission. If the submission is a notebook, the object passed will be a
nbformat.NotebookNode
from the parsed JSON. If the submission is a script, the objet passed will
be a string containing the file text. This method should return a properly-formatted
NotebookNode
or string that will be executed in place of the student’s original submission.
after_grading
#
The after_grading
method will be called after grading has occurred (meaning all tests have been
run and the results object has been created). It will be passed the
otter.test_files.GradingResults
object that contains the results of the student’s submission
against the tests. Any changes made to this object will be reflected in the results returned by the
autograder. For more information about this object, see
Non-containerized Grading.
generate_report
#
The generate_report
method will be called after results have been written. It receives no
arguments but should return a value. The return value of this method should be a string that
will be printed to stdout by Otter as part of the grading report.
AbstractOtterPlugin
Reference#
- class otter.plugins.AbstractOtterPlugin(submission_path, submission_metadata, plugin_config)#
Abstract base class for Otter plugins to inherit from. Includes the following methods:
during_assign
: run during Otter Assign after output directories are writtenduring_generate
: run during Otter Generate while all files are in-memory and before the the tmp directory is createdfrom_notebook
: run as students as they work through the notebook; seeNotebook.run_plugin
notebook_export
: run byNotebook.export
for adding files to the export zip filebefore_grading
: run before the submission is executed for altering configurationsbefore_execution
: run before the submission is executed for altering the submissionafter_grading
: run after all tests are run and scores are assignedgenerate_report
: run after results are written
See the docstring for the default implementation of each method for more information, including arguments and return values. If plugin has no actions for any event above, that method should raise
otter.plugins.PluginEventNotSupportedException
. (This is the default behavior of this ABC, so inheriting from this class will do this for you for any methods you don’t overwrite.)If this plugin requires metadata, it should be included in the plugin configuration of the
otter_config.json
file as a subdictionary with key a key corresponding to the importable name of the plugin. If no configurations are required, the plugin name should be listed as a string. For example, the config below provides configurations forMyOtterPlugin
but notMyOtherOtterPlugin
.{ "plugins": [ { "my_otter_plugin_package.MyOtterPlugin": { "some_metadata_key": "some_value" } }, "my_otter_plugin_package.MyOtherOtterPlugin" ] }
- Parameters
submission_path (
str
) – the absolute path to the submission being gradedsubmission_metadata (
dict
) – submission metadata; if on Gradescope, see https://gradescope-autograders.readthedocs.io/en/latest/submission_metadata/plugin_config (
dict
) – configurations from theotter_config.json
for this plugin, pulled fromotter_config["plugins"][][PLUGIN_NAME]
ifotter_config["plugins"][]
is adict
- submission_path#
the absolute path to the submission being graded
- Type
str
- submission_metadata#
submission metadata; if on Gradescope, see https://gradescope-autograders.readthedocs.io/en/latest/submission_metadata/
- Type
dict
- plugin_config#
configurations from the
otter_config.json
for this plugin, pulled fromotter_config["plugins"][][PLUGIN_NAME]
ifotter_config["plugins"][]
is adict
- Type
dict
- after_grading(results)#
Plugin event run after all tests are run on the resulting environment that gets passed the
otter.test_files.GradingResults
object that stores the grading results for each test case.- Parameters
results (
otter.test_files.GradingResults
) – the results of all test cases- Raises
PluginEventNotSupportedException – if the event is not supported by this plugin
- before_execution(submission)#
Plugin event run before the execution of the submission which can modify the submission itself. This method should return a properly-formatted
NotebookNode
or string that will be executed in place of the student’s original submission.- Parameters
submission (
nbformat.NotebookNode
orstr
) – the submission for grading; if it is a notebook, this will be the JSON-parseddict
of its contents; if it is a script, this will be a string containing the code- Returns
the altered submission to be executed
- Return type
nbformat.NotebookNode
orstr
- Raises
PluginEventNotSupportedException – if the event is not supported by this plugin
- before_grading(config)#
Plugin event run before the execution of the submission which can modify the dictionary of grading configurations.
- Parameters
config (
otter.run.run_autograder.autograder_config.AutograderConfig
) – the autograder config- Raises
PluginEventNotSupportedException – if the event is not supported by this plugin
- during_assign(assignment)#
Plugin event run during the execution of Otter Assign after output directories are wrriten. Assignment configurations are passed in via the
assignment
argument.- Parameters
assignment (
otter.assign.assignment.Assignment
) – theAssignment
instance with configurations for the assignment; used similar to anAttrDict
where keys are accessed with the dot syntax (e.g.assignment.master
is the path to the master notebook)- Raises
PluginEventNotSupportedException – if the event is not supported by this plugin
- during_generate(otter_config, assignment)#
Plugin event run during the execution of Otter Generate that can modify the configurations to be written to
otter_config.json
.- Parameters
otter_config (
dict
) – the dictionary of Otter configurations to be written tootter_config.json
in the zip fileassignment (
otter.assign.assignment.Assignment
) – theAssignment
instance with configurations for the assignment if Otter Assign was used to generate this zip file; will be set toNone
if Otter Assign is not being used
- Raises
PluginEventNotSupportedException – if the event is not supported by this plugin
- from_notebook(*args, **kwargs)#
Plugin event run by students as they work through a notebook via the
Notebook
API (seeNotebook.run_plugin
). Accepts arbitrary arguments and has no return.- Parameters
*args – arguments for the plugin event
**kwargs – keyword arguments for the plugin event
- Raises
PluginEventNotSupportedException – if the event is not supported by this plugin
- generate_report()#
Plugin event run after grades are written to disk. This event should return a string that gets printed to stdout as a part of the plugin report.
- Returns
the string to be included in the report
- Return type
str
- Raises
PluginEventNotSupportedException – if the event is not supported by this plugin
- notebook_export(*args, **kwargs)#
Plugin event run when a student calls
Notebook.export
. Accepts arbitrary arguments and should return a list of file paths to include in the exported zip file.- Parameters
*args – arguments for the plugin event
**kwargs – keyword arguments for the plugin event
- Returns
the list of file paths to include in the export
- Return type
list[str]
- Raises
PluginEventNotSupportedException – if the event is not supported by this plugin