Contents

Creating Plugins

Plugins can be created by exposing an importable class that inherits from otter.plugins.AbstractOtterPlugin. This class defines the base implementations of the plugin event methods and sets up the other information needed to run the plugin.

Plugin Configurations

When instantiated, plugins will receive three arguments: submission_path, the absolute path to the submission being executed, submission_metadata, the information about the submission as read in from submission_metadata.json, and plugin_config, the parsed configurations from the user-specified list of plugins in the plugins key of otter_config.json. The default implementation of __init__, which should have the signature shown below, sets these two values to the instance variables self.submission_metadata and self.plugin_config, respectively. Note that some events are executed in contexts in which some or all of these variables are unavailable, and so these events should not rely on the use of these instance variables. In cases in which these are unavailable, they will be set for the falsey version of their type, e.g. {} for self.submission_metadata.

Plugin Events

Plugins currently support five events. Most events are expected to modify the state of the autograder and should not return anything, unless otherwise noted. Any plugins not supported should raise a otter.plugins.PluginEventNotSupportedException, as the default implementations in AbstractOtterPlugin do. Note that all methods should have the signatures described below.

during_assign

The during_assign method will be called when Otter Assign is run. It will receive the otter.assign.assignment.Assignment instance with configurations for this assignment. The plugin is run after output directories are written.

during_generate

The during_generate method will be called when Otter Generate is run. It will receive the parsed dictionary of configurations from otter_config.json as its argument. Any changes made to the configurations will be written to the otter_config.json that will be stored in the configuration zip file generated by Otter Generate.

from_notebook

The from_notebook method will be called when a student makes a call to otter.Notebook.run_plugin with the plugin name. Any additional args and kwargs passed to Notebook.run_plugin will be passed to the plugin method. This method should not return anything but can modify state or provide feedback to students. Note that because this plugin is not run in the grading environment, it will *not* be passed the submission path, submission metadata, or a plugin config. All information needed to run this plugin event must be gathered from the arguments passed to it.

notebook_export

The notebook_export method will be called when a student makes a call to otter.Notebook.add_plugin_files with the plugin name. Any additional args and kwargs passed to Notebook.add_plugin_files will be passed to the plugin mtehod. This method should return a list of strings that correspond to file paths that should be included in the zip file generated by otter.Notebook.export when it is run. The Notebook instance tracks these files in a list. Note that because this plugin is not run in the grading environment, it will *not* be passed the submission path, submission metadata, or a plugin config. All information needed to run this plugin event must be gathered from the arguments passed to it.

before_grading

The before_grading method will be called before grading occurs, just after the plugins are instantiated, and will be passed an instance of the otter.run.run_autograder.autograder_config.AutograderConfig class, a fica configurations class. Any changes made to the options will be used during grading.

before_execution

The before_execution method will be called before execution occurs and will be passed the student’s submission. If the submission is a notebook, the object passed will be a nbformat.NotebookNode from the parsed JSON. If the submission is a script, the objet passed will be a string containing the file text. This method should return a properly-formatted NotebookNode or string that will be executed in place of the student’s original submission.

after_execution

The after_execution method will be called after the student’s submission is executed but before the results object is created. It will be passed the global environment resulting from the execution of the student’s code. Any changes made to the global environment will be reflected in the tests that are run after execution (meaning any that do not have a call to otter.Notebook.check to run them).

after_grading

The after_grading method will be called after grading has occurred (meaning all tests have been run and the results object has been created). It will be passed the otter.test_files.GradingResults object that contains the results of the student’s submission against the tests. Any changes made to this object will be reflected in the results returned by the autograder. For more information about this object, see Non-containerized Grading.

generate_report

The generate_report method will be called after results have been written. It receives no arguments but should return a value. The return value of this method should be a string that will be printed to stdout by Otter as part of the grading report.

AbstractOtterPlugin Reference

class otter.plugins.AbstractOtterPlugin(submission_path, submission_metadata, plugin_config)

Abstract base class for Otter plugins to inherit from. Includes the following methods:

  • during_assign: run during Otter Assign after output directories are written

  • during_generate: run during Otter Generate while all files are in-memory and before the the tmp directory is created

  • from_notebook: run as students as they work through the notebook; see Notebook.run_plugin

  • notebook_export: run by Notebook.export for adding files to the export zip file

  • before_grading: run before the submission is executed for altering configurations

  • before_execution: run before the submission is executed for altering the submission

  • after_execution: run after the submission is executed

  • after_grading: run after all tests are run and scores are assigned

  • generate_report: run after results are written

See the docstring for the default implementation of each method for more information, including arguments and return values. If plugin has no actions for any event above, that method should raise otter.plugins.PluginEventNotSupportedException. (This is the default behavior of this ABC, so inheriting from this class will do this for you for any methods you don’t overwrite.)

If this plugin requires metadata, it should be included in the plugin configuration of the otter_config.json file as a subdictionary with key a key corresponding to the importable name of the plugin. If no configurations are required, the plugin name should be listed as a string. For example, the config below provides configurations for MyOtterPlugin but not MyOtherOtterPlugin.

{
    "plugins": [
        {
            "my_otter_plugin_package.MyOtterPlugin": {
                "some_metadata_key": "some_value"
            }
        },
        "my_otter_plugin_package.MyOtherOtterPlugin"
    ]
}
Parameters

  • submission_path (str) – the absolute path to the submission being graded

  • submission_metadata (dict) – submission metadata; if on Gradescope, see https://gradescope-autograders.readthedocs.io/en/latest/submission_metadata/

  • plugin_config (dict) – configurations from the otter_config.json for this plugin, pulled from otter_config["plugins"][][PLUGIN_NAME] if otter_config["plugins"][] is a dict

submission_path

the absolute path to the submission being graded

Type

str

submission_metadata

submission metadata; if on Gradescope, see https://gradescope-autograders.readthedocs.io/en/latest/submission_metadata/

Type

dict

plugin_config

configurations from the otter_config.json for this plugin, pulled from otter_config["plugins"][][PLUGIN_NAME] if otter_config["plugins"][] is a dict

Type

dict

after_execution(global_env)

Plugin event run after the execution of the submission which can modify the resulting global environment.

Parameters

global_env (dict) – the environment resulting from the execution of the students’ code; see otter.execute

Raises

PluginEventNotSupportedException – if the event is not supported by this plugin

after_grading(results)

Plugin event run after all tests are run on the resulting environment that gets passed the otter.test_files.GradingResults object that stores the grading results for each test case.

Parameters

results (otter.test_files.GradingResults) – the results of all test cases

Raises

PluginEventNotSupportedException – if the event is not supported by this plugin

before_execution(submission)

Plugin event run before the execution of the submission which can modify the submission itself. This method should return a properly-formatted NotebookNode or string that will be executed in place of the student’s original submission.

Parameters

submission (nbformat.NotebookNode or str) – the submission for grading; if it is a notebook, this will be the JSON-parsed dict of its contents; if it is a script, this will be a string containing the code

Returns

the altered submission to be executed

Return type

nbformat.NotebookNode or str

Raises

PluginEventNotSupportedException – if the event is not supported by this plugin

before_grading(config)

Plugin event run before the execution of the submission which can modify the dictionary of grading configurations.

Parameters

config (otter.run.run_autograder.autograder_config.AutograderConfig) – the autograder config

Raises

PluginEventNotSupportedException – if the event is not supported by this plugin

during_assign(assignment)

Plugin event run during the execution of Otter Assign after output directories are wrriten. Assignment configurations are passed in via the assignment argument.

Parameters

assignment (otter.assign.assignment.Assignment) – the Assignment instance with configurations for the assignment; used similar to an AttrDict where keys are accessed with the dot syntax (e.g. assignment.master is the path to the master notebook)

Raises

PluginEventNotSupportedException – if the event is not supported by this plugin

during_generate(otter_config, assignment)

Plugin event run during the execution of Otter Generate that can modify the configurations to be written to otter_config.json.

Parameters

  • otter_config (dict) – the dictionary of Otter configurations to be written to otter_config.json in the zip file

  • assignment (otter.assign.assignment.Assignment) – the Assignment instance with configurations for the assignment if Otter Assign was used to generate this zip file; will be set to None if Otter Assign is not being used

Raises

PluginEventNotSupportedException – if the event is not supported by this plugin

from_notebook(*args, **kwargs)

Plugin event run by students as they work through a notebook via the Notebook API (see Notebook.run_plugin). Accepts arbitrary arguments and has no return.

Parameters

  • *args – arguments for the plugin event

  • **kwargs – keyword arguments for the plugin event

Raises

PluginEventNotSupportedException – if the event is not supported by this plugin

generate_report()

Plugin event run after grades are written to disk. This event should return a string that gets printed to stdout as a part of the plugin report.

Returns

the string to be included in the report

Return type

str

Raises

PluginEventNotSupportedException – if the event is not supported by this plugin

notebook_export(*args, **kwargs)

Plugin event run when a student calls Notebook.export. Accepts arbitrary arguments and should return a list of file paths to include in the exported zip file.

Parameters

  • *args – arguments for the plugin event

  • **kwargs – keyword arguments for the plugin event

Returns

the list of file paths to include in the export

Return type

list[str]

Raises

PluginEventNotSupportedException – if the event is not supported by this plugin

previous

Gmail Notifications

next

PDF Generation and Filtering