The Python Post Hook Script Test is designed to be simple, but powerful. The test itself is written in python, and has access to a good amount of environment variables related to the test case run, the submission itself, as well as a connection to the database!
How it works
By default, the Python Post Hook Script test case will:
Create a database
Use your specified data set to seed the database
Run the specified query from the submission
Run the post hook, which sets the grade for the submission
The Post Hook
The post hook is a small but powerful script you can put together in python that has access to the live database that is being used to test the query, as well as many environment variables related to the submission and test case itself. The following variables are included in a mimir object that is included within the python context:
mimir.DB # a connection to the database
# see below for more info on what object this is
mimir.COURSE_ID # the course this project/assignment is in
mimir.EMAIL # the submitter's email
mimir.QUERY # the contents of the query
mimir.QUERY_OUTPUT # the output of the query
mimir.QUERY_TIME # the runtime of the query
mimir.SEED_TIME # the runtime of the data set seed operation
mimir.SUBMISSION_ID # the submission id
mimir.SUBMISSION_TIME # the time of the submission
mimir.TEST_CASE_ID # the id of the test case
mimir.USER_ID # the submitter's user id
mimir.set_score(100.0) # a function to set the score for the test
# accepts int/float values
# this value will be minmaxed to 0-100
NOTE: The post hook is currently limited to 10 seconds of runtime (not including loading the dataset)
The mimir.DB Object
The DB variable inside of the mimir module in the post hook can vary, depending on which database you are using. We use the following connectors for each database: