IO Templating is a feature on Mimir Classroom that allows instructors to scan for specific things on their student's outputs. The feature utilizes regex string matches to allow for more customization when checking student output.

Use cases would include:

  • Checking for a certain output type but it could be different for other submissions
  • Checking for structure of output without matching the contents exactly
  • Allowing for fuzzy matching of output

The way the language works is that anything outside of the template tags will be matched exactly and anything inside the tags will be used as a regex in line to match output.

How to use it

To activate the template language for an I/O test case, you just need to include some of the template tags in your expected output. With the test case rework, the IO template language toggle switch has been removed, and by default Mimir Classroom will intelligently determine whether or not to use the templating language.

The template tags are '<~' and '~>' anything in between these two tags is taken as a regex and used to compare the template to the output from the student. It is compared line by line (newlines are strict), and the score returned is the number of matching lines over the total number of lines, expressed as a percentage from 0 to 100. Anything not in tags will be compared exactly.

Examples

This is a sample output from a program:

The answer to the question: 100

a matching template:

The answer to the question: <~[0-9]+~>

Another example:

1 2 3 4 5
hello world
aaaaa
ExAmPlE
abc123

a possible matching template:

<~[0-9]~> <~[0-9]~> <~[0-9]~> <~[0-9]~> <~[0-9]~>
<~[a-z]+[\s][a-z]+~>
<~.....~>
<~[A-Za-z]+~>
<~[a-z]+[0-9]+~>

Better output diffs

The latest update to the IO template language brings more informative diffs, allowing you to see exactly where your regex may be failing to match, as well as line numbers. You can also let your students see this on their submissions if you'd like, but be aware that it will show them the template lines side by side with their output lines.

Example of the new diff format return:

Missed Lines: {number of mismatched lines}/{total lines in the file}

Line {line number of mismatch}
| {template line}
| {output line}
Did this answer your question?