Unit test cases are useful when you want to test individual functionality for a object/class/file/method/function. Currently, you can create Unit test cases for most of our popular languages, including Python (2.7, 3.5, 3.6, 3.7, 3.8), Ruby, Java (7, 8, 9, 10, 12), c#, c, haskell, and c++ (98, 11, 14, 17)
Getting Started
Inside unit test cases, yo are required to provide a test case name, a main file to test, and some unit code. This unit code must be written in the same language that the project is written in. This allows you to interact with objects/methods/functions that are defined in your perfect code or your student's submissions. There are some specific ways you must assert or confirm that the test passed for certain languages; examples for those are given below.
IMPORTANT!
The toggle options at the bottom of the Unit test case dialog box are very influential on the operation of your test case. If you toggle on any of the 'Show' options, your test case input is at risk of extraction when students run their code. For this reason, we encourage instructors to make multiple test cases, some completely hidden (meaning all 'Show' options are toggled off), and some other test cases that give feedback to students.
Python Unit Tests
Python unit tests must contain at least 1 assert line. This is the line that actually tests the program. See the below examples:
# use this
this = 1
that = 1
assert this is that
# or this
assert(this == that)
Example that seems more real:
a0 = Account(0)
a1 = Account(1)
a0.deposit(1000)
a1.deposit(100)
a0.transfer(a1, 500)
assert(a1.balance == 600.0)
try:
a0.withdraw(100000)
assert(False)
except InsufficientFunds:
assert(True)
Important!
For simplicity, python unit tests will auto-import wildcard from all .py files in the root folder of the running test. You can stop this functionality by putting the following at the top (first line) of the unit test code:
#!NO_IMPORT
After that, you can complete necessary imports.
Java Unit Tests
Java unit tests must return either a true or a false at the end of the unit code that you supply. Print statements inside your test will be shown in your debug output, meaning you will be able to see them while creating and working on your test case. If you leave those statements in and allow your students to see their output, then they will be able to see these print statements.
Example 1:
this = 1;
that = 1;
System.out.println("Hello Students!");
return (this == that);
Example 2:
BankAccount b = new BankAccount(1000000, 0.04);
b.deposit(1000.0);
b.monthlyProcess();
b.withdraw(100000.0);
b.deposit(50.0);
b.monthlyProcess();
return b.getBalance() == 901050.0;
Mimir Classroom is able to handle imports in unit tests. These could include anything, but common imports are ArrayList, Random, etc. It's actually quite easy to add in other modules:
import java.util.Random;
long seed = 1024;
Random rand = new Random(seed);
int randomNumber = rand.nextInt(20);
return (19 >= randomNumber) && (randomNumber >= 0); // make sure the number is between 0 and 19 inclusive
Keep in mind that any imports are test case level, not project level, so they will only be included in the test case you have them included in. Also, the import lines should only appear at the top, above all actual test case code.
Only One Class/File
In some cases, you'll only want to compile one file or a set of a few files before using those classes in a Java Unit Test Case. This can be useful for when you want test cases to be able to run without having your students complete every part of a multi-file assignment, or just when you want to be more explicit in a test case. In these cases, you can add the flag:
--only-main-file
into the 'Additional Files to Compile' section of the test case. This directive tells the autograder to not attempt to compile anything other than what was explicitly defined by the main file, and anything else in the 'Additional Files to Compile' section of the test case configuration.
C++ Unit Tests
C++ Unit tests are a bit more intricate than the other Unit test cases. Check out the article about C++ Unit tests here.
C Unit Tests
For unit testing C code, we use the MUnit framework.
There are many different assert types you can use in your tests, but the main one is munit_assert().
Additional asserts can be found below:
munit_assert_double_equal(double a, double b, int precision);
// Assert that two doubles are equal within a tolerance of 1.0×10-precision. // For example, 3.141592654 and 3.141592653589793 are considered equal with // a precision of 9 but not 10.
munit_assert_string_equal(const char* a, const char* b);
// Assert that two strings are equivalent
// (i.e., strcmp(a,b) == 0, not a == b).
munit_assert_string_not_equal(const char* a, const char* b);
// Like munit_assert_string_equal, but make sure they aren't equivalent.
munit_assert_memory_equal(size_t size, const void* a, const void* b);
// This will make sure two blocks of memory contain the same data.
// Failures messages will tell you the offset of the first non-equal byte.
munit_assert_memory_not_equal(size_t size, const void* a, const void* b);
// Make sure two blocks of memory don't contain the same data.
munit_assert_ptr_equal(void* a, void* b);
// Another way of writing munit_assert_ptr(a, ==, b)
munit_assert_ptr_not_equal(void* a, void* b);
// Another way of writing munit_assert_ptr(a, !=, b)
munit_assert_null(const void* ptr);
// Another way of writing munit_assert_ptr(ptr, ==, NULL)
munit_assert_not_null(const void* ptr);
// Another way of writing munit_assert_ptr(ptr, !=, NULL)
munit_assert_true(bool value);
// Check that the boolean value is true.
munit_assert_false(bool value);
// Check that the boolean value is false.
Additionally, µnit contains a munit_assert() macro, which is similar to assert but uses munit's logging facilities, for those cases where more specialized macros will not work.
Examples
Nuances
Just like in our custom test cases, there are some auto-populated environment variables you can use in your Unit tests:
Magic Environment Variables
__M_TEST -> a unique identifier for the test case
__M_PROJECT -> a unique identifier for the project
__M_SUBMISSION -> a unique identifier for the submission
__M_USER -> a unique identifier for the user
__M_EMAIL -> the user's email
__M_SUBTIME -> the time of the submission, in seconds
While you're in the test case create/edit view,
these variables will appear to be slightly different
(but for student submissions they will be filled correctly!):
__M_SUBMISSION -> 'INSTRUCTOR_SUBMISSION'
__M_USER -> 'INSTRUCTOR_USER'
__M_EMAIL -> 'INSTRUCTOR_EMAIL'
__M_SUBTIME -> the time that the project was created