Code quality tests can be very useful to help students adhere to certain coding standards/cleanliness. We recommend only using these test cases when you have explicitly told your students you will be grading for code quality, as well as giving them resources to improve their code quality on their own.

We use a variety of linters and other code quality analysis tools for each language to generate a quality score for your student's code. Below is the list of tools currently used in our analysis per language:

As of right now, only the above languages are supported for this test case type. If your language is not supported above, please contact us via Intercom or Email and we will try to accommodate you!

Since we support a variety of linters for a variety of languages, the options for this test case can vary. Here are some links that should be helpful for decoding the options and their meaning:

Check out the PMD Rulesets - to add new rule sets, find the xml files here and prepend the filename with a java- , then add it to the comma separated list of rule sets to check. The default config is as follows:

We preload the default config for pylint for Python code quality tests. The documentation for all of these configuration options and more can be found here!
The pylint versions used differ based on the language of the question/project you have specified:

For C and C++, we use Google's cpplint. The documentation for this script isn't the best, but you can check the source for some of the error groups, or just run it against your code to get a feel for what errors it will throw. The default config is as follows:

We preload a default config for Scalastyle in Scala code quality tests. If you're interested in more configuration options, you can find more information here!

With the code quality test case setup, it is quite simple to make a new analysis of each student's code. To set up a code quality test case, you need to select a "main file" to be analyzed, a threshold for what code quality score passes the test case, and the input field can contain any other arguments for the linter that you'd like to include. The input field will automatically load in the default arguments for the linter that we recommend.

main file name (file to be analyzed in the code quality test case, required)
input (extra arguments to be passed into the linter to modify scoring)
threshold (a code quality score that is the minimum to receive credit)
show output (toggle this on if you want the students to be able to see the linter's output)

A good threshold value for your project can vary greatly, based on your project and it's complexity. A very safe (also very forgiving for submissions) threshold value to start with is 50, but this is far from perfect. It is advised to experiment with the threshold when integrating a code quality test case into your projects.