Tutorial

This tutorial explains how you can use this Haskell autograder to grade assignments programmed in the Haskell language. This grader works in the context of the Prairielearn framework. This framework is extensible by Docker containers and that is how this grader is supplied.

This tutorial is meant for instructors who want to add Haskell assignments. It supposes that you know the relevant parts of the Prairielearn instructor guide. It explains only the functionality that is particular for this grader.

We illustrate the Haskell autograder with an example, a question in directory a_haskell_question. The contents of the questions directory is:

+ questions
  + a_haskell_question
  - info.json
  - question.html
  + tests
    - test.py
    - ValidCheck.hs
    - InvalidCheck.hs
    + input
      - input1
      - input2
      - empty
      - ...
    + expected
      - answer1
      - answer2

The relevant part of the info.json file is:

{
  ...
  "gradingMethod": "External",
  "externalGradingOptions": {
    "enabled": true,
    "image": "gjgiezeman/grader-haskell",
    "entrypoint": "python3 /grade/tests/test.py",
    "timeout": 30
  }
}

We have to set the gradingMethod to External and, in the options, the image to the value that is displayed. This points to the docker image. For the entry points, you must know that our directory tests will be available under /grade/tests in the image, and that a python interpreter can be used. The file test.py will be described next. You should set timeout (seconds) to a value that works in your environment.

The Haskell autograder supplies a module hsgrader which you should use in your own python script, here with the name test.py:

#! /usr/bin/python3

from hsgrader import GraderBase, Equality

class Grader(GraderBase):
    def tests(self):
        with  self.build("valid-input-check","ValidCheck.hs") as prog:
            prog.run_and_check("valid1",["input1"],"answer1",points=3)
            prog.run_and_check("valid2",["input2"],"answer2",points=3)
        with self.build("invalid-input-check","InvalidCheck.hs") as prog:
            prog.run_with_status("invalid1",["non-existant"])
            prog.run_with_status("invalid2",["empty"])
            prog.run_with_status("invalid3",["negative-number"])
            prog.run_with_status("invalid4",["not-a-number"])
            prog.run_with_status("invalid5",["not-on-separate-line"])
            prog.run_with_status("invalid6",["too-many-numbers"])

g = Grader()
g.start()

The hsgrader module supplies a class GraderBase. The idea is to derive a class from that (Grader, in the example), instantiate it and call start on it. This will, after setting things up and before reporting back to Prairielearn, call the method tests, which you should supply. In this method we can use methods in GraderBase for building and running programs.

The method build builds a program. Its first parameter is a name that is passed to Prairielearn if the building fails. The second parameter is the name of the Haskell file that contains the main function. This file may be located in the tests directory or be supplied by the student. The return value of this method can be used in a with-statement. The variable prog is an instance of class Program.

The method run_with_status runs a program that was built before and checks its exit status. It awards a point if the exit status is correct. The first parameter is a name that is passed to Prairielearn to identify the test run. The second parameter is a list of strings, which will be passed as command line parameters to the program. In this example, the program expects one parameter.

The method run_and_check runs a program and compares the output with the contents of a file. But it first checks the exit status, and only awards the points if both tests pass. The first parameter is again a name for Prairielearn and the second are the command line arguments. The third parameter is a filename with the correct answer. This file should reside in tests/expected and the name is relative to this directory. The keyword parameter points states the number of points that is earned for a correct answer. The default is 1, which we used in the other runs.

If the building of the program fails, the two run-methods do no more than register that the points for the run are not awarded.

In file question.html we have an element:

  <pl-file-editor file-name="Answer.hs" >
    module Answer(
    sumLines
    ) where

    import System.IO

    sumLines :: String -> IO (Either String ())
    -- your code here
</pl-file-editor>

We expect the students to write a module Answer with a function that takes a path as parameter. The function should read the file, do some computations, write results to standard output and return Right (), or, if something goes wrong, return Left msg with a message.

Finally we show the contents of ValidCheck.hs:

import qualified Answer
import GradingUtils
import System.Exit
import System.Environment
import System.IO

main = do
    args <-  getArgs
    status <- Answer.sumLines ((input_path . head) args)
    case status of
        Left msg -> die msg
        Right () -> return ()

We see that the main function calls the student function and checks the return value. The important thing to note is that you can me use of the module GradingUtils which has one function: input_path. This function will translate a filename in tests/input to a path that can be used for reading this file. Here we apply that function to the first command line argument that was passed to the program.