Tutorial
========

This tutorial explains how you can use this Haskell autograder to grade assignments
programmed in the Haskell language.
This grader works in the context of the Prairielearn framework.
This framework is extensible by Docker containers and that is how this grader
is supplied.

This tutorial is meant for instructors who want to add Haskell assignments.
It supposes that you know the relevant parts of the
`Prairielearn instructor guide <https://prairielearn.readthedocs.io/en/latest/instructor-guide/>`_.
It explains only the functionality that is particular for this grader.

We illustrate the Haskell autograder with an example, a question in directory `a_haskell_question`.
The contents of the questions directory is::

    + questions
      + a_haskell_question
      - info.json
      - question.html
      + tests
        - test.py
        - ValidCheck.hs
        - InvalidCheck.hs
        + input
          - input1
          - input2
          - empty
          - ...
        + expected
          - answer1
          - answer2


The relevant part of the info.json file is::

  {
    ...
    "gradingMethod": "External",
    "externalGradingOptions": {
      "enabled": true,
      "image": "gjgiezeman/grader-haskell",
      "entrypoint": "python3 /grade/tests/test.py",
      "timeout": 30
    }
  }

We have to set the gradingMethod to External and, in the options,
the image to the value that is displayed. This points to the docker image.
For the entry points, you must know that our directory tests will be 
available under `/grade/tests` in the image, and that a python interpreter
can be used. The file test.py will be described next.
You should set timeout (seconds) to a value that works in your
environment. 

The Haskell autograder supplies a module `hsgrader` which you should use
in your own python script, here with the name test.py::

    #! /usr/bin/python3

    from hsgrader import GraderBase, Equality

    class Grader(GraderBase):
        def tests(self):
            with  self.build("valid-input-check","ValidCheck.hs") as prog:
                prog.run_and_check("valid1",["input1"],"answer1",points=3)
                prog.run_and_check("valid2",["input2"],"answer2",points=3)
            with self.build("invalid-input-check","InvalidCheck.hs") as prog:
                prog.run_with_status("invalid1",["non-existant"])
                prog.run_with_status("invalid2",["empty"])
                prog.run_with_status("invalid3",["negative-number"])
                prog.run_with_status("invalid4",["not-a-number"])
                prog.run_with_status("invalid5",["not-on-separate-line"])
                prog.run_with_status("invalid6",["too-many-numbers"])

    g = Grader()
    g.start()

The hsgrader module supplies a class `GraderBase`.
The idea is to derive a class from that (Grader, in the example),
instantiate it and call `start` on it. This will, after setting things up
and before reporting back to Prairielearn, call the method `tests`, which
you should supply. In this method we can use methods in GraderBase for
building and running programs.

The method `build` builds a program.
Its first parameter is a name that is passed to Prairielearn if the building fails.
The second parameter is the name of the Haskell file that contains the main function.
This file may be located in the tests directory or be supplied by the student.
The return value of this method can be used in a with-statement. The variable `prog` is
an instance of class `Program`.

The method `run_with_status` runs a program that was built before and checks its exit status.
It awards a point if the exit status is correct.
The first parameter is a name that is passed to Prairielearn to identify the test run.
The second parameter is a list of strings, which will be passed as command line parameters to the program.
In this example, the program expects one parameter.

The method `run_and_check` runs a program and compares the output with the contents of a file.
But it first checks the exit status, and only awards the points if both tests pass.
The first parameter is again a name for Prairielearn and the second are the command line arguments.
The third parameter is a filename with the correct answer. This file should reside in tests/expected
and the name is relative to this directory. The keyword parameter `points` states the number of points
that is earned for a correct answer. The default is 1, which we used in the other runs.

If the building of the program fails, the two run-methods do no more than register that the
points for the run are not awarded.

In file question.html we have an element::

      <pl-file-editor file-name="Answer.hs" >
        module Answer(
        sumLines
        ) where

        import System.IO

        sumLines :: String -> IO (Either String ())
        -- your code here
    </pl-file-editor>

We expect the students to write a module Answer with a function
that takes a path as parameter. The function should read the file, do
some computations, write results to standard output and return `Right ()`,
or, if something goes wrong, return `Left msg` with a message.

Finally we show the contents of ValidCheck.hs::

    import qualified Answer
    import GradingUtils
    import System.Exit
    import System.Environment
    import System.IO

    main = do
        args <-  getArgs
        status <- Answer.sumLines ((input_path . head) args)
        case status of
            Left msg -> die msg
            Right () -> return ()

We see that the main function calls the student function and checks the return value.
The important thing to note is that you can me use of the module `GradingUtils`
which has one function: `input_path`. This function will translate a filename
in tests/input to a path that can be used for reading this file.
Here we apply that function to the first command line argument that was passed to the program.