Some recent developments in Numbas

Innovative mathematics e-assessment in the STEM subjects

Christian Perfect, Newcastle University


Numbas is an open-source system developed by the e-Learning Unit of Newcastle University's School of Maths and Stats, based on many years of use, experience and research into e-assessment.

It's aimed at numerate disciplines.

Everything runs on the client - no backend server, even for maths.

It creates SCORM-compliant exams which run entirely in the browser, compatible with VLEs such as Blackboard and Moodle.

Formative online tests

We've created a wiki to accompany our maths support centre, Maths-Aid. It contains worked examples in text and video form, but also Numbas tests.

Students can run randomised example questions very easily.

Integrated with the rest of the content - links to videos, references.

Template questions

Several different subjects teach the same material, so they want to reuse existing questions.

But subject-specific context is very helpful.

We wanted to make "template questions" that can easily be customised to suit a particular subject.

Template questions

Example question

Marking algorithms

Numbas has in-built algorithms to mark certain kinds of questions: multiple choice; number entry; mathematical expression.

These are good enough for a surprisingly large number of questions.

How do you cater for the annoyingly large number of questions that don't fit the standard schemes?

How do we retain our sanity while doing so?

Plan A:
Come up with an all-encompassing abstraction

The abstraction can be hard to get your head around.

And you're still limited to the built-in tools.

The kind of person who worries about this is almost definitely willing to do some programming.

The kind of person who isn't worried about this will probably hack a work-around with the built-in tools.

Plan B:
Just write your own code!

But marking algorithms are complex.

We want to make life at least a little bit easier than "write your own e-assessment system"

Plan B:
Just write your own code!

Plan B:
Pre-process the student's answer

I added the ability to run custom JavaScript at the start of a Numbas question. We abused this with "hidden gapfills".

I even convinced Geogebra to play along with this method.

Downside: requires an unacceptable amount of "plumbing" by the question author.

Plan C:
Write your own code, but reuse the existing algorithms

Numbas question parts now have fields for custom marking and validating code. You can use the standard algorithm as just one step of a custom algorithm.


A few patterns come up repeatedly - how do we avoid having to repeat code?

Marking mathematical expressions

Numbas marks a mathematical expression by evaluating the student's answer at some randomly-chosen points, and comparing the result with a model answer.

This says nothing about the form of the answer.

What is "correct"?

In increasing order of difficulty to mark automatically:

Assessing the form of an answer

Rewriting into a canonical form loses information, and can still miss some valid answers.

Regular expressions can't mark a mathematical expression - mathematical notation is nowhere near a regular language.

I want something like regular expressions, but for syntax trees.

Pattern-matching syntax trees


Example question




Source code