Numbas is an open-source system developed by the e-Learning Unit of Newcastle University's School of Maths and Stats, based on many years of use, experience and research into e-assessment.
It's aimed at numerate disciplines.
Everything runs on the client - no backend server, even for maths.
It creates SCORM-compliant exams which run entirely in the browser, compatible with VLEs such as Blackboard and Moodle.
Numbas follows the CALM model.
At Newcastle, we used the commercial system i-assess for six years before switching to Numbas.
Development began in 2011 with the aim of replacing i-assess.
An exam consists of several questions.
A question consists of a statement, one or more parts, and an advice section.
Questions are randomised according to a set of variables, defined by mathematical expressions.
A part can provide steps, which give a hint or break the part into smaller steps.
Part types: number entry, multiple choice/response, text extry, mathematical expression.
Numbas uses the SCORM 2004 standard to integrate with compliant VLEs, such as Moodle and Blackboard.
Or you can use it without a VLE.
Computer-aided assessment is great for formative assessment.
Students can try randomised questions over and over until they're happy.
Summative assessment poses problems:
Numbas is used at Newcastle to assess undergraduate-level maths.
Computer-assisted assessment is most useful around the transition to university.
A few final-year modules use Numbas to provide an opportunity for practice at algorithmic tasks.
As it exists today, Numbas isn't a great fit for primary education.
Numbas runs entirely on the client, so it's fast - no calls to external computer algebra systems - and completely standalone.
But doing everything on the client raises the possibility of cheating!
At Newcastle, we wrap it up in a standalone app, meaning students can only cheat in the traditional ways.
Syntax must be simple, memorable and unambiguous.
Numbas uses a syntax similar to a calculator or other computer algebra systems.
The student's answer is rendered next to their input as they type.
Biggest error is omitting multiplication symbol between variable names.
The interface is clean, and accessible by screen readers. Feedback is immediate.
Over several years, and based on student feedback, the interface has improved.
Using the graphical editor, you can create a simple question with no previous knowledge.
Gradually add more complicated elements, such as LaTeX.
Can test run questions immediately.
Template questions make customising existing material easier.
Most high school or undergrad maths questions are easily implemented in Numbas.
Randomising questions can be hard, and marking can be hard.
Variables are generated declaratively; variables can build on other variables.
The definition interface allows you to work interactively: see generated values immediately, and test for properties.
Default marking algorithm is conceptually simple - compare student's answer and correct answer on a random selection of points.
Doesn't assess form - but Numbas has a sophisticated pattern matcher.
Acyclic directed graphs, like STACK, get some of the way there but are both cumbersome and limited.
Numbas uses MathJax to render mathematical notation.
Substitution of random variables into mathematical expressions is complicated.
Numbas has a customisable system of simplification rules to clean up mathematical expressions before they're rendered.
A large driver for Numbas was the lack of customisability in previous systems.
Interface and logic are completely separated in Numbas - custom themes can change the look of tests, or reimagine how they're run.
Extensions allow the addition of new functions, data types, and resources.
Compiled Numbas tests are SCORM packages: they're completely self-contained. Perfect for open access resources.
We want to establish a community of authors and users producing quality open-access material.