Numbas

Christian Lawson-Perfect, Newcastle University

What is Numbas?

Numbas is an open-source system developed by the e-Learning Unit of Newcastle University's School of Maths and Stats, based on many years of use, experience and research into e-assessment.

It's aimed at numerate disciplines.

Everything runs on the client - no backend server, even for maths.

It creates SCORM-compliant exams which run entirely in the browser, compatible with VLEs such as Blackboard and Moodle.

Design goals

History

Numbas follows the CALM model.

At Newcastle, we used the commercial system i-assess for six years before switching to Numbas.

Development began in 2011 with the aim of replacing i-assess.

Structure

An exam consists of several questions.

A question consists of a statement, one or more parts, and an advice section.

Questions are randomised according to a set of variables, defined by mathematical expressions.

A part can provide steps, which give a hint or break the part into smaller steps.

Part types: number entry, multiple choice/response, text extry, mathematical expression.

Integration with a VLE

Numbas uses the SCORM 2004 standard to integrate with compliant VLEs, such as Moodle and Blackboard.

Or you can use it without a VLE.

Formative vs summative use

Computer-aided assessment is great for formative assessment.

Students can try randomised questions over and over until they're happy.

Summative assessment poses problems:

Suitable topics for assessment with Numbas

Numbas is used at Newcastle to assess undergraduate-level maths.

Computer-assisted assessment is most useful around the transition to university.

A few final-year modules use Numbas to provide an opportunity for practice at algorithmic tasks.

As it exists today, Numbas isn't a great fit for primary education.

Current use

Newcastle

Current use

Elsewhere

Client-side design

Numbas runs entirely on the client, so it's fast - no calls to external computer algebra systems - and completely standalone.

But doing everything on the client raises the possibility of cheating!

At Newcastle, we wrap it up in a standalone app, meaning students can only cheat in the traditional ways.

Input of mathematical expressions

Syntax must be simple, memorable and unambiguous.

Numbas uses a syntax similar to a calculator or other computer algebra systems.

The student's answer is rendered next to their input as they type.

Biggest error is omitting multiplication symbol between variable names.

Usability

For students

The interface is clean, and accessible by screen readers. Feedback is immediate.

Over several years, and based on student feedback, the interface has improved.

For authors

Using the graphical editor, you can create a simple question with no previous knowledge.

Gradually add more complicated elements, such as LaTeX.

Can test run questions immediately.

Template questions make customising existing material easier.

Complexity of questions

Most high school or undergrad maths questions are easily implemented in Numbas.

Randomising questions can be hard, and marking can be hard.

Complexity of questions

Variable generation

Variables are generated declaratively; variables can build on other variables.

The definition interface allows you to work interactively: see generated values immediately, and test for properties.

Complexity of questions

Marking

Default marking algorithm is conceptually simple - compare student's answer and correct answer on a random selection of points.

Doesn't assess form - but Numbas has a sophisticated pattern matcher.

Can write custom marking algorithms in JavaScript, but an abstraction for marking schemes would be very useful.

Acyclic directed graphs, like STACK, get some of the way there but are both cumbersome and limited.

Rendering mathematical notation

Numbas uses MathJax to render mathematical notation.

Substitution of random variables into mathematical expressions is complicated.

Numbas has a customisable system of simplification rules to clean up mathematical expressions before they're rendered.

Customisation

A large driver for Numbas was the lack of customisability in previous systems.

Interface and logic are completely separated in Numbas - custom themes can change the look of tests, or reimagine how they're run.

Extensions allow the addition of new functions, data types, and resources.

Open access

Compiled Numbas tests are SCORM packages: they're completely self-contained. Perfect for open access resources.

We want to establish a community of authors and users producing quality open-access material.

Thanks!

Website

Contact

Source code

/

#