Small Tweaks for Better Short Answer Questions

Short answer questions (SAQs) are the workhorse of knowledge assessment in VET. But they are often under-designed and loosely contextualised. 

A short answer question sits somewhere between a single sentence and a few paragraphs, up to about half a page. Beyond that you are into essay, report or project territory. Used well, they are excellent at getting students to summarise ideas, explain concepts and synthesise information to gather evidence against multiple Knowledge Evidence criteria efficiently, or gather sufficient evidence of complex criteria in a single question. Used ineffectively, they create a lot of manual work for assessors with little advantage over auto-markable question types in terms of evidence quality. 

Use them for knowledge, not skills 

Knowledge evidence must be assessed explicitly through oral or written questioning — it cannot be inferred from performance alone. That is the job short answer questions exist to do. 

The corresponding rule is the one most often broken: short answer questions generally cannot assess skills. Asking a student “how would you do this?” or “what would you do in this situation?”, then mapping that question to a performance criterion requiring observation, is not valid evidence of a skill. 

There is a narrow exception. Where the performance criterion describes a thinking skill rather than something done with the hands (verbs like identify, compare, interpret, evaluate, examine, recognise) a written SAQ question may be a valid way to assess it. 

SAQ task design 

Writing good questions is vital, but just as important is the design and timing of the assessment task itself. A few principles to keep in mind: 

• Assess only what you have delivered. Assessing students on knowledge they have not been taught or trained detracts from the Fairness of an assessment task. Building the assessment first, then developing the learning content to match, can mitigate this issue. 

• Get the timing right. Knowledge tasks before a practical confirm readiness and safety. Knowledge tasks after a practical confirm understanding of the knowledge underpinning a practical skills (‘Why did you do it this way?’). 

• Keep SAQ tasks achievable. SAQ tasks are time consuming to complete. One 50-question mega-exam is demoralising for students, slow to mark, and is cumbersome when a resubmission is needed. A series of smaller tasks are more engaging and easier to manage for both students and assessors. 

• Consider splitting question types. Multiple choice, matching and labelling can be auto-marked; short answers cannot. Separating them into distinct tasks is often more efficient than a half-auto, half-manual hybrid. 

SAQ question design 

These are just some of the principles to keep in mind when writing individual SAQs: 

1. Is an SAQ the right question type? SAQs require more student time and assessor time. If you are testing recall of specific facts or figures, a multiple choice or labelling question does the job better. Save short answers for explanation, comparison and synthesis. 

2. Never restate the unit wording. Turning a performance criterion or knowledge evidence dot point into a question by bolting “how would you…” onto the front is poor practice. Your questions should be contexualised to your cohort. 

3. Use question verbs consistently. If “describe” means one thing in question three and something quite different in question eleven, students and assessors may interpret requirements inconsistently. Publish a key — in the task instructions, assessment policy or student handbook — defining what each verb requires in length, detail and type of response. 

4. Write to the right AQF level. Two units within a training package may contain similar knowledge evidence criteria while sitting within substantially different qualifications. What ‘knowledge’ means varies considerably by AQF level, so even though the KE may be very similar, the questions should be different. That is, it’s generally not appropriate to re-use a Certificate IV question at Advanced Diploma level, even if the wording of the mapped KE is similar.  

5. Don’t assume one question per KE is enough. While an advantage of SAQs is the ability to gather more complex (Valid) and deeper (Sufficient) evidence with a single question than can be gathered by a multiple choice question or other auto-marked question type, a single knowledge evidence dot point may contain multiple concepts that cannot necessarily be assessed by a single question (in a way that meets the Principles of Assessment and Rules of Evidence, at least). This is especially true beyond Certificate III level. Carefully map your questions against unit requirements – multiple questions might be required to address a KE in full. 

Assessor Guidance 

Assessors are commonly handed a model SAQ answer and nothing else. This guidance isn’t always enough. For example, if a model answer covers five ideas, does the student need to cover all five? Can any of the ideas come from outside the given list? Assessors may interpret requirements differently, and the consistency of assessment decisions may suffer. Always pair the model answer with a decision-making rule and restate the question’s requirements explicitly in the assessor guide even when they seem self-evident from the question itself. 

A note on AI 

In terms of SAQs, AI can have benefits and pitfalls (just like most other areas!). AI is genuinely useful for generating question pools you can randomise, adjusting a question’s difficulty to a different AQF level, and reformatting question banks for LMS upload. The RTO remains accountable for anything AI gets wrong, so treat its output as a first draft to be polished, never as finished work. 

Unfortunately, short answer tasks are among the most AI-vulnerable assessment tasks, and detection tools are unreliable. But there are things you can do mitigate the misuse of AI, like chunking tasks, conducting post-practical competency conversation, and using invigilated or in-person tasks where needed. 

Want more information? 

This article skims the surface. The full recording, Short Answer Questions: Designing Knowledge Assessment That Lifts Quality, goes into more detail with side-by-side examples of poor and better question wording, a worked comparison of the same knowledge topic assessed at two different AQF levels, a practical formula for structuring effective questions, sample assessor guidance showing decision-making rules in action, and a curated set of further reading. 

Purchase the recording, or unlock this session and our entire library with the VETNexus PD Passport — unlimited access to every webinar, live and recorded, with certificates of attendance that count toward your professional development requirements.