Samiha Haque | Danula Hettiachchi

Samiha’s PhD research investigates how reliably Large Language Models (LLMs) can mark student work and provide meaningful feedback (Haque et al., 2026). Using real student assessment data, her research examines how well AI-generated marks and explanations align with human markers, assessment rubrics, and different levels of learning complexity. The research also explores whether improved prompting and model designs can produce feedback that is accurate, understandable, and trustworthy, with the aim of supporting responsible use of AI in educational assessment.

Samiha Haque is supervised by Dr. Danula Hettiachchi and Dr. Daniel Beck.

Automatic grading with Large Language Models (LLMs) offers scalable assessment but raises concerns about reliability and faithfulness. We study whether LLMs truly evaluate student answers when provided with rubrics and key elements. We compare standard prompting with our rubric-guided approach that decomposes grading into key-element identification, followed by rubric-guided score aggregation and rationale generation. Experiments on the ASAP and SciEntsBank datasets show improved agreement with human grading for a large scale LLM but degraded performance for a smaller model. We also conducted interpretability case studies by experimenting with rationale generation and input token attribution methods. We found that score predictions are primarily driven by instruction and formatting tokens rather than the student response or rubric content. Generated rationales also frequently contain hallucinated rubric elements to justify incorrect scores. These results reveal that while rubric-guided prompting can result in better grading, further work is required in interpretability, which is fundamental in real world assessment scenarios and deployment of LLMs in education in general.

References