A critical thinking rubric and hierarchical classification approach in automated essay scoring of e-learning
Fitrah Asma Darmawan, Aulia Sabril, Edy Sabara

Universitas Negeri Makassar

Abstract

This study evaluates the accuracy level of Automated Essay Scoring (AES) using a combination technique both hierarchical classification approach and critical thinking rubric in e-learning. The first hierarchical process computed by AES is to classify the answer of an essay into two categories, namely shorter essays (below threshold: <50 words) and longer essays (above threshold: ≥50 words). The second is looking for the same word or synonym based on the result of semantic similarity measurement. Lastly, to scoring that answer refers to critical thinking rubric that has been integrated into AES e-learning. Therefore, that final scores of students essays indicate the level of students’ critical thinking skills in online learning. This study employs several automated tools which are Writing Assessment Tool (WAT), Coh-Metrix, and Linguistic Inquiry and Word Count (LIWC) for analysing the accuracy level of AES. The analysing process compares the AES output scores and expert grader scores based on the critical thinking rubric. Our results reveal that AES, using a combination of hierarchical classification approach and critical thinking rubric, performs a high accuracy for assessing students’ critical thinking skills. Thus, AES can be implemented in e-learning for supporting learning goals that prioritize students to think critically.

Keywords: Automated Essay Scoring, Hierarchical Classification, Critical Thinking

Topic: Innovations in Engineering and Education