Rubrics Guided Assessment versus Traditional Assessment Methods in Evaluation of Hand-written Self-Directed Learning (SDL) Assignments in adult learners in this age of artificial intelligence (AI).
- Dr. Sona Truman , Professor, Department of Physiology, Government Medical College, Idukki, Kerala, India.
- Dr. Dhanya P. , Professor, Department of Pharmacology, Faculty at National Nodal Center, Government Medical College, Kottayam, Kerala, India.
Article Information:
Abstract:
Background: Objectives: The primary objective was to compare the scores obtained by different methods of assessment rubric-guided assessment and traditional manual assessment of hand-written SDL assignments submitted by MBBS phase I students in the department of Physiology. In this age of artificial intelligence (AI) we have to find novel methods to make sure the students complete their assignments themselves. This can be enforced better if they are asked to submit hand-written assignments instead of printed ones. Methods: It was a comparative study. We compared the scores obtained after the traditional assessment of handwritten assignments with the scores obtained after using a rubrics-guided assessment. Each hand-written assignment was evaluated by both traditional and rubrics guided methods. A total of 60 students from Phase I MBBS were included in the study. Six SDL assignments were assessed for this study. Rubrics guided assessment scores were written as R. Traditional assessment scores were written as T. Data were analyzed using descriptive statistics, Shapiro-Wilk and Wilcoxon Signed-Rank test. Results: Comparison of traditional assessment and rubrics guided assessment methods of scoring showed significant difference in four SDL assignments. The Wilcoxon Signed-Rank test showed p value of <0.001 in T1 versus R1, T2 versus R2, T5 versus R5 and T6 versus R6. Conclusion: Proper assessment of SDL assignments by rubrics guided tools make assessments more standardized, objective and transparent. It gave one-on-one feedback to students and helps them improve. Therefore, proper evaluation of SDL assignments will increase the value of SDL assignments. This can improve the quality of the assignments. SDL assignment scores are included in internal assessments.
Keywords:
Article :
INTRODUCTION:
Competency-Based Medical Education (CBME)1 was introduced by the National Medical Commission (NMC) with emphases on learner-centred, competency-driven and interactive learning. Phase I topics (anatomy, physiology and biochemistry of the Human body) lays the foundation for the whole MBBS course. Human Physiology is a Phase I MBBS subject that covers a vast number of topics. Most of the first year MBBS students have just graduated high school and come from didactic system of learning which is teacher-driven and examination focused. They are not experienced in self-learning. Children learn what they are told to learn. They can be motivated by rewards. Adult learners are not motivated by rewards. Adult learners are motivated to study what they perceive to be valuable. Adult learning must continue life-long. We should provide a safe, non-threatening environment where the learners are active contributors. The toughest challenge faced by faculty now is to help prepare students for professional practice in a technology-mediated and information –rich workplace. In this age of artificial intelligence (AI) we have to find novel methods to make sure the students complete their assignments themselves. This can be enforced better if they are asked to submit hand-written assignments instead of printed ones.
Self-directed learning (SDL)2 is a key component of the CBME curriculum introduced by the NMC as a teaching-learning method where the primary initiative is taken by the student. Self-directed learning (SDL) is one of the latest methods adopted to improve learning by the adult students. SDL is the process in which the students actively learn specific topics selected by the faculty with the help of references or resources provided and submit an assignment that is evaluated. SDLs are non-memory methods for teaching and learning. It cultivates the ability to think creatively, formulate and solve problems. SDLs cut across all domains of learning. SDL promotes learner autonomy, reflective thinking and long-term retention of knowledge. SDL assignments play a critical role in structured learning reinforcement. It should be guided by the faculty. The faculty should help the students select the topics, provide proper references and resources, give them assignments, proper evaluation of the submitted assignments, proper feedback from students and faculty, etc. The retention of a topic increases when the students are actively involved. The faculty becomes the motivator and supporter. SDLs are not having the desired effect due to less effort from both the teachers and the students. Hand-written assignments3 are submitted by the students after SDLs. This will also improve the handwriting. Writing is a time-tested method of learning. In this era of digital technology students are very reluctant to write. The hand-written assignments can be used as formative assessments. Formative assessments can be diagnostic and help learners and teachers modify the learning-teaching methods to improve the outcome.
Assessment is an integral component of any successful teaching-learning effort. A learning-teaching method is complete only after proper assessment. Assessment should be objective and transparent. Learners need to know the criteria by which they are scored. Educators who want to be good teachers should strive to be authentic assessors. Many studies have shown that objective and transparent assessment will encourage the adult students to study. Adult learner4 study what they perceive to be valuable. Adult learners need to be motivated internally. Their engagement is increased when they know their assignments are properly assessed. SDLs need to be graded properly but there is no standard method to assess them. Traditional methods of assessment can be subjective, are not transparent and may introduce variability and bias due to lack of standardized criteria. It does not take into consideration many factors like punctuality, neat handwriting, flow charts, diagrams, quality of content, etc. Rubrics guided tool5 offers a structured scoring framework with predefined criteria and performance descriptors. Rubrics improve transparency, reliability and formative feedback quality. Very few studies are there comparing different methods of assessment of hand-written assignments after guided SDLs.
Proper grading of hand written assignments make sure that the students have reliable, targeted, defined and structured notes for future reference when preparing for formative and summative exams. Many studies have shown that proper and transparent evaluation will encourage the students to study the subject more regularly. The department can prepare a Rubrics tool unique to the needs of the students, faculty, topics involved, etc. This will ensure uniformity when grading the assignments. This is important since SDL assignments can be graded for giving internal marks to become eligible for summative exams. Assessment scores give feedback to the students and help them improve their assignments. This study aims to compare rubric-guided assessment scores and traditional assessment scores. There are very few studies comparing the scores obtained by different methods of assessment of hand-written SDL assignments particularly in CBME.
Research Question
Does rubrics-guided assessment make evaluation of hand-written SDL assignments more transparent, give better feedback and increase the value of the assignment in adult learners?
Research Objectives
To compare the mean scores obtained by using rubrics-guided assessment and traditional assessment of hand-written SDL assignments.
MATERIALS AND METHODS:
It was a comparative study conducted in the department of Human Physiology at Government Medical College, Idukki for a period of 2 months.
Study Population
All the Phase I MBBS students (2025 admission batch).
Inclusion Criteria
Phase I MBBS students who are willing to participate in the study. Those students who submit all six hand-written SDL assignments.
Exclusion Criteria
Phase I MBBS students who are not willing to take part in the study. Those students who do not submit all six hand-written SDL assignments.
Sample Size Justification
Since this was a paired comparison study the formula for paired comparison:
N = (Zα/2+Zβ)2 x SD2/d2
Assuming:
α = 00.5
Power = 80%
Expected moderate effective size (0.5)
SD estimated from pilot data. Minimum required sample is about 34 students.
Sampling Technique
Considering availability of full batch, universal sampling was used to increase precision. Universal sampling was adopted and all Phase I MBBS students were enrolled. A total of 60 students fulfilled all the inclusion criteria.
Study Procedure
During the study period, all phase I students were given the six SDL assignments covering the six topics. Six exposure topics are endocrinology, central nervous system (CNS), reproduction, special senses, renal and gastro-intestinal tract (GIT). Each SDL assignment contained seven questions that they had to write and submit in one week. The hand-written assignments were collected and assessed by rubrics guided methods and traditional methods. Student identifiers were removed. Unique anonymous codes were assigned. This was repeated for six weeks.
Operational Definitions
Traditional assessment of assignments involves a teacher physically reviewing, marking and do not provide feedback to student or teachers. Rubrics guided assessment of assignment is a structured, objective and efficient method for evaluating. It breaks down complex assignments into specific criteria and proficiency level. This provides a standardized framework to ensure transparency faster. It also gives clear feedback to students and faculty.
Development of Rubric Tools
Six rubrics were developed corresponding to the six SDL topics. Each rubric6 included structured criteria: For content accuracy and clarity - 2 marks, Diagram/flow charts – 1 mark, Neatness – 1 mark, Punctuality – 1 mark. The total = 5 marks. The rubrics tool was validated after review by seniors and colleagues in the department of Human Physiology.
Assessment Procedure
Step 1: Traditional manual evaluation.
Step 2: Washout period of 2 days.
Step 3: Rubrics-guided evaluation.
Bias Control
There was blinding of student identity. There was randomized sequence of assessment methods. There was washout period of 2 days. There was a standardized rubric tool.
Data Analysis
The data was entered into Microsoft Excel and analyzed using SPSS version 26. The verification was done to avoid double entry. The data was made anonymous before analysis. There was secure password-protected storage. Data was analyzed using descriptive statistics, Shapiro-Wilk and Wilcoxon Signed-Rank test. P value <0.05 was considered statistically significant.
RESULTS:
Total of 60 students from Phase I MBBS (2025 admission) participated in this study. All the participants, hand-written SDL assignments were scored by two methods. A total of six SDL assignments were assessed for this study. Rubrics guided assessment scores are R. Traditional assessment scores are T. The scores obtained compared R1, R2, R3, R4, R5 and R6 were compared to the scores obtained by traditional methods T1, T2, T3, T4, T5 and T6.
Normality Test (Shapiro-Wilk)
A Shapiro-Wilk test was conducted on the differences between all paired T (Target) and R (Rated) values to determine if the data follows a normal distribution. W-Statistic: 0.876 p-value: < 0.001
Conclusion: The p-value is significantly less than 0.05, indicating the data does not follow a normal distribution. Therefore, the Wilcoxon Signed-Rank Test is the appropriate non-parametric alternative, confirming the reviewer's suggestion.
Table 1: Descriptive Statistics
|
Metric |
T Values |
R Values |
|
Count |
360 |
360 |
|
Mean |
4.00 |
3.71 |
|
Median |
4.00 |
3.50 |
|
Minimum |
4.00 |
2.00 |
|
Maximum |
4.00 |
5.00 |
|
Standard Deviation |
0.00 |
0.75 |
Wilcoxon Signed-Rank Test Results
Overall Test (All T vs All R): W = 11748.0, p < 0.001
Table 2: Pairwise Breakdown
|
Group Pair |
T Mean |
R Mean |
W-Statistic |
p-value |
|
T1 vs R1 |
4.00 |
3.54 |
198.0 |
< 0.001 |
|
T2 vs R2 |
4.00 |
3.39 |
161.5 |
< 0.001 |
|
T3 vs R3 |
4.00 |
4.14 |
554.5 |
0.200 |
|
T4 vs R4 |
4.00 |
4.10 |
505.0 |
0.164 |
|
T5 vs R5 |
4.00 |
3.59 |
181.5 |
< 0.001 |
|
T6 vs R6 |
4.00 |
3.48 |
250.0 |
< 0.001 |
Comparison of scores by the two methods showed significant p value in T1&R1, T2&R2, T5&R5 and T6&R6. Comparison of scores in T3&R3 and T4&R4 were not significant.
DISCUSSION:
In this present study there is comparison of traditional grading scores and rubrics guided7 grading scores of hand-written assignments. The students were asked to submit one hand-written SDL assignment which had seven structured questions based on the SLO of that topic in a week. Each assignment8 was graded by traditional methods and rubrics guided methods by the same faculty. Only one assessor was able to assess due to less faculty available at GMC, Idukki which is a difficult rural area. A total of 6 assignments were given and the results analyzed. The four SDL assignments with significant P values may be because rubrics guided assessment made the assessment objective. Subjective factors like internal assessment marks and personal bias were removed. Rubrics guided assessment scores were less compared to the traditional assessment. Traditional method of assessment9 did not take into consideration any parameters.
They gave a score of about 80% to compensate for the low scores that may be obtained in formative exams for internal assessment marks to be updated to the University. This is subjective bias. Rubrics guided assessment made sure that the assignments were submitted in time, the handwriting was neat, there were diagrams or flow charts as required and also the contents were adequate. The assessment became more objective.10 We were able to give one on one feedback to each student based on these parameters. Assessment methods that give one on one feedback are the best form of assessment. Rubrics guided assessment improved the quality11 of the assignments over time. Students submitted better quality assignments due to objective and transparent rubrics guided assessment. This also might have caused differences in the scores in latter assignments. This method makes sure that the SDL assignments are useful and they have better quality noted for preparations for formative and summative exams.12
Improving the quality of SDLs is very important especially in the medical education area where the portions are vast, limited time and poor student–teacher ratios. CBME has encouraged us to cover many topics with the help of SDLs. Therefore, proper evaluation of SDL assignments is very important. Most Universities13 in India follow summative examination in the hand written format to promote standardization,14 security and academic integrity. This is specially, important in this era of artificial intelligence (AI). Hand-written SDL assignments also help to improve the hand-writing15 and improve exam scores. This makes this study more unique.
CONCLUSION:
The findings in this study show that better utilization of SDL assignments is useful to both faculty and students to overcome many shot-comings like lack of time, vast portions and poor student-teacher ratio. Time tested traditional methods of writing notes after each lecture class is not possible in medical education due to lack of time. Therefore, SDL assignments can be utilized to help students prepare notes that can aid the preparations for formative and summative exams. Proper and transparent scoring will improve the quality of the notes prepared. There are few studies about the assessment of hand-written SDL assignments. SDL are an underutilized aspect of CBME that when used properly help to overcome time constraints due to late admission of shorter duration of course. CBME is independent of the length of time of training. This is a single institution study. This is short-term evaluation. This study does not assess long-term academic performance.
REFERENCES:
1. National Medical Commission (NMC). Competency Based Undergraduate Curriculum for the Indian Medical Graduate- Module 3: Assessment. 2019:11-21. Website accessed on 02/11/2026 via nmc.org.in
2. Orjika PC. Lessons from Self-Directed Learning Activities and Helping University Students Think Critically. Journal of Education and Learning 2023;12(2):79-87.
3. Rezaei AR, Lovorn M. Reliability and validity of rubrics for assessment through writing. Assessing Writing 2010;15(1):18-39.
4. Elton L, Johnston B. Assessment in universities: A critical review of research.’ York: Learning and Teaching Support Network (LTSN) Generic Centre 2002.
5. Luft JA. Rubrics: Design and use in science teacher education. Journal of Science Teacher Education 1999;10(2):107-21.
6. Allen D, Tanner K. Rubrics: Tools for making learning goals and evaluation criteria explicit for both teachers and learners. CBE-Life Sciences Education 2006;5(3):197-203.
7. Gibbs G. Using assessment strategically to change the way students learn. In: Brown S, Glasner A, eds. Assessment matters in higher education: choosing and using diverse approaches. A. The Society for Research into Higher Education & Open University Press 1999:1250-63.
8. Zeiser K, Scholz C, Cirks V. Maximizing student agency: implementing and measuring student-centered learning practices. American Institutes for Research 2018.
9. Dahal A, Bhat N. Self-Directed Learning, its implementation and challenges: a review. Nepal Journal of Health Sciences 2023;3(1):102-15.
10. Stevens DD. Introduction to rubrics: An assessment tool to save grading time, convey effective feedback, and promote student learning. 2nd edn. Taylor & Francis 2023.
11. Harris CJ, Krajcik JS, Pellegrino JW, et al. Designing knowledge‐in‐use assessments to promote deeper learning. Educational Measurement: Issues and Practice 2019;38(2):53-67.
12. Panadero E, Jonsson A. The use of scoring rubrics for formative assessment purposes revisited: A review.’ Educational Research Review 2013;9:129-44.
13. Singh G, Raksha. Assessing the Process of Self-directed Learning: A Guide for faculty to Assess the MBBS Students. ERA J Medical Research 2025;12(2):30-9.
14. Wolf K, Connelly M, Komara A. A tale of two rubrics: improving teaching and learning across the content areas through assessment. The Journal of Effective Teaching 2008;8(1):21-32.
15. National Medical Commission (NMC). Competency Based Undergraduate Curriculum for the Indian Medical Graduate- Module 3: Assessment 2019:11-21.