Holistic competency is an umbrella term for different types of generic skills, positive values and attitudes (Chan et al, 2017). It can be applied to a wide range of disciplines across academic and professional contexts. Assessment is an important part of teaching and learning. Educators are often eager to know whether they are teaching what they think they are teaching, whether students are learning what they expect them to learn, and what they can do to improve their teaching.
The assessment of holistic competency is important because it:
-
Informs teachers on their students’ holistic competency development
-
Help students to monitor their own holistic competency development
-
Drives students to develop holistic competencies (Medland, 2016)
-
Allows universities to provide evidence of graduate attributes which often include these competencies (Salas Velasco, 2014)
-
Informs employers on the quality and job readiness of graduates (Cummings, 1998)
Debates on Holistic Competency Assessment
The title of one of our journal articles, “To assess or not to assess holistic competencies.” (Chan & Yeung, 2021), implies the debates surrounding the assessment of holistic competency that have been around for a long time (de la Harpe & David, 2012; Gibbs, 2014). There are a variety of reasons including the subjective nature of holistic competencies, no base-line measurements, and the moral principles for assessing certain competencies.
Holistic competency assessment is challenging because there is often no explicit holistic competency learning outcomes in curriculum documents, and thus, no explicit assessment.
The lack of well-established and systematic way to assess these competencies is also a big issue. This problem is often exacerbated by the absence of a common understanding of holistic competencies (Chan, Fong, Luk & Ho, 2017), which makes the development of valid and reliable assessment challenging.
In fact, many holistic competencies may not be amenable to assessment and cannot be easily represented or rated on scales (Badcock et al., 2010; Chan, 2012) due to their nature and properties. For example, moral values such as consideration and respect are humane, and can neither be quantified nor assessed objectively. The personal nature of these competencies is also likely to lead to negative emotions (e.g. frustrations, defensiveness) (Chan & Luo, 2020; Chan & Luo, 2021) when negative feedback is given (Chan & Yeung, 2020).
Even if holistic competencies are assessed, there are arguments concerning whether and how to give students credits for their holistic competency development (Pitman & Broomhall, 2009). In a study conducted by Chan & Luk (2021), the common view among 2150 undergraduates and 215 academics in Hong Kong is that they were somewhat positive towards formal assessment of holistic competencies, but were against the use of quantitative scoring (see Figure 1 – out of a 5-point likert scale). Instead, they prefer having a ‘grade-free’ qualitative record of their holistic competency development, including but not limited to reflective pieces and comments from peers and work supervisors based on observations.
Teachers' Versus Students' Perception
of Holistic Competency Assessment
Assessment Literacy of Holistic Competency: in Teacher and Student
Assessments provide information on student learning, the effectiveness of pedagogy as well as direction for future development. Thus, teachers and students must obtain levels of understanding on the assessment in terms of its principle and practice (Taylor, 2009). Currently, assessment literacy research has largely been based on disciplinary knowledge, Dr. Cecilia Chan and her team were the first to investigate student and teacher assessment literacy in holistic competency (Chan & Luo, 2021; Chan & Luk, 2021).
Dr. Chan believes that “assessing holistic competencies needs to be addressed by developing academics’ understanding and expertise in assessment and involving students in the assessment process and encouraging them to take a more active role in directing and reflecting on this own learning. Establishing holistic competency assessment literacy in students and teachers is a way of raising awareness and creating commitment in students and teachers to the recognition of its importance.
Student Assessment Literacy
Expanded on assessment literacy developed for disciplinary knowledge (Smith et al., 2013), student assessment literacy for holistic competency development consists of four dimensions: knowledge, attitude, action, and critique (Chan & Luo, 2021). In the knowledge dimension, students a) understand the purpose of assessment for holistic competency development and the potential side effects (such as compromising learning agency and unfair judgement) assessment may cause, b) understand the processes of holistic competency assessment are not always explicit (i.e., hidden in the curriculum or inferred in other assignments) and could be embedded within academic tasks, c) understand holistic competencies can be assessed via different approaches (e.g., reflective writing, presentations) and activities (e.g., community service, field tris), and assessment approaches, activities and holistic competency development are connected. The knowledge dimension serves as a threshold to achieve a deeper level of assessment literacy.
Students’ attitudes influence motivation to engage in learning; students’ perceived importance of holistic competency development affect their motivation to enhance these competencies (Chan & Yeung, 2019). In the attitude dimension, assessment literate students appreciate holistic competency development in terms of its value and show a willingness to engage in its assessment and constructively manage the impact of holistic competency assessment on emotions and avoid defensiveness when receiving negative feedback. Critical feedback can be positive or negative depending on students’ emotional competency, such as self-efficacy and emotion handling; emotionally competent students often reflect on themselves, seek further suggestions from peers and teachers.
In the action dimension, assessment literate students develop strategies for different assessment tasks, reflect intentionally, and judge and use assessment and feedback for further holistic competency development. As in assessment literacy for disciplinary knowledge, students’ ability to reflect and make judgements on one’s performance encourages continuous development of the desired learning outcomes (Smith et al., 2013).
In the final critique dimension, assessment literate students recognize that they have the right to challenge and critically examine holistic competency assessment and the feedback provided and proactively and critically engage in dialogues with peers and teachers to improve holistic competency assessment. Students need to understand that they have the right to critique the assessment and feedback provided and take control of their holistic competency development. Opportunities must be created for students to share their insights and concerns to improve the overall assessment mechanism. In sum, students who are assessment literate in holistic competency continuously create, evaluate, analyze, and critique their actions, attitude, and knowledge.
A Four-Dimensional Framework on Students’ Assessment Literacy in Holistic Competencies
Note. Reproduced with the permission of the authors – Chan, C. K. Y. and Luo, J. (2020). A four-dimensional conceptual framework for student assessment literacy in holistic competency development. Assessment & Evaluation in Higher Education.
Teacher Assessment Literacy
Student assessment literacy in holistic competency values student-teacher partnership by emphasizing assessment co-design in the final critique dimension, which places students in a proactive position of acquiring holistic competency outcomes. On the other side of the partnership, holistic competency development requires teachers to understand the complex nature of holistic competency and its assessment. Chan and Luk proposed a four-dimensional model for teacher assessment in holistic competency (Chan & Luk, 2021), including knowledge, attitude, practice, and socio-emotional management.
In the knowledge dimension, assessment-literate teachers have knowledge of holistic competency and its assessment methods. They should possess a clear understanding of the competencies to be assessed, the ability to articulate expectations, the ability to select relevant and practice methods, and the ability to weigh the pros and cons of different methods. In the attitude dimension, assessment-literate teachers need to be open-minded towards the assessment of holistic competencies, be willing to change pre-existing beliefs on assessment, and be receptive to new assessment methods. They also need to possess criticality in making assessment decisions for holistic competencies, including carefully considering how holistic competencies should be assessed and being selective of which holistic competencies are under assessment.
In the practice dimension, assessment-literature teachers need to achieve constructive alignment, provide feedback to students, and engage students in self-reflection and self-assessment. Constructive alignment is achieved by developing a common understanding of assessment criteria between teachers and students, encouraging goal-setting and self-regulated learning, and using observable assessment criteria. Feedback to students should integrate opinions from different stakeholders, and teachers should be careful of when and whether feedback provision is appropriate and feasible. Lastly, teachers should use learning artefacts to stimulate student self-reflection and rubrics to facilitate student self-assessment.
In the final socio-emotional management dimension, teachers need to be aware of ethical dilemma management, social management, and emotional management. First, assessment-literate teachers need to understand the inherently subjective nature of holistic competency performance, embrace the subjective judgements of such assessments, and aim to reward effort instead of performance. Second, in terms of social management, assessment-literate teachers build positive teacher-student relationships and give personal attention based on longitudinal observations. Lastly, in terms of emotion management, assessment-literate teachers are sensitive to how assessment and different forms of feedback impact students’ feelings and motivation. Thus, it is important to maintain a good rapport to lower students’ defensiveness, especially when providing critical feedback.
A Four-Dimensional Framework on Teacher’s Assessment Literacy in Holistic Competency
Note. Reproduced with the permission of the authors – Chan, C. K. Y. and Luk, L.Y.Y. (2021). A four-dimensional framework for teacher assessment literacy in holistic competencies. Assessment & Evaluation in Higher Education.
References
Chan, C. K., E. T. Fong, L. Y. Luk, and R. Ho. 2017a. “A Review of Literature on Challenges in the Development and Implementation of Generic Competencies in Higher Education Curriculum.” International Journal of Educational Development 57: 1–10. doi:10.1016/j.ijedudev.2017.08.010.
Chan, C. K. Y., & Luk, L. Y. Y. (under review). A four-dimensional framework for teacher assessment literacy in holistic competencies. Unpublished Manuscript, Faculty of Education, University of Hong Kong, Hong Kong. 29.
Chan, C. K. Y., & Luo, J. (2021). A four-dimensional conceptual framework for student assessment literacy in holistic competency development. Assessment & Evaluation in Higher Education, 46(3), 451–466. https://doi.org/10.1080/02602938.2020.1777388
Chan, C., & Yeung, N. C. J. (2019). Students’ ‘approach to develop’ in holistic competency: An adaption of the 3P model. Educational Psychology, 40(5), 622–642. https://doi.org/10.1080/01443410.2019.1648767
Chan, C. K. Y., Y. Zhao, and L. Y. Y. Luk. 2017b. “A Validated and Reliable Instrument Investigating Engineering Students’ Perceptions of Competency in Generic Skills.” Journal of Engineering Education 106 (2): 299–325. doi:10. 1002/jee.20165.
Smith, C., Worsfold, K., Davies, L., Fisher, R., & Mcphail, R. (2011). Assessment literacy and student learning: The case for explicitly developing students “assessment literacy.” Assessment & Evaluation in Higher Education - ASSESS EVAL HIGH EDUC, 38, 1–17. https://doi.org/10.1080/02602938.2011.598636
Taylor, L. 2009. “Developing Assessment Literacy.” Annual Review of Applied Linguistics 29: 21–36. doi:10.1017/ S0267190509090035.