Evaluator Competencies Reflection

 My studies in Learning Design and Technologies have recently taken me to a personally unexplored territory called Evaluation of Learning Systems. Something I had never even thought of as an established field before, nonetheless did my research on, Evaluation is giving me a run for my money. As I dive into this unexplored profession, I am coming to realize the relevance and importance of having experience in evaluation as a learning designer and beginning to appreciate the established norms of evaluation. 

Part of this journey has called upon me to initially assess the current state of my evaluator skills, based on the American Evaluation Association’s Evaluator Competencies, I have been tasked with rating myself on a scale of 1 to 6, with 6 being expert, on my standing with these competencies. Reflecting on how I have rated myself, I have come to realize that I have a long way to go when it comes to evaluating. 

AEA’s Evaluator Competencies Professional Practice Domain is one of my areas of strength, yet I only rated myself a 1.5 average for these competencies. Skills and attitudes included in this domain that I feel I possess are acting ethically, using evidence to make judgments, reflecting on evaluations to improve practice, identifying needs for professional growth, and pursuing professional development. Though, I would argue that these attributes are necessary in any profession. However, competencies such as applying AEA foundational documents to evaluations, selecting evaluation approaches and theories, identifying public good, and advocating for the field of evaluation are areas in which I have no experience and are very specific to the field of evaluation. While I rated myself a bit higher in this domain, it's clear that I have a lot of work to put in before I can call myself an expert evaluator. 

The Methodology domain is another of my areas of strength due to my background in chemistry, which gives me some experience with systematic inquiry. However, my experiences did not occur in the context of the evaluation of learning systems, which may have given me an advantage in this self-assessment. While many think of the words evaluation and research as interchangeable, Russ-Eft & Preskill (2009) argue that they often serve different purposes, pertain to different audiences, and communicate their findings in different ways. Therefore, the skills I possess in systematic inquiry may not serve me well in the context of evaluation, and I rated myself an average of 1.5 for competencies in this domain. 

A major domain of weakness for me would be the Context domain, as I feel as though the competencies included in this domain are unique to performing evaluations. For example, clarifying intent with stakeholders, effectively gauging the context of the organization, etc. are very particular to the evaluation profession. I have a lot to learn here. 

I scored myself a bit higher in the domain of Planning & Management, even though I feel that this domain encompasses many skills necessary for evaluation in particular. Competencies such as developing a budget, resources, and timeline for the evaluation, coordinating evaluation processes, and monitoring progress and quality are all areas of inexperience for me when it comes to the evaluation context. Many of these skills I do have experience with are in alternative contexts, however. For example, I am experienced in managing and safeguarding data, planning and documenting processes and products, and teaming with others when appropriate. 

For the Interpersonal domain, I rated myself as an average of 1 on the corresponding competencies, which include fostering positive relationships, understanding different perspectives, building trusting relationships, communication, and constructive interaction. Many of these soft skills I have developed already through my work in education. I don’t have experience with facilitating shared decision-making, attending to effects of power and privilege, or facilitating culturally responsive interaction in the context of evaluation, however. 

Overall, I would rate myself a 1 out of 6 in my effectiveness as an evaluator based on AEA’s Evaluator Competencies. I currently have a lack of experience both witnessing programmatic evaluations and being a part of them in my professional career, significantly contributing to this low score. While I feel more confident in the domains of Professional Practice and Methodology, I struggle with the Planning and Management, Interpersonal, and Context domains. 

Upon reflecting on the competencies required of evaluators, some surprised me. The Context domain is surprising because I never realized how significantly it can impact an evaluation. Evaluations can take on various forms, whether it be developmental, formative, or summative, and they will vary across the organizations they are implemented for as culture plays a large role in the outcomes of an evaluation. Interpersonal competencies also surprise me in their importance of making people who are a part of the organization feel at ease, despite the negative perceptions that evaluations might take. 

I have come to recognize that, much like the field of instructional design, evaluation requires a diverse skill set and that instructional design and evaluation are inseparably intertwined due to the fact that evaluation is a required area of expertise for instructional designers. To gain more competencies as an evaluator, I am looking forward to enhancing my skills throughout this course and engaging on a more knowledgeable basis in the professional field of evaluation in the future. I understand that this will make me a more effective learning designer in the long run and I can’t wait to dive in!