Evaluator Competencies Reflection #2

    Last month I wrote about a new field of exploration for me: the world of evaluation of educational programs. I’m grateful that the Learning Design and Technologies graduate program I’m in at Arizona State University is so diverse and realistic in terms of real-world applicability as to include this as a required course. Truly, I have learned so much and I’m excited to share with you what skills I have gained from my most recent coursework. If you’ve read my last blog post, you’ll know that I completed an initial self-assessment of my evaluator skills based on the American Evaluation Association (AEA) Evaluator Competencies. The perceptions I had about my evaluator skills at that time were quite low. However, while I am certainly no evaluation expert at this time, I am happy to report that I have shown some impressive growth. I’ve also gained a new appreciation for the work of evaluators and the value of evaluation as a whole. This time around I rated myself a 3.5 on the Evaluator Competencies Self-Assessment, a 2-point difference compared to my initial rating. 

    While the progress that I have made in this evaluation course is clear, I find it interesting that my self-perceived areas of strength have shifted. Previously, I felt most confident in my skills in the Planning and Management and Methodology domains. As an individual with a background in chemistry, this made sense to me. I’m a very left-brained individual who finds joy in structure and prescribed methods (shocking, I know). I tend to be highly detail-oriented and methodical, which is probably why I volunteered for many of the Evaluation Methods sections of our group project for this class. But despite more analytical tasks coming naturally to me, it seems I have shifted my evaluation skills to the Context domain. In particular, I find that I am easily able to define the educational program and its impact both internally and externally, communicate evaluation processes and results effectively, and foster a shared understanding of the evaluation with stakeholders. However, I think I am still working on developing the skill of engaging a diverse range of stakeholders. 

    The Interpersonal domain is where I scored myself lowest upon reflection of what I’ve learned recently. I rated myself high on supporting shared decision-making in the evaluation process, but I felt that I am still a novice at considering alternative perspectives, understanding the impact of power and privilege in evaluation, and communicating in a constructive and culturally responsive manner. Since my graduate program is online and does not require students to perform an actual program evaluation, it makes sense that I scored myself lower here, as I don’t have much experience working interpersonally with stakeholders. However, I do feel that the context of the evaluation content I encountered throughout this course was very realistic, and I rated myself much higher for this competency than I did at the beginning of the class. 

    Out of all the competencies included on the AEA’s list, I still find myself surprised about a few. Specifically, competencies 1.8, 2.12, and 4.4. Competency 1.8 refers to how evaluation practice can promote social justice and public good. Unfortunately, I haven’t had the opportunity to see that just yet from my current set of experiences. I am also a bit hesitant to agree that I am competent in identifying the strengths and limitations of evaluation designs and methods, as I’ve only had the opportunity to learn about evaluations throughout this short course. Competency 4.4 relates to planning for an evaluation’s use and influence. While I can imagine how the outcomes of an evaluation might be used, I am not yet confident in being able to recognize potential influence on a wide scale. Of course, these are all competencies that a skilled evaluator would possess. 

    A learner can’t expect to learn everything there is to know about a particular subject in seven and a half weeks, let alone seven and a half years. I’m satisfied knowing that I took something valuable away from this course and I feel more confident than ever in my knowledge and skills related to the evaluation of educational programs. 

    One of the first questions asked of students in this course was about their previous experiences with evaluation. My answer included a statement about how when I worked for Coconino Community College in Flagstaff, Arizona in recent years, I recall there being a sort of frenzy around campus when Quality Matters was getting involved in the production of our online courses. This was my introduction to evaluation, and I believe that my involvement was the best introduction I could have received. I am a believer that our experiences have a profound impact on our learning – something I strive to provide my students in everyday practice. While I don’t think a career in program evaluation is in my future, I look forward to learning more about how I can contribute to the endeavors of evaluations in my work. That close interaction will surely allow me to be able to grow my own personal evaluation skills so that one day I may not be such a novice. In fact, I have learned that stakeholder exposure to evaluation is one of the goals of skilled evaluators. Either way, I am super grateful to have had this exposure to the field and I know that it will serve me well as an educator and instructional designer in the future.