Assessment

It’s October (already!), which brings to mind scary movies, suspense, ghosts, and goblins for Halloween. Let’s be honest, I think about the candy, but that’s a different story. I’m not a fan of scary movies or suspense, but I bet that some of you would rather watch a horror movie than work on assessment. Do you think something is going to jump out at you (the deadline is when?) or trip you up (like that log in the woods) or chase you down (is that a chainsaw I hear in the background)? Assessment doesn’t have to be scary or suspenseful. It can be as sweet as candy with the right planning. You might even think of assessment as fun if you have a creative approach.

[Read more…] about Assessment is Scary?!?!

In the assessment field, some folks emphasize the difference and conflict between assessment for accountability and assessment for improvement. I don’t see them in opposition to each other. In my mind, they are actually closely related and even complementary, especially if you can plan in advance.

Accountability can elicit a negative connotation. It typically is associated with external requirements, reporting to another entity with specific information which may or may not be seen as important by the people doing the reporting. Other people or organizations are in charge of the process. There may be very specific criteria, or there may be some general guidelines that give the unit some latitude around what they report. It may even be seen as drudgery and certainly not fun!

Improvement, on the other hand, can be seen as internally focused and in control of the person doing the assessment. The assessor decides what to assess, how to assess, when to assess, and who to assess. The process can be flexible in data collection and use. It may even be fun because there is ownership in the content and process.

Realistically, both perspectives serve a purpose. I like to think of it this way: if you are taking care of improving your programs by assessing and using results in a systematic way, then the accountability piece will typically take care of itself for the most part. Obviously, it is important to know what accountability expectations are (when reports are due, the needed information, etc.) so you are not scrambling right before some deadline trying to find information that meets the requirement. Ideally, the process you use to collect data for improvement will meet many of the requirements of accountability processes. Be strategic in planning your assessment, so that you are getting the information you need to improve and are able to use that same process in your reporting and accountability requirements.

In student affairs, we don’t often talk about benchmarking and how it can be used for improvement. I know I spend more time thinking about student learning or satisfaction as my focus, but I wanted to plant the seed that benchmarking should be in the mix of options, depending on what you need to know. (Assessment projects should always start with what you need to know.

Benchmarking can be used in a couple of ways. One, you can benchmark against a standard or baseline that you want to meet. For example, the Council for the Advancement of Standards in Higher Education has created a lot of functional-specific self-assessment guides that provide guidance on how well a functional area performs based on a set of criteria that a unit should meet. This evaluation tells you where your strengths and areas of improvement are in relation to an external standard or requirement. It can provide a comprehensive view of unit performance, leading to decisions about where to take action. This can be done fairly informally as a staff discussion or part of a more formal review process, such as accreditation application.

Benchmarking can also be used to compare one functional unit to other, similar functional units. The comparison groups could be similar units at similar institution types, similar functions outside of higher education, or units that might be considered to have best practices. For example, a health center at a large, public institution might compare themselves to other large, public institutions in the country or region. But, they also might look at similar institutions who have the same accreditation they do. Even further, they could look at local health care facilities, especially if they are looking at a specific part of their operation. Maybe the health center sees a need to improve their scheduling process to be sure they can see as many student patients as possible, as quickly as possible. They could ask local facilities about the scheduling software they use, how they schedule their providers, and what the average/acceptable wait time is between making an appointment and seeing a health care professional. Based on the information collected, the health center could implement changes that they think would work in their environment.

The process of benchmarking does not have to be overly complex. Sometimes, it’s as easy as looking at other units’ websites to find the information you need. Alternatively, it might be reaching out through your networks (listservs, professional associations, informal peer groups, etc.) with several focused questions. Obviously, it could me more complex and time-intensive if you are looking at a comprehensive program review or accreditation process.

I hope that gives you a glimpse of benchmarking as an assessment tool. If you need assistance, Student Life Studies is always here to help.

The National Institute for Learning Outcomes Assessment (https://www.learningoutcomesassessment.org/) has an amazing wealth of resources for campuses interested in all aspects of assessment of student learning in higher education. There are articles, case studies, webinars, frameworks, and more. You really should check it out!

Earlier in 2020, NILOA sponsored a webinar on evidence-based storytelling (https://youtu.be/TutW9VCam8A). While I cannot do it justice (you really should check it out!), the premise is that evidence-based storytelling is supporting claims of student learning using evidence through stories to persuade a particular audience (time 6:07-7:35). It promotes improvement and accountability. Stories are different than just sharing assessment results. They are about sharing the decisions we make and who is impacted by what we do. The webinar provides a number of tips and ideas in the webinar that can help your guide the narrative you are developing.

NILOA created an evidence based storytelling toolkit, which can be accessed at https://www.learningoutcomesassessment.org/wp-content/uploads/2019/10/EBST-Toolkit.pdf. You really should check it out! The toolkit gives steps to make the process fairly easy: defining your audience(s), determining the goals and arguments for your story, choosing compelling evidence, figuring out the type of story (i.e., compliance, improvement, etc.), creating the characters, writing the plot, selecting the visuals that will best tell your story, and determining the medium to get your story out to your audience(s).

If you are an assessment person, reach out to your marketing/communications people. If you are a marketing/communications person, reach out to your assessment people. If you are a program coordinator, bring your assessment and marketing/communications people together. If you need a place to start, contact Student Life Studies, so we can help you. You really should check us out!

Many student affairs/co-curricular programs have moved to a virtual environment, commonly on Zoom. This has changed the way we can do data collection with a “captive audience.” Before the pandemic, programs could have students complete an assessment form in person, which usually yielded a high response rate and immediate feedback. Because we are not all physically in the same room, collecting data in the moment can be a challenge.

If you have not checked out the Zoom polling function, you might want to do that. They have a step-by-step tutorial at https://support.zoom.us/hc/en-us/articles/213756303-Polling-for-meetings. Like any good assessment, you will want to plan ahead. It’s much easier to create your questions ahead of time, rather than while you are in your meeting. Remember, if you are asking bad questions, it doesn’t matter how fancy your platform is—you will still get bad answers that won’t help you.

You can ask both choose one and choose all of the above types of questions. Unfortunately, you cannot use a text response type of question. You can specify if you want the responses to be anonymous or connected to your participants’ information. You can also have up to 10 response options in each question. Plus, you can have multiple questions inside each poll.

When you are in the meeting, you can launch the poll by clicking on the poll icon at the bottom of the screen. Your participants can then answer the question. When you end the poll question, you can also share it with your participants if you would like to spark discussion or clarify information. You might want to have several poll questions in each program, so you keep participants engaged.

After the meeting, you can sign into the Zoom web portal to access your results, which will be generated in an Excel spreadsheet. If you chose for the responses to be anonymous, you will see that indicated in the spreadsheet. If you did not choose that option, you will see the user name and user email. This allows you to follow up if you need to contact participants. You can then analyze the results for any information that will help you improve what you are doing.

While no one asked to shift all of our program delivery to an online format, it does give us an opportunity to learn and use new technology in data collection. It’s okay to not be perfect in it every time, but lots of people can empathize and help. We can stop assessment, or we can embrace the opportunity. I hope that you embrace the opportunity!