Planning

It’s that time of the year again…setting your plans for the year about what you want to accomplish and how you are going to know if you accomplished it. Some of you may even have assessment plans that you have to submit to the university. My hope is that you don’t think those plans are just busywork where you report things that are not important to you. With reflection and connection, assessment plans can be a useful tool for what you already do. Assessment plans typically have outcomes, measures, and targets that guide the plan for implementation, reflection, and decision-making. Let me explain the jargon and how you can frame your planning.

[Read more…] about Outcomes, Measure, and Targets…Oh My!

As a new school year is on the horizon, and we are asking staff to plan for their upcoming assessment, I think back to the foundation of assessment that is meaningful and manageable. To me, the core of assessment is “What do you really need to know in order to do the best you can?” Sounds simple, right? Obviously, it can get complicated, but let’s not overthink it.

If satisfaction is important, you can ask your customers/clients/users about their experiences, overall or for specific areas. If you are imparting knowledge to students, you can ask about learning (or have them demonstrate it, which would be even better). If you want to know your audience’s needs, you should get their feedback. When you get the results, you can make changes to improve what you do.

[Read more…] about What Do You Really Need to Know?

Lately, I have been thinking about the similarities and differences between assessment, marketing, and development/fundraising. They might have somewhat different purposes, but there are a few things they have in common that should make us think about how we can better collaborate with each other. And, let me admit upfront that I have a lot of assessment experience, but not much formal education with marketing/communications and development/fundraising. I know, a little knowledge can be dangerous, but I don’t think I’m too far afield.

[Read more…] about The Overlap of Assessment, Marketing, and Fundraising

Here we are in May 2021. If you are like me, you are thankful the semester is over and that you have made it this far. You may be experiencing a lot of emotions—relief, exhaustion, contentment, languishing… or all of the above wrapped into one. You might even have a sense of long-term fatigue.

In Student Life Studies, we talk about how many surveys students and staff get throughout the year. We know how many we send out and try to make efforts to be focused, short, and relevant. We don’t know how many other surveys are sent by other units across campus, but anecdotally, I think it is a lot. Students experience SURVEY FATIGUE—it’s a real thing. Students do not have a lot of free time to take surveys and are certainly not going to waste time on poorly designed or executed surveys. You have to make each survey count (and stop doing so many surveys!). In Student Life Studies, we see some surveys with really low response rates and completion throughout the year for a variety of reasons.

[Read more…] about Fatigue, Survey Fatigue

Assessment professionals use the term “close the loop” a lot. (The irony is that if it is a loop, it is already closed.) The gist is that you reassess the changes you made based on some assessment you did, which led to the decision to make that change in the first place. Makes sense, right?

I think the challenge is that we move quickly and move on to the next thing quickly. In order to really “close the loop,” we need to plan and document. Let’s take a look at this step by step as a long(er) term process.

Let’s say you have an idea to assess something…a conference, student learning, climate, etc. You spend quality time thinking about what you really want to know, from whom, in what manner, and when. You create a great assessment (survey, focus group, observation, etc.) that answers your question(s). Usually, you confirm that you are doing many things right. Your audience is having a positive experience, learning what you intended them to learn, etc. Most of the time, there are a couple of things that you could do better based on your assessment results. (Remember, your assessment should be very focused because you can’t work on 100 things in the next year.)  As a good student affairs professional, you make plans to change something in your program/service/activity and move on to the next project or program.

But wait! You’re not done. As you decide to make a change (or two or three), You need to make sure you actually implement them. You also need to make a plan to reassess the thing(s) you change. Don’t just walk away and assume the change is beneficial. Do you have a rationale as to why you think your change will be an improvement? When will you reassess the change to be sure it was positive? Where will you document all of that good stuff?

Let’s look at a simple example.  If you host an annual conference and collect data through a survey after each iteration, you may have a built in mechanism. Last year’s participants really disliked the food, which rated a 2.3 out of a 5-point scale and expressed displeasure in their comments. You discussed the results with your staff and decided to seek out a new caterer. After taste testing three options (what a great assessment gig!), you choose the new caterer that you think will meet your needs within the budget you have. This year, the assessment results changed dramatically, with a score of 4.4 out of 5.0. You decide to renew your contract with that caterer for next year and note that in your transition documents. While that is a very basic example, it illustrates that you assessed, reviewed the results with others, made a decision about change, implemented the change, assessed the change, and documented the process and assessment.

While this process is the right thing to do, it also helps with accountability. Departmental and institutional assessment plans seek documentation about assessment, change, and reassessment. It can be really easy to overlook the reassessment step if there is not accountability to remind us the importance of doing it.

I hope this information was helpful in framing “closing the loop.” Feel free to reach out to Student Life Studies for assistance. We are always here to help.

A student affairs assessment colleague at another institution recently reached out to me with a challenge at her institution. Her problem is not lack of staff motivation to assess, it’s actually the opposite. They are trying to assess every activity all the time, particularly using surveys as the data collection method. Of course, that has led to survey fatigue, a common ailment when staff catch the assessment bug.

It’s not uncommon for the pendulum to swing from no assessment, or outright antipathy about it, to assessing every…single…thing…all…of…the…time. But, not only is that unnecessary, it leads to some unintended consequences, including participant fatigue, staff fatigue, information overload, inability to make a decision or changing direction too frequently, and lack of focus on what’s truly important. How do you overcome that? Here are a few suggestions.

–Focus on what’s important to the program, the unit, the department, the division, and the institution. The alignment is important and helps you understand how what you do fits into the larger context. What are the goals and strategic plans that guide practice? If your program or services do not align with guiding documents, you might need to assess if you should be doing it at all.

–Develop and revisit outcomes frequently. Similar to aligning within the organizational structure above, looking at the outcomes helps you stay internally focused on what is important. If you have developed student learning outcomes, how will you know students have actually learned what you wanted them to learn? If you are assessing program or process outcomes, how are you keeping track of those to know you are effective in your practice?

–Vary your assessment methods. Surveys are overused in the student affairs assessment. Perhaps you have the ability to set up focus groups or interviews as a follow up to survey responses or you focus on individual experiences and perceptions. If you are promoting student learning with a small group, you could create rubrics for self and other evaluation. For longer term, deeper experiences, participants could journal or reflect on photos they have taken.

–Create a calendar for both short term and long terms assessment practices. In the short term, look at the number of programs/activities you do, the academic calendar, and planning and reporting timelines. If you present a canned program 10 times a semester, do you really need to assess it 10 times, or can you take a sample of them? Maybe you pick every second or third program in the first semester to determine what changes need to be made for the following semester. If you are looking at usage of services, maybe you pick one or two “typical” weeks in a semester to review, rather than 15 weeks. In the long run, you can create a calendar of important topics you want to know about and how you plan to assess them. Maybe one year focuses on satisfaction (using a survey), while the next year focuses on student learning (using rubrics or exit interviews), and the year after that focuses on tracking usage (using observation and preexisting data). You assessment will be ongoing, but will also give you focus areas of improvement each year.

–Be brief. See my previous blog, “You Only Get Five Questions.”  People are much more likely to respond to a few quick questions than a long, involved survey. Focus on what your NEED to know, not what you just want to know.  Besides, you can’t address 100 things in a year, so you don’t need to ask 100 questions on a survey. Moreover, if you have asked the same questions for several iterations, the answers have been (acceptably) consistent, and you have no plans to change that area, STOP asking it about it for a while. You already know the answer.

–If you haven’t used past data for change, don’t reassess yet. Why would you think the answer would be any different? Change and improvement take time to implement, especially for large scale changes. Keston Fulcher and his colleagues at James Madison University, wrote a great article, “A Model for Learning Improvement: Weigh Pig, Feed Pig, Weigh Pig.”  If you assess something (weigh pig) without taking action (feed pig) before assessing again (weigh pig), you will likely get the same results. You can assess too frequently, which becomes a waste of time and resources, for both you and the participants.

I hope this gives you some ideas about just saying no to over assessing. You need time to do the great things you do for students and staff; you don’t need to assess everything all the time. Take that advice from someone who loves assessment and works with staff to do it well every day!