Last month I was asked to present at a virtual conference related to COVID-19 with a colleague from NYU. If you know me, you might be saying, “Darby, you are a doctor, but you are not that kind of doctor.” So, true. The point of the presentation, though, was to talk about the value of rapid data collection in times of crisis or the unknown when information, decisions, actions can and do change very quickly.
I thought I would share some of the take-aways from that session, in case you find yourself in a global pandemic again or some other major campus crisis.
Good is better than perfect. (Okay, there is no perfect assessment, as much as we try!) In times of rapid change, we don’t always have time to plan the perfect data collection tool, and we can’t wait until we have all the information that we need to plan. While that doesn’t give you an excuse to launch a really bad survey, it does give you a little latitude to do the best you can with the information you have.
Collaborate. Collaborate. Collaborate. Other people may already have information that you need, so it would be detrimental to re-collect data. It is a waste of your time and annoying to your audience. While everyone may feel pressure to immediately collect data and make decisions in their own silo, there should be some coordinated effort that makes sense and streamlines the assessment and communication process.
Small samples and few questions are okay. Sometimes in assessment and research, we get caught up in having a large, representable sample and a high response rate. Those are great things, but in times of crisis, you may not have that luxury. If you can collect reasonably accurate and representative responses from a small group in a short period of time, that gives you the ability to make a decision, test an action, and determine whether that new action is working. When you re-assess using another small sample, you’ll generally know whether that new intervention worked. If it doesn’t, then you can try something else fairly quickly. Think about the minimum data points you need to maximize the information you can use.
Assessment can be low-tech. It could be as easy as tracking the topic of phone calls/emails that you get from students. Within a day, you probably could make a check sheet of common topics. Fairly quickly (within days or a week), you probably could summarize the main concerns and communicate them to the appropriate person/office. When some action is taken (e.g., putting more information on your website), you can determine if the number of questions about that topic have decreased. If you saw students in person, you could ask them to write main concerns on an index card. If you are calling students, you can also track their concerns/questions.
Equity is still a key component in assessment. In times of crisis, we need to recognize that some groups are impacted more than others. We need to collect data from a variety of populations, while not adding a burden to marginalized populations who are already experiencing greater trauma. In addition, when we are making decisions, we need to include multiple voices at the table. Policy decisions, new procedures, and new requirements can have an inequitable impact, even when we think they might be fair to all (e.g., wearing a face covering, requiring a laptop, etc.).
I hope that gives you some insight about collecting data in these times of rapid change. Student Life Studies is always here to help you in that process. Please let us know how we can help.