When I was done designing the video game scenarios for families in Quest Atlantis, I thought the most challenging part of my dissertation was over. I was absolutely wrong! I found out that data collection can be as challenging as the design work. Here, I would like to share lessons I’ve learned from collecting data for my dissertation.
I believe collecting data for dissertation is way more stressful than other data collection experiences. First of all, the stakes are higher; if you don’t get it right you will continue to be in purgatory. Second, the financial and human resources can be quite limited, especially if you are doing your dissertation on something different than what your advisor is doing. My dissertation is especially challenging because I want parent-child pairs to come to a site at least two times to play through my game scenarios when families are already busy and can’t find time to spend together (note: the goal of my study is to use educational video games as a platform to bring parents and children together). My life would have been much easier if I were to work with kids in a classroom setting. I would have had to connect with a teacher who has 20 students to collect my dissertation data without worrying about losing participants.
The plan for my dissertation data collection was to run a Family Quest program at the local Boys and Girls Club and a Family Quest Saturday Academy at a computer lab at IU to get 16 mother-child pairs to play through my designs. Connecting with the local Boys and Girls Club was easy since I ran a Family Quest program there before. As usual people at the Boys and Girls Club were welcoming and very supportive. We scheduled a start date and time for the program. I decided to run the program once a week for four weeks between 6:00-7:30pm.
For the Family Quest Saturday Academy, things were little different. I was able to reserve the computer lab between 10:30am-12:00 and 2:30-4:00pm to run two sessions on Saturdays. The challenge was to find parent-child pairs when I had no personal connection to schools, homeschooling community, or other places that might allow me to reach parents and children.
Lesson #1: Keep in mind that you are the ONLY person who cares about your study.
Some people are willing to help and some are not. You will find that most people do not care about your study. Most people (a) do not know what a research study means, (b) do not have the time to deeply engage with something they don’t have to, and (c) are forgetful. Your dissertation study is YOUR baby, not theirs! Some people will not return your calls or e-mails. Some will show interest but forget to follow up. You will also bump into administrative issues. For the Family Quest Saturday Academy, for example, I decided to put up posters and drop off brochures to public places (e.g. the YMCA, the library, coffee shops, etc.) to spread the word about the program. Some places, due to their regulations around promotion and advertisement, did not allow me to put up a poster and leave brochures even though my research study involved a FREE educational family program that could benefit the community. They said: “no” and showed me the door. Like I said, you are the ONLY person who cares about your study. Just because people do not see the value of your research and are not willing to help does not mean that what you are trying to do is worthless. So, ignore those people who are unwilling to help you, and keep on finding new venues to connect with your targeted group of people.
Lesson #2: Have solid recruitment materials.
Preparing recruitment materials such as posters, flyers, and brochures takes time regardless of your skills in using the tool to create them. You should treat your recruitment materials like advertisement: When advertising, you need to have a target audience in mind. Same is true for all your recruitment materials. Ask yourself: Who is going to read this poster? Who am I going to give this brochure to? To recruit participants for my dissertation study, I’ve prepared two versions of everything: a child version, and a parent version. However, I kept the design consistent across different recruitment materials so that parents and children have a similar sense of what the experience/study (the product you are advertising) will look and feel like. You should use colors in your recruitment materials. Don’t be stingy about spending money on color printing! Preparing recruitment materials takes time. Once you created your posters, flyers, and/or brochures, show it to other people. Explain them what you are trying to do and get their feedback. Keep in mind that you might send your recruitment materials directly to potential participants or someone who can forward it to potential participants (e.g. teachers). Make sure that the file size is reasonable to share with other people via e-mail. Also, summarize the necessary information in your e-mail. Don’t just say: “See attachment”. Make sure that you use a professional voice in your e-mails. Use a catchy title and a subtitle in your recruitment materials. For adults, provide all the information needed on the recruitment material. For children, however, I found it more useful to have a catchy title in the form of a question, and tell kids to grab an adult or stop by at the front desk to learn more about how to participate. Kids easily forget about logistical information and their parents are the ones who sign them up for activities. Kids will drag their parents if your poster sparked an interest in them.
Lesson#3: Be prepared to get tons of questions.
Since you put all the necessary information, all you need is to hear from your potential participants about whether they are participating in your study, right? Wrong. You will get tons of questions from people. This, however, does not mean that they are going to participate in your study. So, you have to be very careful about how you answer questions and how you interact with people. One of the challenging things about recruitment is to gracefully say “no” to those who are not eligible to participate in your study. For example, although on all my recruitment materials it was clearly stated that children must be ages between 9 and 13, I got responses from parents who had children ages between 6 and 8. In my response, I’ve always thanked parents for their interest first, then I let them know that the study requires that children are ages between 9 and 13. I’ve always ended my conversation with a note that I will keep them in mind for future studies and that I’d appreciate it if they could share the information if they know someone with a kid in the right age range. I think this allows for the conversation to end on a positive note. You will also get questions that you never thought through before. Some questions will catch you off guard. It is ok to tell people that you do not have answer and that you will get back to them as soon as you know the answer.
Lesson #4: Send reminders to your participants.
You have participants for your study. Great! Now, you have to face the issues of retention. Your participants are busy people. You have to remind them about the dates and times of your study, otherwise they will forget. It is good to send at least two reminders to your participants. Reminders will allow people to put the event on their calendar. And remember, even those who put the event on their calendar might forget about their commitment. You can make phone calls, send text messages, and/or e-mail to remind your participants about the time and place. Let your participants decide which medium they want you to contact them. For example, I run the Family Quest for four weeks at the club on Thursdays. Every Monday and Thursday morning, I sent out reminder e-mails. Some parents responded and told me whether they were able to participate that week or not. This let me know what to expect that week, and helped me manage my disappointment of not having the participant for that week.
Lesson #5: Keep your spirit high.
Data collection can be an emotional roller-coaster. You send out bunch of e-mails to teachers, administrators, parents and you hear nothing for days or worse you get a negative response. You recruit parents and children for your study, and they cancel on you the day before the implementation or worse during implementation. You find yourself putting 110% of energy into your data collection and get 10% return. During data collection phase, you need to remind yourself that things are going to be OK. You need to continue providing information, checking in, reminding, answering questions, sending e-mails, making phone calls, and running your study with a smile on your face. Even though you are exhausted, try to keep your energy high, and do not let your participants feel the stress you feel inside! Be grateful to have each and every participant you have and enjoy the experience with them.
Lesson #6: Accommodate your participants.
Do whatever it takes for you to collect data you need from your participants. This means you might have to change the time and place you meet or do things in a slightly different way. Don’t be shy about suggesting alternatives to your participants when they say they can’t continue to participate in the study. For example, one of my parent-child pairs came to the Family Saturday Academy only once, and never contacted me to let me know what was going on. I sent the parent reminder e-mails each week, and she finally responded and said that something came up and they can’t make it on Saturdays. So, I offered her to come to my office for 90 minutes and finish going through the game scenarios with her son. She agreed. Because it would have been unreasonable to ask her to come to my office twice (one to play and one for an interview), I decided to go ahead and let them go through one game scenario for 60 minutes and interview the parent for 30 minutes (as opposed to 45 to 60 minutes). Something similar happened at the club where the pair could not make it to the last session, so I scheduled to meet with them later in the week at the club where they played through the scenarios at the computer lab when other kids were around. The point is things do not always go according to plan. You need to trust that your participants want to help you and that you need to make sure that you come up with a different plan to make things happen.
Lesson#8: Change your game plan when needed.
Before graduate students start to collect data for their dissertation, they write a “dissertation proposal” that they later defend in front of their dissertation committee. Once you discuss what you are going to do for your dissertation with your committee members, the path to getting your degree seems quite clear. All you need to do is to go out and do it. It all sounds great except the real world may not necessarily collaborate with you in carrying out the exact dissertation study you proposed to your committee. What are you going to do? You will adapt and find ways to carry out your study even if that means you have to slightly modify your research questions or methodology. Keep in mind that you should always update your dissertation chair about the data collection process and check in with him or her about the changes you think is necessary for successfully complete data collection. In my case, I did not get enough parent-child pairs for my dissertation study through the Family Quest programs I ran. Because making a multiple-day commitment was an issue for parents, I changed my game plan so that parent-child pairs came to a place for 2 90 minute session separated by catered lunch. That meant that I had to carry interviews on the phone or via skype but that was something that would not change the core of dissertation study. I contacted with teachers to help me set up one-day Family Quest event at their schools and with a help of few teachers I was able collect plenty of data. When you have disappointing results shake off the hopelessness and helplessness quickly, and use your critical thinking skills and creativity to overcome the challenges real world introduces. Every research project will have the tension between what’s envisioned (theory) and what’s actually happened (practice), so you have to learn to adapt and be comfortable with changing plans.
Lesson #9: Be patient.
If you are not doing a survey study (and sometimes even if you are), data collection can take several months. Accumulating data is a slow process, so rather than focusing on your failure to achieve your BIG goal of recruiting certain number of participants or collecting certain kinds of data, pay attention to what you achieve everyday towards getting your data collection done. You really have to be patient. Sometimes, you will collect data that you think is useless. Don’t jump to conclusions; wait until you analyze the data to make your decision to exclude the data from your results.
Lesson #10: Treat dissertation data collection as another learning experience.
Like I said, collecting data for your dissertation can be more stressful than other data collection experiences. You focus so much on the task of getting your degree that you might forget to enjoy the process of running your dissertation study. Dissertation is a great learning opportunity regardless of how much previous research experiences you have (and in the end, every research project comes with its own challenges, and thus is a new learning experience). Try things that you never tried before during your data collection to improve your research skills. I, for example, videotaped myself during parent interviews to reflect on my tone of voice, body posture, and gestures to make sure that I wasn’t reacting to my participant’s responses in a way that shut them down or let them know what I was looking for. Although it might seem like a burden at times, enjoy the data collection process and learn from it!