| dc.description.abstract |
It is well established that Level 1 of the Kirkpatrick model is the most usedtraining evaluation method and is typically carried out by distributing reactionnaires (i.e.,smile sheets) at the end of a training program. While the open-ended questionscommonly included in reactionnaires can help capture critical program evaluationinformation, there is a lack of guidance in the literature for instructional designers onanalyzing this type of data, referred to as qualitative reactions. Thus, there were twopurposes for this qualitative study: first, to examine the methods used by instructionaldesigners to analyze qualitative reactions, and second, to examine the underlying reasonsfor using those methods of analysis. Semi-structured interviews were conducted with 13instructional designers. An analysis of the interview data revealed three themesregarding the methods used to analyze qualitative reactions: most participants used atleast one surveying platform to collect the qualitative reactions; most participants used arange of tactics to look for keywords, patterns, or themes in the data; and mostparticipants reviewed the data as a team. Three themes also emerged regarding theunderlying reasons for the methods of analysis: all participants had at least one primarymotive for analyzing the data; most participants incorporated their unique skills,knowledge, and abilities when analyzing their data sets; and most participants had asupport system in place to help with the data analysis. The results of this study provideinsight into an under-researched aspect of Level 1 evaluation and can be used to helpeducate instructional designers on qualitative data analysis (QDA) practices. |
en_US |