Online focus groups (FGs) and individual interviews (IDIs) are increasingly used to collect qualitative data. Online data collection offers benefits (eg, geographic reach), but the literature on whether and how data collection modality affects the data generated is mixed. The limited evidence base suggests that data collected via online modalities may be less rich in terms of word count, but more efficient in terms of surfacing thematic content. There is also limited evidence on the comparative costs of online vs in-person data collection.
The purpose of this study was to compare data generated from FGs and IDIs across 4 data collection modalities: (1) face-to-face; (2) online synchronous video-based; (3) online synchronous text-based; and (4) online asynchronous text–based. We also aimed to compare participant experience and data collection costs across modalities.
We used a cross-sectional quasi-experimental design. We systematically assigned participants to 1 of the 4 modalities (according to a rolling sequence) on enrollment and randomly assigned them to either IDIs or FGs. We held constant the interviewer and question guide across 24 FGs (n = 123 participants) and 48 IDIs, conducted between September 2016 and October 2017. Participants also completed a brief survey on their experiences of data collection. A team of 3 analysts performed inductive thematic analysis of the qualitative data, generating and applying emergent theme-based codes. We also used a priori codes to tag sensitive information across modalities. Analysts were not masked to data type, but all transcripts were coded independently by 2 analysts and compared to reach final consensus coding. We operationalized data richness in terms of volume of participant data, measured by word count, and thematic content, measured by the number of thematic codes applied per modality. Using time and expense data from the study, we calculated average cost per data collection activity.
Visual (face-to-face and online video) modalities generated significantly greater volume of data than did online text-based modalities; however, there were no significant qualitative differences in the thematic content among modalities for either IDIs or FGs. Text-based online FGs were more likely to contain a dissenting opinion ( = 0.04) than visually based FGs, although this level of significance should be interpreted cautiously due to multiple comparisons. Participant ratings of data collection events were generally in the moderate to high range, with statistically significant differences in participant experience measures by modality for FGs: participants rated online video FGs lower than others on several measures. Without travel, online video data collection had the highest average costs for both IDIs and FGs; however, if estimated travel costs are included, then in-person data collection was more expensive.
Among our sample, online modalities for conducting qualitative research did not result in substantial or significantly different thematic findings than in-person data collection. We did not find that online modalities encouraged more sharing of personally sensitive information, although we observed more instances of dissenting opinions in online text-based modalities. The homogeneity of the sample—in terms of sex, race, educational level, and computer skills—limits the wider generalizability of the findings. We also did not have a geographically distributed sample, which prevented us from having actual travel expenses for the cost analysis; however, the findings from this study were largely consistent with previous comparative research.
