Zhang Yi, Wei Fan, Li Jingyi, Wang Yan, Yu Yanyan, Chen Jianli, Cai Zipo, Liu Xinyu, Wang Wei, Yao Sensen, Wang Peng, Wang Zhong
First Primary School, Beijing 100071, China.
Fangzhuang Primary School, Beijing 100078, China.
Biol Methods Protoc. 2025 Aug 11;10(1):bpaf062. doi: 10.1093/biomethods/bpaf062. eCollection 2025.
The use of children's drawings to examining their conceptual understanding has been proven to be an effective method, but there are two major problems with previous research: (i) The content of the drawings heavily relies on the task, and the ecological validity of the conclusions is low. (ii) The interpretation of drawings relies too much on the subjective feelings of the researchers. To address this issue, this study uses the Large Language Model (LLM) to identify 1420 children's scientific drawings (covering nine scientific themes/concepts) and uses the word2vec algorithm to calculate their semantic similarity. The study explores whether there are consistent drawing representations for children on the same theme and attempts to establish a norm for children's scientific drawings, providing a baseline reference for follow-up children's drawing research. The results show that the representation of most drawings has consistency, manifested as most semantic similarity >0.8. At the same time, it was found that the consistency of the representation is independent of the accuracy (of LLM's recognition), indicating the existence of consistency bias. In the subsequent exploration of influencing factors, we used Kendall rank correlation coefficient to investigate the effects of "sample size," "abstract degree," and "focus points" on drawings and used word frequency statistics to explore whether children represented abstract themes/concepts by reproducing what was taught in class. It was found that accuracy (of LLM's recognition) is the most sensitive indicator, and data such as sample size and semantic similarity are related to it. The consistency between classroom experiments and teaching purpose is also an important factor, many students focus more on the experiments themselves rather than what they explain. In addition, most children tend to use examples they have seen in class to represent more abstract themes/concepts, indicating that they may need concrete examples to understand abstract things.
使用儿童绘画来检验他们的概念理解已被证明是一种有效的方法，但以往的研究存在两个主要问题：（i）绘画内容严重依赖于任务，结论的生态效度较低。（ii）对绘画的解读过于依赖研究人员的主观感受。为了解决这个问题，本研究使用大语言模型（LLM）识别了1420幅儿童科学绘画（涵盖九个科学主题/概念），并使用词向量算法计算它们的语义相似度。该研究探讨了儿童在同一主题上是否存在一致的绘画表现，并试图建立儿童科学绘画的规范，为后续的儿童绘画研究提供基线参考。结果表明，大多数绘画的表现具有一致性，表现为大多数语义相似度>0.8。同时发现，表现的一致性与（LLM识别的）准确性无关，表明存在一致性偏差。在后续对影响因素的探索中，我们使用肯德尔等级相关系数来研究“样本量”、“抽象程度”和“焦点”对绘画的影响，并使用词频统计来探索儿童是否通过再现课堂上学到的内容来表现抽象主题/概念。结果发现，（LLM识别的）准确性是最敏感的指标，样本量和语义相似度等数据与之相关。课堂实验与教学目的之间的一致性也是一个重要因素，许多学生更关注实验本身而不是实验所解释的内容。此外，大多数儿童倾向于使用他们在课堂上看到的例子来表现更抽象的主题/概念，这表明他们可能需要具体的例子来理解抽象事物。
