Chen Valerie, Bhatt Umang, Heidari Hoda, Weller Adrian, Talwalkar Ameet
Carnegie Mellon University, Pittsburgh, PA, USA.
University of Cambridge, Cambridge, UK.
Patterns (N Y). 2023 Jul 14;4(7):100780. doi: 10.1016/j.patter.2023.100780.
Machine learning (ML) practitioners are increasingly tasked with developing models that are aligned with non-technical experts' values and goals. However, there has been insufficient consideration of how practitioners should translate domain expertise into ML updates. In this review, we consider how to capture interactions between practitioners and experts systematically. We devise a taxonomy to match expert feedback types with practitioner updates. A practitioner may receive feedback from an expert at the observation or domain level and then convert this feedback into updates to the dataset, loss function, or parameter space. We review existing work from ML and human-computer interaction to describe this feedback-update taxonomy and highlight the insufficient consideration given to incorporating feedback from non-technical experts. We end with a set of open questions that naturally arise from our proposed taxonomy and subsequent survey.
机器学习（ML）从业者越来越多地承担起开发与非技术专家的价值观和目标相一致的模型的任务。然而，对于从业者应如何将领域专业知识转化为机器学习更新，人们考虑得并不充分。在本综述中，我们考虑如何系统地捕捉从业者与专家之间的互动。我们设计了一种分类法，将专家反馈类型与从业者更新相匹配。从业者可能会在观察或领域层面收到专家的反馈，然后将这种反馈转化为对数据集、损失函数或参数空间的更新。我们回顾了机器学习和人机交互领域的现有工作，以描述这种反馈-更新分类法，并强调在纳入非技术专家的反馈方面考虑不足。我们最后提出了一系列自然产生于我们提出的分类法及后续调查的开放性问题。
