Wang Wang, Zhang Ping, Sun Changxia, Feng Dengchao
College of Information and Digital Engineering, Luoyang Vocational College of Science and Technology, Luoyang, 471023, Henan, China.
College of Mathematics and Statistics, Henan University of Science and Technology, Luoyang, 471023, Henan, China.
Sci Rep. 2024 Aug 27;14(1):19838. doi: 10.1038/s41598-024-71089-9.
In unmanned retail store, providing smart customer service requires two stages: understanding customer needs, and guiding the customer to the product. In this paper, we propose an end-to-end (Customer-to-Shelf) software service framework for unmanned retail. The framework integrates visual recognition technology to detect retail objects, large language models to analyze customer shopping needs and make proper recommendations. First, deep neural network based image recognition models are studied for implementing effective stock keeping units (SKUs) object recognition on the shelf. Second, a novel method is proposed to fine-tune large language models (LLMs) with limited training dataset. Metaheuristic approaches are used to optimize the mask locations in a low dimensional parameter space, resulting a more efficient parameter updating method for limited downstream data. Third, by facilitating an automatic analysis of customer preferences powered by large language models, we present a smart recommender system based on domain-specific knowledge, which completes the Customer-to-Shelf software service framework. Experimental results show that our proposed fine-tuning method, is more efficient than other state-of-the-art training methods for limited downstream domain dataset. Using fine-tuned large models, we can successfully create a seamless shopping experience for customers by understanding personalized needs and providing shopping advice in the unmanned retail store.
在无人零售商店中，提供智能客户服务需要两个阶段：了解客户需求并引导客户找到商品。在本文中，我们提出了一种用于无人零售的端到端（从客户到货架）软件服务框架。该框架集成了视觉识别技术以检测零售物品，以及大语言模型以分析客户购物需求并做出适当推荐。首先，研究基于深度神经网络的图像识别模型，以在货架上实现有效的库存保有单位（SKU）物体识别。其次，提出了一种用有限训练数据集对大语言模型（LLM）进行微调的新方法。使用元启发式方法在低维参数空间中优化掩码位置，从而为有限的下游数据生成更有效的参数更新方法。第三，通过促进由大语言模型驱动的客户偏好自动分析，我们提出了一种基于特定领域知识的智能推荐系统，从而完善了从客户到货架的软件服务框架。实验结果表明，对于有限的下游领域数据集，我们提出的微调方法比其他现有技术的训练方法更有效。使用经过微调的大模型，我们可以通过了解个性化需求并在无人零售商店中提供购物建议，成功为客户创造无缝的购物体验。
