Zhang Hongrui, Chen Yanjin, Wang Zhuo, Cui Tie Jun, Del Hougne Philipp, Li Lianlin
State Key Laboratory of Advanced Optical Communication Systems and Networks, School of Electronics, Peking University, Beijing, 100871, China.
State Key Laboratory of Millimeter Waves, Southeast University, Nanjing, 210096, China.
Nat Commun. 2024 May 8;15(1):3869. doi: 10.1038/s41467-024-48115-5.
Solving ill-posed inverse problems typically requires regularization based on prior knowledge. To date, only prior knowledge that is formulated mathematically (e.g., sparsity of the unknown) or implicitly learned from quantitative data can be used for regularization. Thereby, semantically formulated prior knowledge derived from human reasoning and recognition is excluded. Here, we introduce and demonstrate the concept of semantic regularization based on a pre-trained large language model to overcome this vexing limitation. We study the approach, first, numerically in a prototypical 2D inverse scattering problem, and, second, experimentally in 3D and 4D compressive microwave imaging problems based on programmable metasurfaces. We highlight that semantic regularization enables new forms of highly-sought privacy protection for applications like smart homes, touchless human-machine interaction and security screening: selected subjects in the scene can be concealed, or their actions and postures can be altered in the reconstruction by manipulating the semantic prior with suitable language-based control commands.
解决不适定逆问题通常需要基于先验知识进行正则化。迄今为止,只有以数学方式表述(例如,未知量的稀疏性)或从定量数据中隐式学习到的先验知识才能用于正则化。因此,源自人类推理和识别的语义表述先验知识被排除在外。在此,我们引入并展示基于预训练大语言模型的语义正则化概念,以克服这一棘手的限制。我们首先在一个典型的二维逆散射问题中进行数值研究,其次在基于可编程超表面的三维和四维压缩微波成像问题中进行实验研究。我们强调,语义正则化能够为智能家居、非接触式人机交互和安全筛查等应用实现新形式的备受追捧的隐私保护:通过使用合适的基于语言的控制命令来操纵语义先验,可以在重建过程中隐藏场景中的选定对象,或者改变他们的动作和姿势。