Suppr超能文献

可扩展的水印技术用于识别大型语言模型输出。

Scalable watermarking for identifying large language model outputs.

机构信息

Google DeepMind, London, UK.

Google, Mountain View, CA, USA.

出版信息

Nature. 2024 Oct;634(8035):818-823. doi: 10.1038/s41586-024-08025-4. Epub 2024 Oct 23.

Abstract

Large language models (LLMs) have enabled the generation of high-quality synthetic text, often indistinguishable from human-written content, at a scale that can markedly affect the nature of the information ecosystem. Watermarking can help identify synthetic text and limit accidental or deliberate misuse, but has not been adopted in production systems owing to stringent quality, detectability and computational efficiency requirements. Here we describe SynthID-Text, a production-ready text watermarking scheme that preserves text quality and enables high detection accuracy, with minimal latency overhead. SynthID-Text does not affect LLM training and modifies only the sampling procedure; watermark detection is computationally efficient, without using the underlying LLM. To enable watermarking at scale, we develop an algorithm integrating watermarking with speculative sampling, an efficiency technique frequently used in production systems. Evaluations across multiple LLMs empirically show that SynthID-Text provides improved detectability over comparable methods, and standard benchmarks and human side-by-side ratings indicate no change in LLM capabilities. To demonstrate the feasibility of watermarking in large-scale-production systems, we conducted a live experiment that assessed feedback from nearly 20 million Gemini responses, again confirming the preservation of text quality. We hope that the availability of SynthID-Text will facilitate further development of watermarking and responsible use of LLM systems.

摘要

大型语言模型 (LLMs) 能够生成高质量的合成文本,这些文本通常与人类编写的内容难以区分,其规模之大足以显著影响信息生态系统的性质。水印技术可以帮助识别合成文本,并限制意外或故意的滥用,但由于严格的质量、可检测性和计算效率要求,它尚未在生产系统中得到采用。在这里,我们描述了 SynthID-Text,这是一种可用于生产的文本水印方案,它可以在保持文本质量的同时实现高精度的检测,同时具有最小的延迟开销。SynthID-Text 不会影响 LLM 的训练,仅修改采样过程;水印检测具有计算效率,无需使用基础 LLM。为了实现大规模的水印,我们开发了一种将水印与推测性采样相结合的算法,这是一种在生产系统中经常使用的效率技术。在多个 LLM 上的评估经验表明,SynthID-Text 提供了比可比方法更高的可检测性,并且标准基准和人机并排评级表明 LLM 能力没有变化。为了展示水印在大规模生产系统中的可行性,我们进行了一项现场实验,评估了近 2000 万 Gemini 响应的反馈,再次证实了文本质量的保持。我们希望 SynthID-Text 的可用性将促进水印技术的进一步发展和负责任地使用 LLM 系统。

https://cdn.ncbi.nlm.nih.gov/pmc/blobs/ec9d/11499265/406d68054215/41586_2024_8025_Fig1_HTML.jpg

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验