Krajancich Brooke, Padmanaban Nitish, Wetzstein Gordon
IEEE Trans Vis Comput Graph. 2020 May;26(5):1871-1879. doi: 10.1109/TVCG.2020.2973443. Epub 2020 Feb 13.
Occlusion is a powerful visual cue that is crucial for depth perception and realism in optical see-through augmented reality (OST-AR). However, existing OST-AR systems additively overlay physical and digital content with beam combiners - an approach that does not easily support mutual occlusion, resulting in virtual objects that appear semi-transparent and unrealistic. In this work, we propose a new type of occlusion-capable OST-AR system. Rather than additively combining the real and virtual worlds, we employ a single digital micromirror device (DMD) to merge the respective light paths in a multiplicative manner. This unique approach allows us to simultaneously block light incident from the physical scene on a pixel-by-pixel basis while also modulating the light emitted by a light-emitting diode (LED) to display digital content. Our technique builds on mixed binary/continuous factorization algorithms to optimize time-multiplexed binary DMD patterns and their corresponding LED colors to approximate a target augmented reality (AR) scene. In simulations and with a prototype benchtop display, we demonstrate hard-edge occlusions, plausible shadows, and also gaze-contingent optimization of this novel display mode, which only requires a single spatial light modulator.
遮挡是一种强大的视觉线索，对于光学透视增强现实（OST-AR）中的深度感知和真实感至关重要。然而，现有的OST-AR系统使用分束器将物理内容和数字内容叠加在一起——这种方法不容易支持相互遮挡，导致虚拟物体看起来是半透明的且不真实。在这项工作中，我们提出了一种新型的具有遮挡功能的OST-AR系统。我们不是将真实世界和虚拟世界相加组合，而是采用单个数字微镜器件（DMD）以乘法方式合并各自的光路。这种独特的方法使我们能够在逐个像素的基础上同时阻挡来自物理场景的入射光，同时还能调制发光二极管（LED）发出的光以显示数字内容。我们的技术基于混合二进制/连续分解算法，以优化时分复用二进制DMD图案及其相应的LED颜色，以近似目标增强现实（AR）场景。在模拟和使用原型台式显示器时，我们展示了硬边遮挡、逼真的阴影，以及这种仅需单个空间光调制器的新型显示模式的注视相关优化。
