Zhang Junteng, Zhang Junzhe, Ding Dandan, Ma Zhan
IEEE Trans Vis Comput Graph. 2025 Apr;31(4):1985-1998. doi: 10.1109/TVCG.2024.3375861. Epub 2025 Feb 27.
The emergence of holographic media drives the standardization of Geometry-based Point Cloud Compression (G-PCC) to sustain networked service provisioning. However, G-PCC inevitably introduces visually annoying artifacts, degrading the quality of experience (QoE). This work focuses on restoring G-PCC compressed point cloud attributes, e.g., RGB colors, to which fully data-driven and rules-unrolling-based post-processing filters are studied. At first, as compressed attributes exhibit nested blockiness, we develop a learning-based sample adaptive offset (NeuralSAO), which leverages a neural model using multiscale feature aggregation and embedding to characterize local correlations for quantization error compensation. Later, given statistically Gaussian distributed quantization noise, we suggest the utilization of a bilateral filter with Gaussian kernels to weigh neighbors by jointly considering their geometric and photometric contributions for restoration. Since local signals often present varying distributions, we propose estimating the smoothing parameters of the bilateral filter using an ultra-lightweight neural model. Such a bilateral filter with learnable parameters is called NeuralBF. The proposed NeuralSAO demonstrates the state-of-art restoration quality improvement, e.g., 20% BD-BR (Bjøntegaard delta rate) reduction over G-PCC on solid points clouds. However, NeuralSAO is computationally intensive and may suffer from poor generalization. On the other hand, although NeuralBF only achieves half of the gains of NeuralSAO, it is lightweight and exhibits impressive generalization across various samples. This comparative study between the data-driven large-scale NeuralSAO and the rules-unrolling-based small-scale NeuralBF helps to understand the capacity (i.e., performance, complexity, generalization) of underlying filters in terms of the quality restoration for compressed point cloud attribute.
全息媒体的出现推动了基于几何的点云压缩（G-PCC）的标准化，以维持网络服务供应。然而，G-PCC不可避免地会引入视觉上令人讨厌的伪影，降低体验质量（QoE）。这项工作专注于恢复G-PCC压缩的点云属性，例如RGB颜色，并研究了完全基于数据驱动和基于规则展开的后处理滤波器。首先，由于压缩属性呈现嵌套块状，我们开发了一种基于学习的样本自适应偏移（NeuralSAO），它利用一个使用多尺度特征聚合和嵌入的神经模型来表征局部相关性，以进行量化误差补偿。后来，考虑到量化噪声呈统计高斯分布，我们建议利用具有高斯核的双边滤波器，通过联合考虑邻居的几何和光度贡献来加权邻居，以进行恢复。由于局部信号通常呈现不同的分布，我们提出使用一个超轻量级神经模型来估计双边滤波器的平滑参数。这种具有可学习参数的双边滤波器被称为NeuralBF。所提出的NeuralSAO展示了最先进的恢复质量提升，例如，在实心点云上，相对于G-PCC，BD-BR（Bjøntegaard增量率）降低了20%。然而，NeuralSAO计算量很大，并且可能存在泛化性差的问题。另一方面，尽管NeuralBF仅实现了NeuralSAO增益的一半，但它很轻量级，并且在各种样本上表现出令人印象深刻的泛化性。这项对数据驱动的大规模NeuralSAO和基于规则展开的小规模NeuralBF的比较研究有助于从压缩点云属性的质量恢复方面理解底层滤波器的能力（即性能、复杂性、泛化性）。
