End-to-end learning of joint noise shaping and probabilistic shaping for OAM mode division multiplexing transmission.

Intensity modulation direct detection (IM/DD) orbital angular momentum (OAM) mode division multiplexing (MDM) technology can greatly expand the capacity of a communication system, which is a promising solution for the next generation of high-speed passive optical networks (PONs). However, there are serious obstacles such as mode coupling, device nonlinear impairment, and quantization noise in an IM/DD OAM-MDM system with a low-resolution digital-to-analog converter (DAC). In this Letter, we propose a novel, to the best of our knowledge, end-to-end (E2E) learning scheme based on a double residual feature decoupling network (DRFDnet) emulator with joint probabilistic shaping (PS) and noise shaping (NS) for the OAM-MDM IM/DD transmission. Our DRFDnet emulator can accurately build a complex nonlinear model of an OAM-MDM system by separating the signal impairments into linear and nonlinear. Meanwhile, a DRFDnet-based E2E scheme for joint PS and NS is presented with the aim of compensating the signal impairment for the OAM-MDM IM/DD system. An experiment is carried out on a 200 Gbit/s PON system based on the OAM-MDM IM/DD transmission. The experimental results demonstrate that the proposed DRFDnet-based joint PS and NS scheme is a promising solution to effectively mitigate nonlinear distortions and outperforms the CGAN-based joint PS and NS scheme and traditional joint PS and NS scheme with receiver sensitivity improvements of 1.2 dBm and 2.5 dBm under hard-decision forward error correction (HD-FEC) thresholds, respectively. Our experimental results demonstrate that the proposed DRFDnet emulator-based E2E learning scheme is a viable candidate for future PON.