Memory-aware end-to-end learning of channel distortions in optical coherent communications.

We implement a new variant of the end-to-end learning approach for the performance improvement of an optical coherent-detection communication system. The proposed solution enables learning the joint probabilistic and geometric shaping of symbol sequences by using auxiliary channel model based on the perturbation theory and the refined symbol probabilities training procedure. Due to its structure, the auxiliary channel model based on the first order perturbation theory expansions allows us performing an efficient parallelizable model application, while, simultaneously, producing a remarkably accurate channel approximation. The learnt multi-symbol joint probabilistic and geometric shaping demonstrates a considerable bit-wise mutual information gain of 0.47 bits/2D-symbol over the conventional Maxwell-Boltzmann shaping for a single-channel 64 GBd transmission through the 170 km single-mode fiber link.