Polarization-ring-switching for nonlinearity-tolerant geometrically-shaped four-dimensional formats maximizing generalized mutual information

In this paper, a new four-dimensional 64-ary polarization ring switching (4D-64PRS) modulation format with a spectral efficiency of 6 bit/4D-sym is introduced. The format is designed by maximizing the generalized mutual information (GMI) and by imposing a constant-modulus on the 4D structure. The proposed format yields an improved performance with respect to state-of-the-art geometrically shaped modulation formats for bit-interleaved coded modulation systems at the same spectral efficiency. Unlike previously published results, the coordinates of the constellation points and the binary labeling of the constellation are jointly optimized. When compared with polarization-multiplexed 8-ary quadrature-amplitude modulation (PM-8QAM), gains of up to 0.7 dB in signal-to-noise ratio are observed in the additive white Gaussian noise (AWGN) channel. For a long-haul nonlinear optical fiber system of 8,000 km, gains of up to 0.27 bit/4D-sym (5.5% data capacity increase) are observed. These gains translate into a reach increase of approximately 16% (1,100 km). The proposed modulation format is also shown to be more tolerant to nonlinearities than PM-8QAM. Results with LDPC codes are also presented, which confirm the gains predicted by the GMI.