Applications of expectation maximization algorithm for coherent optical communication

In this invited paper, we present powerful statistical signal processing methods, used by machine learning community, and link them to current problems in optical communication. In particular, we will look into iterative maximum likelihood parameter estimation based on expectation maximization algorithm and its application in coherent optical communication systems for linear and nonlinear impairment mitigation. Furthermore, the estimated parameters are used to build the probabilistic model of the system for the synthetic impairment generation. It is shown numerically and experimentally that iterative parameter estimation based on expectation maximization algorithm is a powerful tool in combating system impairments such as non-linear phase noise, inphase and quadrature (I/Q) modulator imperfections and laser linewidth. We show experimentally that for a dispersion managed polarization multiplexed 16-quadrature amplitude modulation (QAM) system at 14 Gbaud a gain in the nonlinear system tolerance of up to 3 dB can be obtained. For, a dispersion unmanaged system this gain reduces to 0.5 dB. Moreover, we show that joint estimation of carrier frequency, phase, signal means and noise covariance, can be performed iteratively by employing expectation maximization. Using experimental data we show that joint carrier synchronization and detection offers an improvement of 0.5 dB in terms of input power compared to hard decision digital phaselocked loop (PLL) based carrier synchronization and demodulation.