Replacing the soft-decision FEC limit paradigm in the design of optical communication systems

A. Alvarado, E. Agrell, D. Lavery, R. Maher, P. Bayvel

Research output: Contribution to journalArticleAcademicpeer-review

73 Citations (SciVal)
3 Downloads (Pure)


The FEC limit paradigm is the prevalent practice for designing optical communication systems to attain a certain bit error rate (BER) without forward error correction (FEC). This practice assumes that there is an FEC code that will reduce the BER after decoding to the desired level. In this paper, we challenge this practice and show that the concept of a channel-independent FEC limit is invalid for soft-decision bit-wise decoding. It is shown that for low code rates and high-order modulation formats, the use of the soft-decision FEC limit paradigm can underestimate the spectral efficiencies by up to 20%. A better predictor for the BER after decoding is the generalized mutual information, which is shown to give consistent post-FEC BER predictions across different channel conditions and modulation formats. Extensive optical full-field simulations and experiments are carried out in both the linear and nonlinear transmission regimes to confirm the theoretical analysis.
Original languageEnglish
Pages (from-to)707-721
Number of pages15
JournalJournal of Lightwave Technology
Issue number2
Publication statusPublished - 15 Jan 2016


  • Bit error rate
  • forward error correction
  • generalized mutual information
  • mutual information
  • soft-decision decoding


Dive into the research topics of 'Replacing the soft-decision FEC limit paradigm in the design of optical communication systems'. Together they form a unique fingerprint.

Cite this