Discovering Novel Communication Algorithms Via Machine Learning

Major Goals: Coding theory is a central discipline underpinning wireless communications that are the workhorses of the information age. Significant progresses have been made over the past century, with enormous impact on humanity. Landmarks include convolutional codes, Turbo codes, Low-Density Parity-Check codes, and most recently polar codes, which are standards in cellular, satellite, and deep space communications. However, such progresses in coding theory is largely driven by individual human ingenuity, and breakthroughs have inevitably been sporadic over the past century. The objective of this project is to automate the discovery of coding and decoding algorithms via deep learning. Canonical channel models of communication systems provide excellent platforms for training neural networks, as unlimited training examples can be generated, there is no issues of overfitting, and there are gold standards on evaluating the learned coding schemes. If successful, this provides a new data-driven framework for designing codes, which can fill-in the current gaps in performance of existing schemes in the feedback channel. Accomplishments: We design the first family of codes obtained via deep learning, which significantly beats state-of-the-art codes designed over several decades of research. The communication channel under consideration is the Gaussian noise channel with feedback, whose study was initiated by Shannon; feedback is known theoretically to improve reliability of communication, but no practical codes that do so have ever been successfully constructed. We break this logjam by integrating information theoretic insights harmoniously with recurrent-neural-network based encoders and decoders to create novel codes that outperform known codes by 3 orders of magnitude in reliability. We also demonstrate several desirable properties of the codes: (a) generalization to larger block lengths, (b) composability with known codes, (c) adaptation to practical constraints.