This book is mainly about theoretical foundations of information theory and coding . Also,

some practical applications of these theories are mentioned in some occasions. On the basis of

introducing measures of information, it mainly focuses on the introduction of the theory of

lossless source coding, limited loss source coding, channel coding and their applications. In

addition, it pays attention to basic concepts and uses popular texts to explain them. In the

background of the current communication system, it includes many examples and figures to

elaborate these concepts and theorems. At the same time, in this book, we do our best to avoid

getting entangled in proving the equations. To deepen the understanding of the knowledge it

touches upon, there are some exercises attached at the end of every chapter, which can be

used as the teaching material for college students, and also, it can be used as the reference

material for career people in the area of communication, telecommunication and electronics.

无

Preface

The information theory is mathematical theory of communication theory. It is science

to study information measures and coding by using mathematical statistics method. It can

be divided into generalized information theory and narrow information theory which is

also called classical theory of information or probabilistic information theory.

This book is on the foundation of probabilistic information measures and their

application of coding theorems for information sources and noisy channels. It pays

attention to elaborate the basic theories, concepts and methods. With this idea, there are

many examples throughout this book which will make these concepts and theorems clearer

and easier understanding. A mathematical book has the potential danger of scaring away

beginners. Therefore, We make every effort to use simple mathematics tools in the

argument of information theory, which may help beginners on the way of mastering it.

It can be used as teaching material in Electronics Department for college students of

upper grade or for graduate students. The whole book is composed of five chapters. which

are both lively introduced and attached by the detail references of the fascinating world of

information theory.

Chapter 1 is introduction, it’s the outline about the basic problems of information

theory and coding from the angle of history. For example, the concept of information and

measures it takes, the formation and development of information theory, and so on.

Chapter 2 deals with some basic concepts on information measurement. Through the

study of this chapter you may gain the ability of knowing how much information you can

get from a trial of tossing a coin and read the result of it. Also, this chapter introduces some

theorems on continuous source and discrete source.

Chapter 3 deals with lossless source coding theorems. In chapter 3, you will know

that source which needs compressed to suit for the transmission. In this part, it answers

how to compress the information of source kept.

Chapter 4 deals with limited distortion source coding. Former chapters are all under

the condition of lossless distortion, but in real world, to a certain degree distortion can’t be

avoided by people, for the information in transmission and records can be more or less.

This chapter is about the theorems and practice based on this idea.

Chapter 5 is about channel. For channel, it uses channel capacity to measure the

transfer ability. The channel model and the capacity calculation of some typical channels

discrete or continuous ones are the focus. It also introduces the coding theory. Some

famous and useful coding methods are included in it, such as CRC code, and Hamming

PREFACE

2 信

息

论

与

编

码

(

英文版)

code, etc. After the study of this part, readers can see the real situation of information

transmission.

The contents of this book is from simple to complex. To improve the ability of

analyzing and solving problems, there are some questions and exercises attached at the end

of each chapter.

This book is the harvest of hard work and wisdom of a group of people. The members

participating the writing of the book are Liang Jianwu (Chapter1-3), Luo Xiying (from

Human University of Science and Technology, Chapter 4), Guo Ying (Chapter 5). Fu

Liaoyan, the English department dean of Changsha medical college, is the master auditor

of this book. Besides, Liu Junjun, Long Xiaomei, Tian Ye, Zhou Yuanyuan, He Zhibin, Li

Huawei, Fu Shifeng, Tan Hailong, Wen Zheng, did great favors for the writing of this book.

We have tried our best to free it from all errors. Unfortunately there does not exist a

foolproof error control technique for that. Therefore, you are welcome to point out our

mistakes and give us your opinions.

Authors of this book in Central South University

2015-03-18

本书力图用较多的例子和图表来阐述概念和理论，同时尽量避免纠缠于繁琐难懂的公式证明之中，为了加深读者对所讲述知识的理解，每章最后都配有适量的练习题供读者选用。