EE533 INFORMATION THEORY
| Course Code: | 5670533 |
| METU Credit (Theoretical-Laboratory hours/week): | 3 (3.00 - 0.00) |
| ECTS Credit: | 8.0 |
| Department: | Electrical and Electronics Engineering |
| Language of Instruction: | English |
| Level of Study: | Graduate |
| Course Coordinator: | Assoc.Prof.Dr. BARIŞ NAKİBOĞLU |
| Offered Semester: | Fall Semesters. |
Course Objectives
By the end of the course, the students will be able to
1. Comprehend how to quantify information and uncertainty.
2. Apply concepts of information theory to the data compression problem.
3. Apply information theory to noisy communication channels.
Course Content
Mathematical analysis of discrete and continuous information sources and communication channels. Concepts of mutual information and entropy as mathematical measures for sources and channels. Introduction to rate distortion theory. Channel capacity, source and channel coding theorems.
Course Learning Outcomes
By the end of the course, the students will be able to
1. Comprehend how to quantify information and uncertainty.
1.1 Calculate the information content of a random variable as generated from a random experiment based on its probability distribution.
1.2 Measure the information content of multiple random entities.
1.3 Determine the statistical relation between random variables based on mutual information.
1.4 Utilize identities and inequalities commonly used in information theory.
1.5 Analyze the characteristics of typical sequences based on information theoretic measures.
2. Apply concepts of information theory to the data compression problem.
2.1 Sketch the proof regarding the limits of data compression.
2.2 Design efficient data compression schemes for a given information source.
2.3 Interpret performance of a data compression scheme based on source coding theorems.
2.4 Differentiate between lossless and lossy source coding techniques and their performance limits.
3. Apply information theory to noisy communication channels.
3.1 Define channel capacities and properties.
3.2 Sketch the proof regarding the limits of error-free communication.
3.3 Understand the impact of channel coding on noisy communication.
3.4 Calculate the capacity of communication channels.
3.5 Generalize the results for the discrete channels to continuous channels and signals.
Program Outcomes Matrix
| Contribution | |||||
| # | Program Outcomes | No | Yes | ||
| 1 | Depth: Our graduates acquire in depth knowledge in one of the various specialization areas of Electrical and Electronics Engineering, they are informed about current scientific research topics and they implement innovative methods. | ✔ | |||
| 2 | Breadth: Our graduates get familiarized in other subspecialty areas related to their specialization in Electrical and Electronics engineering and/or relevant areas in other disciplines. | ✔ | |||
| 3 | Research: Our graduates acquire the skills to conduct and to complete scientific research by accessing contemporary knowledge in their specialty areas. | ✔ | |||
| 4 | Life-long learning: Our graduates develop their life-long learning habits. | ✔ | |||
| 5 | Communication skills: Our graduates concisely communicate their ideas and work related results in written and oral form. | ✔ | |||
| 6 | Ethics: Our graduates internalize rules of research and publication ethics as well as professional ethics. | ✔ | |||