Skip to content

ECE417

ECE 417 (Multimedia Signal Processing) is a 4-credit-hour course that satisfies the Technical Electives requirement for ECE majors. It is only offered in fall semesters.

Content Covered

  • Characteristics of speech and image signals
  • Subspace methods
  • Bayesian networks
  • Hidden Markov models
  • Biometrics
  • Human-computer interaction
  • Audio-visual databases

The beginning of the course is a review of linear algebra and signal processing, which will be the basis of much of the rest of the course. Then, you'll learn about the STFT, which leads into learning about signal reconstruction from a spectogram with no phase information using an adaptive algorithm known as Griffin-Lim. You'll also learn about neural nets and a specialized type of neural net involving the weighted averages of weak classifiers which is often used for face detection known as Adaboost. As one of the prerequisites to this class is ECE313, there is a bit of review of ECE313 material sprinkled throughout as a preview into the more specialized algorithms for image/audio processing. One of these is the Hidden Markov Model (HMM), which you'll come to learn can be very useful in speech segmentation. The last part of the course has a strong focus on linear algebra, with PCA being an important aspect to recurrent neural nets (RNNs), and a specialized RNN known as LSTM used in many deep learning applications, such as video synthesis. The course closes with a discussion of voice encoding, in which a voice sample of one human speaker can be converted into the voice of another human speaker saying a completely different thing, of which PCA is also an integral aspect to.

Prerequisites

Basically, the course prerequisites are 1. a course in digital signal processing, which is ECE310 for ECE majors; and 2. a course in random variables, which is most of the time ECE313 or sometimes STAT400 for ECE majors. These prereqs are helpful, but not necessary, as they are reviewed throughout the course where needed. Another prereq that may be helpful is linear algebra, whether it be through the math department here at the university or through prior coursework elsewhere, but again, it is not necessary since a review is given in lecture where needed. There is also a bit of overlap between this class and ECE448/CS440 - Artificial Intelligence, so if you take that class concurrently/before this class, it might also be beneficial to your experience in this class.

When to Take It

Though one of the prereqs is ECE310, having an interest in DSP or signal processing in general is not necessary for this course. This course, although named "Multimedia Signal Processing," actually is more focused on specialized ML algorithms which can be applied to speech and image signals, so if you also have an interest in AI/ML, then you should take this course concurrently or after you've taken one of the AI/ML courses offered by the ECE/CS department (e.g. ECE449/CS446 - Machine Learning or ECE448/CS440 - Artificial Intelligence) if you found those to be interesting.

Course Structure

This course has bi-weekly homeworks and MPs, alternating between the two each week (i.e., homework due one week, MP due the next week, homework due the next next week, and so on). This course also has two midterms and a final.

There are six homeworks, all graded on completion only, and 3-4 questions each. They tend to involve deriving expressions discussed in lecture as well as calculations. While they are only graded on completion, it's important to understand the solutions after they're released, as these concepts are fair game for the exams.

There are also six MPs, which, in contrast to the homeworks, are graded on accuracy. The system in place is much like that used in ECE448/CS440 - Artificial Intelligence with a Gradescope submission which runs an autograder on your code and checks if it passes the tests, with half of the tests being visible in the MP starter files. All of the MPs are implementing an algorithm discussed in lecture using Python, so as long as you understand the intuition behind the algorithms in lecture, the MPs should not be too difficult. There are also extra credit opportunities for every MP which are just extensions based off of what you implemented in the base portion of the MP, also not too difficult if you understood the MP.

The exams and final are not insanely difficult, but still tough. For all of them, it is not only important to understand the intutition behind the homework solutions and what you implemented on the MPs, but it is also important to consider everything discussed in lecture and the "edge cases" surrounding the lecture material. The exams tend to be fairly in-depth on the theory discussed in lecture, so be sure to at least understand the lecture material before each exam. You are allowed one double-sided 8.5x11 cheatsheet for exams 1 and 2 (since exam 2 is not cumulative), then three 8.5x11 double-sided cheatsheets for the final (which is cumulative). For each exam, there is one long practice exam given, with questions covering all the topics covered, so it's important to go through and understand the solutions to all of them before each exam. In recent semesters, this course has been taught by Professor Mark Hasegawa-Johnson, and before each exam he'll walk through a few select problems from these practice exams as an exam review in the lecture prior to the exam, so it is also beneficial to attend those specific lectures.

Instructors

As mentioned before, this course has been taught by Professor Mark Hasegawa-Johnson in recent years, which explains why the course structure is very similar to that of ECE448/CS440 - Artificial Intelligence, which he also teaches, but in spring (this course, ECE417, is normally only offered during fall semesters).

Course Tips

Since there is not yet a set of course notes or a recommended text for the course, it's especially important to fully understand the material discussed in lecture. Ask questions in lecture - Professor Hasegawa-Johnson is very receptive and patient when answering questions and wants to make sure you understood him at every step, and if you still feel you need more explanation, attend OH and ask either the TA or Professor more questions, whether it be for homeworks, MPs, or general lecture material.

Life After

If you found the content of this course interesting, you might consider taking more courses in AI/ML offered by the ECE/CS department, including the ones mentioned before (ECE448/CS440 - Artificial Intelligence and ECE449/CS446 - Machine Learning), and also CS441 - Applied Machine Learning, CS443 - Reinforcement Learning, and CS444 - Deep Learning for Computer Vision.

Infamous Topics

  • Neural Nets: they are the basis for many of the algorithms discussed throughout the course and consequently are very important to understand

Resources