I took Machine Learning with Professor Hsu in Fall B 2020 and I truly loved the course. He is a very chill person but also obviously a genius (that h-index though!); that combination made the class really rewarding, at least in my opinion. I really liked his lecturing pace and methodology, as he would carefully walk us through the math of everything step-by-step, so that we could truly understand the statistical and linear algebra foundations of all the different modeling algorithms. I also really enjoyed the homeworks; I felt they were highly challenging but doable in the end -- especially with groupwork!! Maybe the downside is that the class is not curved, so I felt high pressure to achieve perfect grades on everything. But, that's better than getting straight 50s in Verma's class, in my opinion; you can focus on truly understanding and appreciating the material as opposed to "beating the curve." Overall, lovely class, and I miss it.
This was an advanced class in learning theory that focused on "interactive learning" which attempts to model situations where a "learner" can "interact" with his "teacher". Each week, we read a few papers in the field and went through the details in class. At the start of the class, we did some classic results in using expert advice and bandit problems; this material is definitely worthwhile if you have not seen it before. But the rest of the papers were a lot more recent. However, I was not particularly inspired by the papers; they seemed to spend a lot of time defining new learning models which were often infeasible in practice and not overwhelmingly interesting in their own right. Aside from definitions, few of the actual algorithms had novel proofs. More than half of the classes had student lecturers which were of a rather inconsistent quality. I think I would have learned more if Professor Hsu had taught them all. Hsu also has a bit of a stick up his rear. He spilled a lot of ink making very detailed class requirements, but in the end he gave almost everyone near-perfect grades. He constantly complained about barely late students and has needlessly specific instructions for everything, which made the assignments more work than they were worth. Often, he does not seem to understand student questions or answers them kind of dismissively. Despite being a good lecturer and an expert in the area, Hsu is not a great teacher because he lacks good faith in his students.
I am about to say some things that will scare you, but when it comes down to it, you should take this class. Daniel: Daniel is an expert. This is both good and bad. It is bad because he will sometimes explain things as though we are all on his level, when really no one else is. Additionally, the homework problems can be very challenging and overly mathematical. However, Daniel is more than happy to answer any questions during lectures, so make sure to ask if something gets glossed over. Additionally, you will really understand machine learning after taking this course. I recommend that you print out the slides and take notes on them, as for me they were useless without commentary (because I am not on Daniel's level). TAs: Our TAs tried really hard, but a few of them fell very short. They often could not help with homework and were unable to answer questions about exams. However, homeworks were graded fairly leniently to make up for this. I was fine with the compromise as I still learned a lot, though it was very frustrating at times when trying to complete difficult homework problems. The best thing was there were a lot of TAs and there were at least one set of office hours each day. Homeworks: Most homeworks involved a MATLAB component that was never too difficult, and was always my favorite part of the assignments. It was cool to actually apply everything and have a classifier that had a low error. Some people thought that these parts relied too heavily on understanding the tricks of MATLAB, but I always just had long runtimes and didn't bother too much with optimization. I never got points taken off for this. The written parts of the homework could be too mathy for me. Additionally, everything had to be typeset so you couldn't just write it out and then take a picture of it, which always added at least an hour to the write-up. Exams: The first exam was extremely long. I don't think anyone finished early. It was tough but I think it worked out pretty fair with the curve. The second exam wasn't so long but I thought it was extremely hard. There were some questions that were straight linear algebra without even having an ML twist on them, which I thought was ridiculous as I had taken CLA 2 and a half years before. However, the curve on that ended up okay too and I think the curve overall was also very fair. I did put in at least 90% effort into this class, but I still ended up with a higher grade than I expected. Readings: There is a statistics textbook that I couldn't even read it was so over my head. However, the actual ML textbook can be very helpful and is great to read after lectures to clarify things. There weren't as many readings in the second half of the class and I pretty much stopped doing them after the first exam.
This is a decent class where you will be challenged, put up with lousy instruction but learn a good deal in the process. Daniel Hsu is a super smart professor but about as engaging as a piece of wood. He knows the material in and out and leave you with an appreciation for ML by the end. The class runs pretty smoothly from an operational perspective. The response time on Piazza is insane - Daniel gets back to you in a few hours, especially right before HW deadlines and exams. His answers aren't generally helpful. You see a lot of monosyllabic "No"s as answers and he will put you down if your question is stupid or poorly worded (thank god for anonymity). However, when he does deem your question worthy, you get a thorough response. The course assumes you have a decent background in calculus, statistics and linear algebra. Daniel does not pull any punches on the math. If you've taken Calc III, Linear Algebra and Intro to Probability and Stats, you should be fine. You have to be familiar with some matrix calculus, basic probability and stats (multivariate Gaussians, expectations, Baye's rule), projections, eigenvectors and SVD from linear algebra. Daniel focuses a lot on a lot of theory in machine learning and less on practice. Lots of long proofs and details on the math behind classification models. I personally think this is excellent since ML is inherently a very mathematical subject and understanding it at this level is key. We covered MLE, generative models, decision tress, linear classifiers, perceptrons, SVM, kernels, some ML theory, boosting, regression, PCA, clustering, entropy, HMMs and some math behind a lot of these topics in detail in their own dedicated lectures. Lots of good stuff! It was impossible to stay awake in class since he turned off the lights in 501 NoCo and is a pretty boring lecturer. He doesn't engage with students unless someone asks a question. Thankfully, his slides are great and available online. I encourage you to try to pay attention in class because sometimes the slides are hard to understand on your own right before the exam at the last minute. The homeworks were reasonable and consisted of a mix of theory and programming questions. They were challenging but nothing you couldn't do by spending some time pouring over the slides and going to office hours. Some of the theory questions were hard and it took my hours reading lecture notes or going to office hours to figure them out. The programming part tends to be simple but you have to vectorize your code for it to take minutes not hours, which can be a bit tricky. Assuming you don't pay attention in lecture (because it's really hard to) and that you have to read the slides from scratch to solve the HW, it takes 8+ hours of work on average. TAs were not helpful since they often couldn't solve the homework themselves. His exams are impossible. Numerical grading on tests and homeworks was very generous and countered the difficulty of the exams. However, his letter grade assignments are on the harsh side. I heard StatML with Cunningham is fantastic. He's a great lecturer and focuses on intuition. If you want a more practical course, I would suggest that. If you're a CS major in the Foundations or AI tracks and need ML to count towards your track requirements, you're out of luck. Daniel's class is not bad and I would take it again even though it was a little painful.
The only reason not to take this class with Daniel Hsu is if you already have a full workload for the semester. It's great but pretty difficult. The overall curriculum of the course is heavily mathematical. There is relatively little concern with "real-world" issues like cleaning data or choosing good features. There were some students who sounded like they wanted a course that would teach them how to apply canned algorithms from sklearn or something. It's more about how to implement those algorithms. The first half covered classification problems of various kinds, followed by some learning theory. There's some statistics which comes from a really interesting and unique perspective. It's the first time I've felt statistics wasn't boring. The second half covered unsupervised learning (clustering, etc.) and Markov models. Sprinkled throughout are theoretical discussions of optimization techniques used to fit various models. Finally, the last few lectures were just topics that the professor thought would be enriching, most notably a lecture on how rating prediction systems like Netflix actually work. The class structure is pretty normal -- readings (that you can get by without doing), lectures, and problem sets, plus two exams. The course had a Piazza board, which the professor was always watching. He would give clear, excellent answers, even at weird hours in the middle of the night. He also remained available the night before a problem set. I thought it was nice that he didn't pretend we had good time management skills, although I agree floating late days would have been good, or at least penalties for late assignments rather than refusing to accept them outright. The lecture slides are all posted online and are detailed and excellent. I didn't need to take any notes. But this doesn't mean it's not worth going to lectures! Prof. Hsu is a good lecturer, although sort of quiet. His explanations perfectly complemented the slides. When students asked questions, he listened carefully and gave very clear and cogent answers. We covered a ton of ground, at a decent level of mathematical sophistication. I think perhaps the professor overestimated our mathematical competence. However, those high expectations along with his good teaching skills made this a great class. Hopefully Columbia keeps him around.
TL;DR: Hsu is a solid professor if you're a math-y sort of person who wants derivation, proofs and (by CS standards) rigor. But if you want an easy ML survey course, go take AI, which is pretty much that. The good: Professor Hsu is by no means the most charismatic lecturer at Columbia, but he teaches a solid ML course. His lectures are thorough and rigorous -- he does not succumb to the temptation to use teaching a survey course as an excuse to skim easy topics at a high level -- and the online lecture notes are quite good. Also, it was really nice that the two textbooks the course uses are both available online for free, legally, and that he provided us links to them, unlike so many professors who assign the $300 latest edition of [insert textbook] without giving students' finances a second thought. I was absolutely amazed how fast he responds to Piazza questions; he was single-handedly more responsive than a crew of about 8 TAs in another of my classes. The responses, always within a few hours, are sometimes not what the poster is hoping for (monosyllables aren't unusual: quite a few questions get a "no." and no more, particularly if they're stuff like extension requests), but the question is invariably answered in full. And that means stuff like "what should we review for the final?" gets answered with a 28-point bulleted list. The bad: No floating late days. There's a reason almost every CS class offers them, and it's both because they avoid all the whining on Piazza from people who put their work off to the last minute and because they provide a way to grant a bit of deadline flexibility without being unfair to anyone. Professor Hsu, if you read this, you really should consider having a few floating late days. The first midterm was to a considerable degree just a speed test (caveat: I didn't feel that way; I mostly missed points for not being able to remember material. But everyone else I spoke to felt it was a speed test) and the curve was, by CS department standards, fairly tough. The lecture hall (the huge one in NoCo) was dimly lit and has poor acoustics, which didn't make staying awake through lecture any easier given that Hsu isn't exactly lecturing at John McWhorter levels of energy most of the time.