Course Syllabus

Course description

This course provides a place for students to practice the necessary mathematical background for further study in machine learning — particularly for taking 10-601 and 10-701. Topics covered include probability, linear algebra (inner product spaces, linear operators), multivariate differential calculus, optimization, and likelihood functions. The course assumes some background in each of the above, but will review and give practice in each. Some coding will be required: the course will provide practice with translating the above mathematical concepts into concrete programs.

The course is split into two minis, which form a sequence (10-606 is a prerequisite for 10-607).

Lecture time and location

Time: MW 1:30-2:50p
Location: Lectures, assignments, and tests will be conducted completely online. There is no physical classroom. Office hours will be available either online or in person.

You are expected to view and participate in all lectures synchronously. (It's OK to use DVR features like pause and rewind — being a few minutes off of real time shouldn't be harmful. If you pause the stream, YouTube can play at 150% or 200% speed so that you can catch up to real time.) While we do not take attendance, there may be graded assignments (such as quizzes) during lectures.

Accessing online lectures and office hours

Online lectures will be streamed live on YouTube, at https://www.youtube.com/user/professorgeoff/live. Past lectures are saved in a playlist at https://www.youtube.com/playlist?list=PL7y-1rk2cCsAqRtWoZ95z-GMcecVG5mzA. If you have questions during a lecture, please post them on Piazza, or contact a TA directly. The TAs will monitor Piazza and answer questions during lectures; they will let me know about any common questions, and I'll try to answer them in the lecture stream. There is a short extra delay, typically a minute or two, due to buffering in the stream.

To attend office hours online, you can use Google Hangouts (see contact information on Piazza).

Contact information

The most up-to-date contact information for the professor and TAs is on Piazza: https://piazza.com/cmu/fall2017/1060610607/staff. Your professor is Geoff Gordon; the best way to contact me is to post on Piazza. (Note that Piazza has an option to post privately if desired — e.g., if you are asking questions that reveal details about the solution to an assignment.)

Topics and schedule

The main topics of the course include

• Logic
• Sets and types
• Probability
• Linear algebra and vector spaces
• Functions and function spaces
• Matrices and linear operators
• Matrix differential calculus
• Statistics and inference

Please see the live syllabus post on Piazza, https://piazza.com/class/j1z1bzwb6tb7cd?cid=8, for an up-to-date schedule of topics. For the schedule of assignments, see the Assignments tab of Canvas, or the Course Summary below. (For assignments that are not submitted through Canvas, we will add a dummy assignment on Canvas to show the due date.)

The preliminary topics for the second mini include

• Information and entropy
• Distributions and modeling
• Linear regression
• Gaussian graphical models
• Optimization

We will post a more detailed schedule for the second mini as it approaches. (The second mini will also continue to build on the topics listed above for the first mini.)

Textbooks

There are no required textbooks for this course. However, the following textbooks are recommended as a way to get an alternate presentation of some of the course material:

• Matrix Analysis (2nd ed.). Roger A. Horn, Charles R. Johnson. Cambridge University Press, 2013.
• Introduction to Probability (2nd ed.). Dimitri P. Bertsekas, John N. Tsitsiklis. Athena Scientific, 2008.
• The Elements of Statistical Learning (2nd ed.). Trevor Hastie, Robert Tibshirani, Jerome Friedman. Springer, 2008.

Tools

The main tool to access course content is Canvas: https://canvas.cmu.edu/courses/603. We'll link to all other tools from there.

We'll use Piazza for questions and discussion. Our class page is https://piazza.com/cmu/fall2017/1060610607/home, or use the Piazza tab from the navigation menu in Canvas.

Most coding assignments for the course will be in Python, although we may use other languages (such as Matlab) from time to time.

We plan to use several tools to handle submission and feedback for assignments. Some assignments will be submitted directly through Canvas. Others will go through Gradescope (paper-based assignments) or repl.it (coding assignments).

We'll use YouTube and Google Hangouts for course video and audio: YouTube for streaming, and Hangouts for conferencing.

Course time conflicts

Some students have asked whether — as an online-only-course — it is possible to take 10-606/607 even though the listed lecture time conflicts with another course. Unfortunately the answer is typically no: while the class does not meet in person, it does meet synchronously, and there will be work assigned during class periods (such as quizzes).

Required background

A few people have asked what kind of background is required for this course:

• We're assuming mathematical maturity: you should be familiar with the idea of derivations and proofs, and of building formal representations for objects given their English-language description.
• We'll assume that you can take scalar derivatives.
• We'll assume that you've seen real vector spaces and matrices (linear operators) and know how to reason about things like matrix multiplication and solving systems of linear equations.
• We'll assume that you've seen sets and data types.
• We'll assume that you've worked with probabilities (e.g., conditioning, Bayes rule).
• We will also assume familiarity with coding. Most of the assignments will be in Python, with possibly some Matlab/Octave.
• For the above topics, we intend to give complete definitions and/or resources to learn from, but often we will go over them quickly: we expect that most people will know a lot of the definitions, and will be willing to work outside of class to understand the ones they don't know. So, for each topic that you're unfamiliar with, you should expect to do some extra work; if you're unfamiliar with several of them, that could make it hard to keep up.

Learning objectives

Given the above background, our goal is to build up familiarity with the topics listed under "topics and schedule": by the end of the course, students should be able to fluently solve problems that involve these topics. Examples include rapidly taking derivatives of matrix-valued expressions, and solving for the probability of an outcome given a description of a probabilistic model.

Grading

Participation: 5%
Participation includes questions and answers on Piazza as well as discussions with course staff inside and outside of office hours. You will be graded on how much work and insight you bring to these interactions, not the raw number of them.

Exam: 15%
There will be an in-class exam on the last day of class: see the schedule on Piazza. You'll be able to turn in this exam online; we'll post details prior to the exam.

In-class quizzes: 10%
We will frequently give short quizzes in class. These will typically be due at the end of the class period. Each quiz will be equally weighted, unless we announce otherwise: e.g., a 5-point quiz will be divided by 5 before averaging. Late submission of quizzes will not be allowed; instead, you may skip up to two quizzes for any reason, and we will replace them by your average score on the remaining quizzes.

Short problem sets: 30%
We plan to have 7-10 short problem sets, each taking 1-3 days. Each will be equally weighted, unless we announce otherwise. Many of these problem sets will contain mastery problems: problems where you submit repeatedly, with automated feedback each time, until you get a correct answer. For example, some problem sets will contain coding problems that you submit until you pass a set of unit tests.

Homeworks: 40%
These are larger, longer-term assignments. We will have 2 or 3 homeworks, equally weighted.

Extra credit:
We may announce opportunities for extra credit. These will truly be optional: your grade cannot decrease if other students make more or better extra credit submissions.

Grade cutoffs:
≥ 97% A+
≥ 93% A
≥ 90% A-
≥ 87% B+
≥ 83% B
≥ 80% B-
≥ 77% C+
≥ 73% C
≥ 67% C-
otherwise F

Late or missing work

In all cases, students must, before the original deadline, request (and receive) permission to submit late work. We will not grant permission to submit after solutions are released, nor more than three days after the submission deadline.

There will be a penalty for late or missing work, unless we explicitly say otherwise. In exceptional cases we may accept late work without penalty. Such cases include illness and religious obligations, so long as advance planning would not have helped to avoid late submission.

Quizzes will not allow late submission. Instead, students may skip up to two quizzes. If you need to skip more than two quizzes for exceptional reasons (e.g., an extended illness or multiple religious obligations), please contact us for accommodation.

Auditors and pass-fail students

If you are an auditor or a pass-fail student, you will be graded on participation, in-class quizzes, and short problem sets. That is, you may skip the homeworks and the exam, although you are welcome to submit all or part of these (without credit) for feedback if desired. To pass, you must achieve at least a 67% average. Grading criteria are identical for auditors and pass-fail students.

To audit this course, please register as usual, then ask the instructor to sign an audit form. (The best way to do this is to email a partly-filled-out PDF form to the instructor.)

You should be able to switch to pass/fail grading without intervention from the instructors.

Regrades

We allow regrades by request. The procedure for requesting regrades depends on the tool used to submit the assignment. You must request regrades within a week after you receive feedback on an assignment; we may announce an earlier deadline for assignments near the end of the mini. If you request a regrade on part of an assignment, we may look at other parts of the same assignment as well, and your overall grade may go up or down.

Collaboration

It goes without saying that you are expected to submit your own work for every assignment. That means both that you should not copy someone else's work, and that you should not allow someone else to copy your work.

It is fine to form study groups, and to discuss general strategies for solving problems — in fact, we encourage this (except on tests or quizzes). But, when it comes time to work out the actual solution to a problem, you should put away all of your notes from study groups or discussions with other students, and work out the solution on your own. If you work directly from such notes, it's too easy for the solution process not to stick in your brain; putting away the notes forces you to reconstruct the solution process on your own.

You must disclose any collaborations. A note to the instructor is a good way to do so. Please include the Andrew IDs of your collaborators as well as a description of the collaboration — e.g., "I discussed the solution strategy for HW2 problem 3.1 with bovik@andrew.cmu.edu."

We may use statistical analyses or other methods to check for suspicious behavior. Penalties for cheating may include reducing your grade for an assignment, reducing your grade for the entire course, or failing the course (all without the option of withdrawing). These penalties are in addition to any disciplinary action that the department, college, or university may impose.

Accommodations for Students with Disabilities

If you have a disability and have an accommodations letter from the Disability Resources office, please discuss your accommodations and needs with me as early in the semester as possible. I will work with you to ensure that accommodations are provided as appropriate. If you suspect that you may have a disability and would benefit from accommodations but are not yet registered with the Office of Disability Resources, I encourage you to contact them at access@andrew.cmu.edu.

Statement of Support for Students’ Health & Well-being

Take care of yourself. Do your best to maintain a healthy lifestyle this semester by eating well, exercising, avoiding drugs and alcohol, getting enough sleep and taking some time to relax. This will help you achieve your goals and cope with stress.

All of us benefit from support during times of struggle. There are many helpful resources available on campus and an important part of the college experience is learning how to ask for help. Asking for support sooner rather than later is almost always helpful.

If you or anyone you know experiences any academic stress, difficult life events, or feelings like anxiety or depression, we strongly encourage you to seek support. Counseling and Psychological Services (CaPS) is here to help: call 412-268-2922 and visit their website at http://www.cmu.edu/counseling/. Consider reaching out to a friend, faculty or family member you trust for help getting connected to the support that can help.