How to Learn Linear Algebra
Linear algebra is the most useful mathematics most people were taught badly. The standard course opens with row reduction and determinants — bookkeeping a computer does better than you ever will — and postpones the one idea that makes everything else obvious: a matrix is a function that moves space. Plan on about 200 hours for a genuinely solid command of the subject, or roughly 60 focused hours if your only goal is to read machine learning papers without the notation stopping you. The prerequisite that trips adults up is not arithmetic; it is fluency in reading notation as a description of a process.
Updated
Why Learn Linear Algebra?
It is the actual mathematics of machine learning
Embeddings are vectors, layers are matrix multiplications, attention is a product of learned projections, and dimensionality reduction is a decomposition. When practitioners say a model is "just linear algebra with nonlinearities sprinkled in," they are being close to literal. Every other piece of ML math sits on top of this one.
It is the highest-leverage math per hour spent
Calculus takes hundreds of hours before it changes what you can build. Linear algebra pays out early: knowing what a projection is gets you least-squares regression, and knowing what eigenvectors are gets you PCA, PageRank, stability analysis, and vibration modes. Few subjects convert this directly into working tools.
It teaches you to reason about high dimensions
Data does not live in three dimensions, and your visual intuition quietly lies about what happens in three hundred. Linear algebra gives you algebraic handles — rank, norm, orthogonality, condition number — that keep working when pictures fail. That is the difference between using an embedding and understanding one.
It is the shared language across fields that never talk to each other
Computer graphics, quantum mechanics, econometrics, structural engineering, control theory, and network analysis all reduce to the same handful of matrix operations. Learn them once and you can read technical work far outside your own field, because the objects are identical even when the vocabulary is not.
Your Learning Path
Start with vectors as geometry, not as lists
~20 hoursAddition, scaling, dot products, length, and angle in two and three dimensions, drawn every time. The dot product is the one operation worth over-learning early: it measures alignment, and every later idea — projection, orthogonality, correlation, cosine similarity — is that measurement wearing a different name.
Learn a matrix as a transformation before a table of numbers
~25 hoursSee what a matrix does to a grid: rotate, stretch, shear, flatten. Then derive matrix multiplication as composing two of those moves, which is the only explanation that makes the row-times-column rule and the non-commutativity feel necessary rather than arbitrary. Most people who find linear algebra meaningless skipped this reframe.
Solve systems, and learn what a solution set looks like
~30 hoursGaussian elimination, pivots, rank, and the geometry of "no solution," "one solution," and "infinitely many." The real payload here is not the algorithm but the rank-nullity relationship: how much information a transformation destroys, and why that predicts whether your data problem is even solvable.
Take the abstraction step: vector spaces, span, basis, dimension
~30 hoursThe chapter everyone tries to skip as "theory" is the chapter that makes the rest cheap. Once a basis is a choice of coordinates rather than a definition to memorize, change of basis, diagonalization, and feature transformations all become the same idea. Expect this to be the slowest week and to feel like it is not paying off yet.
Get orthogonality, projections, and least squares
~30 hoursOrthogonal bases, Gram-Schmidt, the projection of one vector onto a subspace, and the normal equations. This is where linear algebra hands you linear regression as a geometry problem — fitting a line is projecting your data onto the space your model can reach. Statistics stops being a set of formulas at this step.
Learn eigenvalues and eigenvectors as behavior over time
~30 hoursEigenvectors are the directions a transformation does not rotate, and eigenvalues say how much it stretches them. Frame them as "what happens when you apply this repeatedly" and Markov chains, PageRank, stability, population models, and PCA all become one question asked in six vocabularies.
Reach the singular value decomposition and use it
~25 hoursSVD works on any matrix, decomposes it into rotate-stretch-rotate, and ranks directions by importance. It is the practical engine behind PCA, low-rank approximation, recommender systems, and image compression. If you only fully understand one decomposition, make it this one rather than the eigendecomposition.
Finish with numerical linear algebra and real libraries
~20+ hours, ongoingFloating-point error, condition number, why nobody inverts a matrix in production, and how NumPy or a GPU actually performs these operations. Hand computation teaches concepts; this step teaches you why a mathematically valid method still returns garbage on real data at real scale.
Common Mistakes to Avoid
Learning matrix multiplication as a procedure instead of composition
For every product AB you compute, say out loud what each matrix does geometrically and in which order the moves happen. Do the 2x2 cases on graph paper with a drawn unit square until "AB means do B, then A" is reflexive. Everything from dimension mismatches to why AB is not BA follows from that one sentence.
Grinding row reduction by hand for weeks
Elimination is worth doing by hand perhaps a dozen times, to see pivots and rank appear. After that, delegate it. Spend the reclaimed hours on the two things a computer cannot do for you: choosing the right representation for a problem, and interpreting what the numbers mean once it finishes.
Skipping the abstract vector-space material as impractical
Do not skip it — instantiate it. Every time you meet a definition (span, basis, null space), write down two concrete examples and one non-example: polynomials of degree at most three, real-valued functions, a plane through the origin. The abstraction becomes usable the moment you can produce your own instances of it.
Treating the determinant as a formula rather than a signed volume
Learn it as the factor by which a transformation scales area or volume, with sign meaning orientation flip. Then "determinant zero means non-invertible" needs no memorization — it means space got flattened and information was destroyed. Compute the 3x3 formula rarely; use the meaning constantly.
Studying entirely on paper and never in code
After each concept, spend twenty minutes in NumPy: build the matrix, apply it to points, plot the result, check your prediction. Doing this exposes gaps paper hides — sign errors, dimension confusion, ill-conditioning — and it builds the muscle you will actually use, since no professional does this arithmetic by hand.
Structured Roadmaps
Follow a guided learning path on Mochivia:
Frequently Asked Questions
How much linear algebra do I need for machine learning?
Is linear algebra hard?
Should I learn linear algebra or calculus first?
Do I need calculus before linear algebra?
How long does it take to learn linear algebra?
Why does linear algebra feel pointless while I am learning it?
Learn Linear Algebra in short lessons
Interactive lessons, a few minutes each, in the Mochivia app for iPhone.
See the roadmap →