Gilbert Strang still teaches with thick chalk and four subspaces

Lex Fridman introduces Gilbert Strang as a professor of mathematics at MIT and one of the most famous teachers of the subject in the world. His OpenCourseWare lectures on linear algebra have been viewed millions of times; Fridman, as an undergraduate, was one of those millions, working exercises in *Introduction to Linear Algebra* and watching matrices become tools for signals, images, optimization, robotics, vision, deep learning, and graphics. Asked how it feels to be a modern-day rock star of mathematics, Strang says he does not feel like one. That is crazy talk for an old math person. What is true is that the linear-algebra videos he made way back in 2000 have been watched a lot.

Part of the reason, he says, is the subject itself. Linear algebra has surged in importance. Part of it is that he had taught the class a bunch of times, so it was organized, and he enjoyed doing it. The videos were just the class, posted on OpenCourseWare and YouTube and translated. Fridman presses on the chalkboard and the simplicity of the early explanations; at his own university the course used Strang’s book, and Strang was, in effect, his instructor. Putting lectures online was, Strang thinks, a wonderful idea. The story he has heard is that President Charles Vest appointed a committee to figure out how MIT could market its work the way other universities did. They did not see a way. After a weekend they came back with an inspiration: what if we just gave it away? Vest decided that was OK.

If a university is an institution that creates a product, Fridman notes, knowledge is the product, and giving it away sounds crazy. Knowing a little of Vest, Strang says, it was like him, and it was the right idea. MIT is known for high-level technical things, and this is the best way to show what MIT is really like. In his case the 18.06 videos are just teaching the class. People write to say he has a sense of humor; he is not sure where that comes through, except that he is friendly with a class, likes students, and likes the subject. Linear algebra deserves most of the credit for having come forward in these years.

Asked for a favorite topic to lecture on, a story to tell students, Strang stays on the teaching side. It is not deep mathematics at all, but he is proud of the four fundamental subspaces. The idea was known long before he named it. A matrix is a rectangle of numbers, n columns and m rows, the same numbers in both directions, so there have to be connections, but they are not simple: columns may be longer than rows, and the numbers are mixed up. The first space to understand is the columns, which are vectors, points in n dimensions. For Strang a vector is a column of numbers more than an arrow; he is in high dimensions right away, in n-land.

In lecture he will readily admit that he has no good image of an arrow in ten-dimensional space. You can still add one bunch of ten numbers to another and multiply a vector by three. If you know those two operations, you have linear algebra in ten dimensions. Fridman asks about the gap between string theory’s extra dimensions and what we can visualize. Strang says he is not a very geometric person; he is probably thinking in three dimensions. The beauty of linear algebra is that it goes on to ten dimensions with no problem. Adding two vectors in 3-D is adding components; in 10-D you just add ten of them. He cannot say he has a picture, but he still tries to push the class to think of a flat surface in ten dimensions, a plane.

Take all the columns and all their combinations and you get a flat surface, a vector space. His imagination is a piece of paper in 3-D. That is space number one, the column space. Then there is the row space, different, but from the same numbers: all combinations of the rows. Those words are easy to say and hard to draw, even with his thick railroad chalk, which everybody likes, and which he would not now use anything else. The other two spaces are perpendicular to those. A plane in 3-D has a line perpendicular to it; that is the null space. Column space, row space, and two perpendicular spaces: four of them, fitting together in a beautiful picture of a matrix. It is not a difficult idea. It comes pretty early in 18.06. Calculus, he admits, came earlier historically, with Newton and Leibniz, and feels easier to see into. Linear algebra, to him, is the starting point, because it is all about flat things. The complications of calculus come from curves. In linear algebra the surfaces are all flat. Nothing bends.

Calculus also comes first in high school and as freshman math, AP calculus, until Strang says enough of it, get to the good stuff. Should linear algebra come first? He is OK with it not coming first, but it should get simpler billing because everything is flat. Calculus sticks to one dimension, or multivariate, which basically means two. Linear algebra takes off into ten dimensions with no problem. It feels scary to go beyond two dimensions, but if everything is flat you cannot go wrong.

The most beautiful concept, if he sticks to linear algebra, starts from a matrix as a rectangle of numbers, maybe of data: every column a drug, every row a patient, a positive number if the patient reacted well. The big problem is to understand all those numbers, the patterns. One way to break the matrix into simple pieces is singular values, which became fundamental in his lifetime. Eigenvalues are in every engineering-math course, but they are restricted to square matrices, and data comes rectangular. You have to take that next step. He is always pushing math faculty: do singular values. They find the important pieces of the matrix, which add up to the whole. The first piece is the most important part of the data, the second the second most important. If a data scientist can find those first and second pieces, stop there. The rest is probably roundoff or experimental error.

The theorem is that every matrix, rectangular or square, can be written as a product of three very simple special matrices: a rotation, a stretch (a diagonal matrix, otherwise all zeros except on the one diagonal), and another rotation. Rotation, stretch, rotation is the breakup, the structure. Geometrically the action of a matrix is not so easy to visualize, he freely admits, but everybody can visualize a rotation: two-dimensional space turned around the center, three-dimensional space with a pilot’s three turns, ten dimensions with ten ways to turn. A stretch is equally visible. Breaking a matrix full of numbers into something you can see, rotate, stretch, rotate, is pretty neat.

Fridman has been watching what people connect with on YouTube and keeps seeing math, not only Strang’s lectures but channels that chat about extremely complicated things. Strang thinks it is wonderful. We are conditioned to think math is hard, abstract, just for a few people, and it is not that way. A lot of people quite like it. He gets messages from retirees who have decided to learn more math, which he finds encouraging. What people like is that there is order, a lot of order, and things are not obvious but they are true. Math reveals what is true, Fridman says, that is the point of proving things, and yet the real world is messy. Strang is not much of a philosopher. He just likes numbers. In the dentist’s chair, before you had to go in when they were filling teeth, he would take powers of two: two, four, eight, sixteen, until it stopped hurting. The dentist said he was counting. You know where you are. Two multiplied by itself ten times is 1024, period. Everybody is going to get that.

Is math a powerful tool or an art form? Both, and that is one of the neat things. You can be an artist and like math; you can be an engineer and use it. Strang is in between, certainly not an artist-type, philosopher-type person, even if he sounds that way this morning. He really enjoys teaching engineers. They go for an answer. Within the MIT math department, most people enjoy teaching students who get the abstract idea. He is good with engineers looking for a way to find answers. For new concepts, many of us like examples first, even a pretty abstract-sounding one such as a three-dimensional rotation, and some of us keep going until there are numbers, ten axes, ten angles. The great mathematicians probably work in the space of examples too; for them an example might be a highly abstract thing to the rest of us. Brains connect with structure in examples.

Fridman mentions Andrew Yang, then a presidential candidate, with MATH on his hats, Make America Think Harder. Strang will vote for that. Yang’s name even rhymes with his. Math, science, and engineering are not really part of political discourse. We need people comfortable with numbers and quantities, people who, if you say this leads to that, see it as undeniable. It is too bad there are almost no elected officials in Congress, or obviously the president, with engineering degrees. You would need people who understand engineering or science and can also make speeches and lead. As president of SIAM around 2000, a pretty small society, Strang caught a moment when math was getting more attention in Washington. He got ten minutes with a House committee, and one member, from New Hampshire, had been in his class. Those years were good; then other things took over, and math is not so visible. There is excitement and concern about artificial intelligence in Washington now, and the core of that is math, maybe hidden, wearing a different hat.

Data is swamping the computers of the world. Deep learning is a particular approach to understanding it, and it uses a lot of linear algebra, which is how Strang got into it. He starts from the idea: you have inputs for which you know the right outputs, and you want a pattern, a rule, that will take a new, unseen input to an output. A million training pairs, known outputs, and then the rule on new inputs. A neural net is part of the structure of that new way to create a rule. Linear algebra is a big part of finding it, not all of it. Matrices still do something special: straight lines and flat planes. Data is not quite like that. You have to introduce some function that is not a straight line.

It turned out to be enough to use a function that is one straight line and then a different one halfway: piecewise, one piece with one slope, the other with a second. That simple nonlinearity opened the problem. You use that piece over and over, a million times. The graph has a fold. Fold something a million times and you have a pretty complicated, pretty realistic function. Why does formulating an objective that is not a plane, a function with lots of folds, find a rule that is not known to be optimal and still seems pretty good? Strang is beginning to have a better intuition. Piecewise-linear flat pieces with folds between them are like the roof of a complicated house, almost curved, every piece flat. Engineers already use that idea big-time: the finite-element method. Design a bridge, a building, an airplane, and you are using piecewise flat as a good, simple, computable approximation. The expressive power, how complicated a thing those piecewise-flat guys can express, is very complicated. The old limit was computational; supercomputers keep getting more powerful, so that limit has moved to allow more and more complicated surfaces.

How much of the world can be expressed this way? The whole idea of deep learning, Strang says, is that there is something there to learn. If the data are totally random, produced by random-number generators, you will not find a useful rule, because there is not one. The other extreme is knowing Newton’s law: hit a ball and it moves. Newton, Einstein, and petroleum engineers who understand how oil sits in an underground basin found rules. The new idea of artificial intelligence is to learn the rules instead of figuring them out with help from Newton or Einstein. The computer is looking for the rules. That is another step. If there are no rules at all, if it is totally random, you have no science to discover. It is an automated search. There will be a lot of random parts; he is not knocking randomness, there is a lot built in. There has got to be some signal. If it is all noise, you are not going to get anywhere. The world around us does seem always to have a signal of some kind, waiting to be discovered.

For himself Strang is a theory person, speaking freely about applications without being a physicist, chemist, or neuroscientist. He likes the structure, the flat spaces, the relation of a matrix’s columns to its rows. Science is a spectrum from practical questions, through people like him in the middle, to the geniuses finding fundamental rules. Among big fields, algebra as a whole sits beside calculus and differential equations, and geometry deserves to be thought of separately, the geometry of high-dimensional surfaces. His personal view of undergraduate math, what millions of students study, is that we overdo calculus at the cost of algebra. In his lifetime the importance of data, which comes in matrix form, set the subject up for algebra, not for differential equations. Probability and statistics have also jumped forward. We now have three major areas: calculus, linear algebra and matrices, and probability and statistics. They all deserve an important place. Calculus has traditionally had the lion’s share, a disproportionate share of the love and attention from excited young minds.

His favorite matrix is square, with twos running down the main diagonal, minus ones just above and just below, and otherwise all zeros: three nonzero diagonals. It comes up all over engineering. In calculus it is analogous to the second derivative, not just how fast something is changing but how fast the change is changing, how the graph is bending. Einstein showed that is fundamental to understanding space, so second derivatives should have a bigger place in calculus. The linear-algebra versions of those second-derivative matrices are neat. Everything comes out right with those guys.

What has he learned about learning by teaching so many students? He admits he is not really a good teacher, because he does not get into the exam part. Exams, grading, giving A or B, he does because he is supposed to. He tells the class at the beginning, and he does not know if they believe him, that he is here to teach them math and not to grade them. They are thinking when it is going to be, A-minus or B-plus. The sexy part is the start, telling somebody for the first time about a matrix. Fridman asks whether he sees the moment in their eyes when they fall in love with the world, or the opposite, when they say they are just not good at math. Maybe because of the approach in the past they were discouraged. Don’t be discouraged, Strang says. It is too good to miss. In a big class he thinks he knows when they get it: after the four fundamental subspaces and the fundamental theorem of linear algebra, the relation of those four spaces. He feels the class gets it when they want to see.

Advice to a student starting mathematics today is hard. He hopes they have a professor who is still enjoying what he is teaching, still looking for new ways to teach and to understand, because that is the pleasure, the moment when you say, oh, yeah, that works. It is more about the fun, the moment of getting it, than about which topics, though algebra is his. There are beautiful things in geometry. What is wonderful is that in the end there is a pattern, rules that are followed in biology as in every field. The life of a mathematician is much of it wonderful, except for giving the grades. Looking back, he feels good that he started MIT’s 18.06, that a lot of students take it and quite like it. He is happy when he feels he is helping make a connection between ideas and students, between theory and the reader. He gets a lot of very nice messages from people who watch the videos, and he takes the chance to say thank you. Fridman closes with Feynman: study hard what interests you the most, in the most undisciplined, irreverent, and original manner possible.