17. Orthogonal Matrices and Gram-Schmidt
Vložit
- čas přidán 23. 09. 2019
- MIT 18.06 Linear Algebra, Spring 2005
Instructor: Gilbert Strang
View the complete course: ocw.mit.edu/18-06S05
CZcams Playlist: • MIT 18.06 Linear Algeb...
17. Orthogonal Matrices and Gram-Schmidt
License: Creative Commons BY-NC-SA
More information at ocw.mit.edu/terms
More courses at ocw.mit.edu
Removed long video delay after the title.
Where can I download these videos in 480p? I checked the ocw website, the download links are for 360p or less quality. And, thank you for this awesome course.
That just also removed many comments under the original video.
Sounds like a git commit message
@@biao9957 😂
@@shubhamtalks9718 youtube-dl
The fall from 2M views lecture one to 18k views lecture 17 (half way through) is crazy. Congratulations if you're still here. I've followed every lecture so far (notes and all) and I've gotta say, it is 100% worth my time. I might not have a degree from MIT, but I can say I've experienced how superb their education is firsthand. Thank you to Professor G. Strang and MIT OCW for making these lectures available online. You have made learning for me interesting again.
Lecture 18 has 200k views. The fall is still big, but I believe this lecture has 20k for the last 6 months after the video was replaced.
Hello, this video replaces one from 2009 because the original video had a delay problem, if you look you can see that it was posted only recently in September of 2019. So unlike all the other videos in the series, it didn't have 10 years to accumulate views, but to be honest, seeing that 20 000 people have come this far in the course in only 6 months is quite astonishing!
chill dude why does it matter
calm down bruh, we get it. You're studying lmao.
Well as long as students have to deal with linear Algebra, they will meet Gilbert Strang - lol.
Gram-Schmidt portion of the lecture begins at 26:07 for anyone only interested in that
This should have more likes
Isn't it crazy we still follow 15 years old lecture because its still best. What a legend G.Strang
@glyn hodges no you wouldn't wanna watch pythagoras's videos
" I don't know what Schmidt did" ahhh priceless.
Poor Schmidt. Legacy ruined.
His legacy is already tainted from being a Nazi unfortunately
19 years and still the best lecture on gram-schmidt process on youtube ! hats off sir
In order to understand the A= QR part, my suggestion is to take advantage of the fact that Q^T is equal to the inverse of Q. So, multiply the both sides of the primary equation with the Q^t from the left. You will get Q^t A = R.
In this form, you will see more easily that why the elements of R matrix should be what they are and why R is an upper triangular matrix.
Thank you, i was really thinking how he arrived at R.
Thanks a lot!!!
Won't we get the transpose of R then? The elements will be q1ta1, q1ta2 and so on, right?
you are the savior of all mankind
Q^T isn't quite the inverse of Q. Q may not have an inverse (if it is not square)
instead think: A = QR (multiply both sides by Q^T from the left) -> Q^T * A = Q^T * Q * R -> Q^t A = R ( because Q^t Q = I by definition of Q)
From here you can see that R is just the dot product between q1, q2,q3 ... and a, b, c...
So, q2 dot a = 0 (because q2 is just b - the component of b in a's direction)
It amazes me how much a good teacher helps in grasping the subject. Thank you professor Strang
This is another fantastic lecture by the ring master of MIT mathematics DR. Gilbert Strang. GRAM-Schmidt is a classic topic in linear algebra. An individual student cannot have too much linear algebra.
pulling an all nighter studying linear algebra and I would have never done that if I hadn't stumbled across his lessons, hands down best teacher I had ever witnessed. I might fail in the exam but these lessons have helped me follow and understand much more than ever.
Αλέξανδρε το πέρασες τελικά?
@@theodorei.4278 Δυστυχώς οχι,το ξαναδεινω φέτος ομως σε συγκριση με τον καθηγητή του πανεπιστημίου έμαθα πολυ περισσότερα απο αυτά τα βίντεο.
Edit: Για λιγο δεν περασα,0.25 που ηταν να γραψω 5 μοναδες παραπανω στο τελικο, ανεξάρτητα ομως με βοηθησε πολυ
@@azrael6882 Σπουδάζεις μαθηματικά, φυσική, μηχανική ή επιστήμη υπολογιστών;
@@kkounal974 επιστήμη υπολογιστών
its been a while... did you end up passing your final??
Thank you professor Strang for all these lectures! I've remembered why I used to love math!
Professor Gil Strang has spoilt our learning process, so much intuition is mesmerizing. I started LA for the first time to learn on my own, and if I know this subject then all credits to him and MIT. ❤️
I don't have much time left for my final exams, but here I am watching all the lectures of the playlist.
Dr. Gil strang is a legend.
thanks OCW :) for this wonderful series !!
Conclusion: A set of orthonormal vectors combine into a matrix called Q. Q'Q always get an identity matrix. If Q is square, it's an orthogonal matrix, and Q'=Q^(-1). If we use Qx=b instead of Ax=b to solve the projection problem, it will be much easier. Because the projection matrix P=Q(Q'Q)^(-1)Q'=QQ'.
For the A'Ax=A'b problem, if we use Q instead of 'A' we can simplify it as x hat equal to Q'b.
Gram-Schmidt method: A way to transform independent column vectors into orthonormal basis.
A=QR => Q^(-1)A=Q(-1)QR => Q'A=R
wow, this professor just get rid of all my confusions about the formula of Gram-Schmidt
his chalk on the blackboard sounds are just so satisfying 😍 why doesn't all chalk sound like this??
calm down girl hes not interested 😂
For the QR decomposition ( 47:00 ) you can use Q^t * Q = I. With this you get:
R = q1^t * a1 q1^t * a2
q2^t * a1 q2^t * a2
but since we work in real numbers the internal product
A^t. B = B^t. A
Using it, we are left with:
R = a1^t * q1 a2^t * q1
a1^t * q2 a2^t * q2
amazing lecture. Thank you Professor Strang, Thank You MIT!
2020 still there, following every tape and note so far,
2023 still here😆
00:00 Orthonormal basis and matrices have many advantages in numerical linear algebra
06:42 Orthogonal matrices have orthonormal columns and their transpose is their inverse.
13:10 Orthogonal matrices have orthonormal columns.
19:21 Projection matrix onto orthonormal basis is Q Q transpose
25:25 Graham-Schmidt algorithm produces orthonormal vectors.
31:12 Finding the perpendicular vector using Graham's formula
37:07 Graham-Schmidt process for orthonormal basis
42:44 Graham-Schmidt produces orthonormal columns and a triangular connection matrix.
I like your style man! I wish I had those blackboard oh man.. Thanks for the passion you put in
27:00 *"Gram had, like, one idea. I, I, ...I don't know WHAT Schmidt did"* 😂
Two seconds later: *"Meh, we don't need either of them actually."* 😂😂😂
I read the comments in advance, then waited patiently for that roast 🦃🍗. Not disappointed 😂👽
Only to finish him of "maybe thats what Schmidt did - he, brilliant Schmidt - thought okay, divide by the length. Okay, that - is Smiths contribution :') Absolute favorite part.
@@jan-willemgmeligmeyling8393 Schmidt was a Nazi sympathizer.
fav lecture so far in the series
Completeing all lectures before 1 day of exam...
And it worth it..💯💯💯
breathtaking
what an amazing lecture, I already finished college, but I still need some of these classes. Thanks for put this on youtube!
All these videos previously had a delay between the audio and video. These are reuploads, and that's why there's so few views
great as always , half way there motivated to learn more, that is why i love your teaching
Selfeducation with such good teachers is the future!
Thank you Professor Strang.
University was always 95% self-education. That's not really what the professors are there for.
"hm I don't know if I've done this too brilliantly"
I also say that after my algebra exams !
Now I get to know that why the children's of MIT are genious. This is just because of such a brilliant professors knowledge delivery. Best lecture ever had in maths.
got my degree from there but i disagree. you have plenty of great mathematicians in other universities too
11:20 the matrix is not a rotation of any angle, it is actually a reflection.
(You can see that from the fact that det(Q)=-1)
you are supposed not to know the determinants
@Avi Aaron if you are refering to the Q from the top it is indeed a rotation
It's both. The cool thing about math is that you can think about things in different ways.
I am grateful for your lectures and your teaching.. "oh, it's no big deal, maybe that's what' Schmidt did". :) :)
Watching this in 2021 and gasping when Prof Strang coughs "oh no, he better not have Covid"
Thank you Professor Strang and MIT.
The classes are wonderful, excuse me, what linear algebra book do you work with?
The readings are assigned in: Strang, Gilbert. Introduction to Linear Algebra. 4th ed. Wellesley-Cambridge Press, 2009. ISBN: 9780980232714. See the course on MIT OpenCourseWare for more info at: ocw.mit.edu/18-06S05. Best wishes on your studies!
Instead of writing "Eg" he writes "Example"
.
DEDICATION LEVEL
35:00 This is like a typical home work quiz problem.
somethings wrong in the last A=QR? should the transpose on the q_1 q_2 not on the a_1 a_2??????
Thank you for saving 10 points
i am like schmdit in my group projects.
Five minutes in and my mind is blown
19:28 since the fact that Q transpose times Q equals identity matrix is true for all matrix with orthonomal columns, doesnt Q times Q transpose also equals I ? the P= I always holds true for matrix with orthonomal columns?
Can someone explain the difference between two approaches of solving the least square problem 1) using Transpose(A)*A*x = Transpose(A) * b, or 2) Using QR decomposition, i.e. R * A = Transpose(Q) * b
he ask ques to audience and answer himself before they gave that and it is interesting point in his lecture . it seem like he teaches himself not the audience .
My neurons are enjoying...
thank a lot
Sir....could we have any solution book of your book linear algebra and its applications fourth addition.
Around 33:00, for the right-hand side of 'B', how did we obtain A'b/A'A * A?
The previous lesson, we obtained 'p' as aa'/a'a * b. How did the aa' *b in the numerator get switched to A'b * A?
I also have the same query??
He place it wrong
actually, since the A here is treated like a n dimentional vector, i.e. a nX1 matrix, where the properties of vectors and matrix algebra is the same.
Since A' just a row, A'b a scalar, the switch is fine
At 15:34. Did he make a mistake here? The two normalized vectors 1/3(1, 2, 2) and 1/3(-2, -1, 2) aren't orthonormal because their inner product is not zero. It is 1/3(1*-2 + 2*-1 + 2*2) = 4/3.
what's the intuition behind 'Ax = b has no solution, however Transpose(A) Ax = Transpose(A) b has a solution'.
I am having a hard time
Multiplying by transpose A is the idea that was used in projection when vector b is not in the column space of A.
Also, Transpose A * A makes it a square matrix which has unique solution or infinitely many solutions.
A further elaboration of 강병우
's comment. Ax = b has no solution when b is not in the column space of A, meaning no non-zero combination of columns of A gives b. The best solution \hat{x} expresses the projection of b onto the column space, p = Pb in the column space, so that p = A \hat{x}. This means b is actually decomposed into p and the error vector e in the subspace that is perpendicular to the column space of A (left nullspace), i.e., b = e +p. This error vector has been dropped, but its length is minimized. 'Best' is in this sense. As the error vector is perpendicular to the column space, A'e=0-->A'(b-A\hat{x})=0, thus we have A'Ax=A'b. Recall the projection matrix P = A (A'A)^-1A', which has an inverse when A has independent columns. Alternatively, multiplying A' to both sides of A\hat{x} = Pb, we also get A'Ax=A'b. To solve this new equation, just use the inverse of ( A'A), so that \hat{x} = (A'A)^-1A'b.
@@angfeng9601 that looks like some serious LaTeX lover!
When A has independent columns AtA is invertible , actually the nullspace of A is identically the same as of AtA
guys at the end of the video, why isn't b(transpose) * q1 = 0, like a(transpose)*q2 is 0 ?
Let's make Matrix Orthonormal Again!
Poor Schmidt. He didn't deserve the shade
Oh, he did.
Hi, does anybody know the name of the book hes always referring to? :)
2nd half: 25:06
@33:10 , is he dividing by A^TA , because A is currently having just one column, and hence A^TA will be a scalar value? Otherwise, it should be
(A^TA) inverse, right?
I think the secret to this is that we're finding parts of triangles.
I guess we could also take the 3rd vector by the cross product of the 2 earlier ones.
Good point. Thanks
But in higher diminsions, how do you gaurantee that the cross product gives the vector that is in the space spanned by the 3 vectors that we started with (a,b,c).
Gram-Schmidt Algo starts from 25:30
In finding the perpendicular vector why the subtraction why not project (dot) then cross?
How would that work? Why project first?
DID HE JUST DIS MR. SCHMIDT👀😂😂
what is the reason behind R being upper triangular?
We know that we choose 'A' (the first ORTHOGONAL vector) as 'a1'. Later, we see that 'q1' is just 'a1/||a1||'. This means that 'q1' is the unit vector of a1. Now, in the equation, the first column of the LHS is 'a1'. Similarly in the RHS, the first column in the first matrix is 'q1'. Since q1 and a1 are the just same vectors with different directions, all you need is q1 to be scaled by a constant to become a1. This means that the q2 vector is not required. Not required naturally means the coefficient is 0 (remember that a1 and q2 are orthogonal vectors and their dot product gives zero ). Since the second matrix in the RHS is a 2x2 matrix, the cell at row =2 and column = 1 is 0, the matrix becomes upper triangular. That's all.
What my lecturer took 2 hours,you sir took 50 mins.
Why the position of A is changed in the projection (B = b - A(A^T.b)/A^TA ?) ?
It doesn't matter because A is a vector so (A^T b)/(A^T A) is a constant
Placement of A does not matter in case where A is a vector, but does matter in case where A is an nxn matrix. In latter case, projection matrix is P = A(A^T A)^-1 A^T, where projection matrix P acts on some input (vector b, for example). Full illustration of P acting on vector b is: A(A^T A)^-1 A^T b
Thanks!
why is Q times Q transpose not equal to the identity if the matrix is not square ?
As he had showed initially Q transpose times Q = Identity and if Q is square it will have inverse (Since all columns are independent). So we can show Q inverse = Q transpose. Now A inverse times A = A times A inverse = Identity If A is an invertible matrix. Using this we can show Q times Q transpose = Q tranpose times Q = Identity. When Q is not square then we don't have an inverse. So this is not possible and we are left with Q times Q transpose. Correct me if I am wrong!
@@helikthacker413 It seems right in this formula way, but when I wanna explain it in multiplication way, sth is wrong there... When Q times Q^T we got the (1,1) item in the result from the row1 in Q times col 1 in Q^T, it desnt get 1.... Hope you could help with this puzzle. Thx
@@dianel.9238 What Q are you talking about? A square one or a non-square one. In square taking a few examples you will get Identity matrix. But in non-square it is expected that it will not be an identity matrix and so you wont get a 1 in diagonal. This is my understanding, please correct me if I am wrong.
@@helikthacker413 It is the columns in Q are orthogonal ,right? not the rows. In this case, the (1,1) item in Q*Q^T is the result of row1 in Q times col1 in Q^T ,ie row1 in Q... how come that gives 1 (in this case square or not doent seem matter)?
@@dianel.9238 czcams.com/video/0MtwqhIwdrI/video.html This shows if Q is square, Q^T = Q^-1 since Q*Q^T = Identity. So after that it is clear that Q^-1 can be left multiplied or right multiplied to Q to get Identity. I don't know the proof but taking any Q and right multiplying by Q^T gives Identity.
Maybe we couldn’t hear the audience. I hope his joke that Schmidt didn’t have much to do got a laugh.
Really nice lecture!
P = A.x(hat) ,right ?
so at 32:10 why is he substituting P as x(hat).A.
Please help me with the intuition behind that.
From the lecture on projection: for projection in two-dimensional space, you use the following formula: A * A^t / (A^t * A) * b, which is the projection matrix times the projected vector b: p = P * b. You can rewatch the video on how he derived it. Basically, you find out what is x(hat), substitute it in P = A * x(hat), and you'll get P = A * A^t / (A^t * A) for the two-dimensional space.
@@TheSalosful not quite
43:30 How can column space of Q be same as column space of A?? I know that their dimensions are same, but Q is composed with orthogonal vectos and A is not. I need some helps guys {{{(>_
@CoeusQuantitative SO If I suppose a,b are basis of matrix A, then the error part, e, is a linear combination of a and b. that's why column space of Q is same as column space of A. Is this what are you trying to say?
@CoeusQuantitative ok, wait, I am figuring it out.....
anyway I am really appreciate your replies. Live long and Prosper.😄👍🙏
@CoeusQuantitative what a great picture!! I finally fully understand the picture. The problem was I was confused vector 'b' becasue of merely overused terminology........ In the previous projection and least sqare chapter, vector 'b' was out of C(A) and we tried to project vector 'b' onto C(A). That is Ax=b, the right side.
However, in orthonomal and Gram process(pf.Gilbert doesn't like SChmidt so...😅) chapter, vector 'b' is just one of the basis of matrix A. That's why I was so confused 😥. My thought was the error vector 'e' should be in the left null space, and how can vector 'e' be in C(A)?? Just using a1, a2, a3 would be much easier to understand....I figured this out by solving some questions in the text book. And I watched your video, that was awesome. you made it for me😉. I want to say thx again.👍
the vector 'b' in orthonormal bases chapter is the one which is in the right side of eqation. It's not the basis. Be careful guys.
Why does he do 1/root2 at 11:00?
to make column vector orthonormal.
Because he wants it to be an orthonormal matrix. Which means all columns are PERPENDICULAR to each other and of UNIT LENGTH.He shapes his matrix in such a form that the 2 columns , when dotted with each other ,return 0 , which is what we would expect of perpendicular vectors.However They do not have unit length. The length of each column vector would be sqrt(1²+1²) = sqrt(2). So if we wanna "normalize" the vector , meaning turn it into a unit length vector , then we just divide by the length. Since everything iin the matrix is divided by the same number , the dot product remains 0 (you can just factor the divisor out),and now the column vectors have length 1
@@DeadPool-jt1ci thank you so much! This is helpful!
Unit vectors | Matrix transformations | Linear Algebra | Khan Academy
czcams.com/video/lQn7fksaDq0/video.html
Why did we divide by square root of 2 at 42:30?
Converting to unit vectors(it’s already 4months I hope u get it already lol
To normalize
47:34 A = Q*(Q^T*A) and obviously Q^T*A is an upper triangular matrix.
It is said A^T*Q=R, but it should be Q^T*A=R, am I wrong, is that what you are saying?
85 years old not a single white strand of hair in sight.
*65
25:32
justice for schmidt
👍👍
This man hates Schmidt. XD
w lecture
What did Schmidt do actually?
kookoo transpose
I think Prof. Strang really hates Mr. Schmidt ;)
Dislikes(if they exist) came from Schmidt fans
I think im just to stupid for this .......I just dont get it .......
what dont you get? a lot of this is computational.