Video

Fantastic video, well done! I'm watching for path tracing rather than ML :)

The cost of housing grows faster than the busy beaver.

Random projections enthusiast here, had some expectation with the video until the definition: "Linear Algebra is the "mathematics" of vectors and matrices"... hmmmm... but one can do linear algebra with polynomials, and "derivative" operators, I don't know... that sounds far beyond an oversimplification.

I wonder what the largest computable, but not stupid function would be, like not just doing the power to tetration to pentation to hexation to so on or something like that.

Now the true question is: how can one be clearer than that? Wonderful work, thank you so much

Actually, randomizing algorithms is very common and little to do with linear algebra specifically. The reason why it works depends on the algorithm, it's a bit like magic honestly, not well understood at all. But more often then not it has to do with choice. To solve many algorithms, you have to explore certain paths. This can be recursive, that is, once you start exploring a choice path, you then have to explore more choices. The choice function matters, some functions work better then others, but computing statistics to make the best choice is expensive. One option is to choose at random instead. The reason this sometimes works is because making one "bad" choice based on the statistic you know will likely mean making more "bad" choices with the same statistics in many situations. Thus, by choosing randomly, you tend to settle for an "average". This can be applied to other choice functions. Say you want to compute some statistics to make a choice function, but computing them every branch is expensive so you want to do it at intervals only, but how many intervals is good? One way is to ransomizs the intervals. Again it's the same thing, you don't know the best interval so you settle for an "average".

These are some busy ass beavers

You're already doing least squares - so you're already accepting you're going to an have an approximate answer. If the randomized solution adds little to the error but is significantly faster, that's great.

I don’t get the part of the 27-state machine. Let’s imagine that I create one 27-state machine and I define such as the state to transition into is Halt for all state. Then it will Halt at first step. Have I just proven that Goldbach conjecture is false? I know I haven’t, so could anyone be so kind to clarify what he meant by « there exists »?

Linear Algebra doesn't really deserve to be attacked like this. What did he do to you?

It looks like sketch-and-solve is an automated version of dimensionality reduction, akin to principal component analysis?

So, analog computers are back at the game?

If you actually wrote this out as python code, rather than a bunch of fancy looking math, it's not that impressive

You brought up a good point though. Maybe the current limitation is not the speed of compute; rather, it is the speed of data transfer.

Where is the paper?

1:32 Could you make a video that goes through these 10 algorithms? That'd be awesome!

max cringe detected

great stuff

The Theory of Computation has some of the most beautiful and surprising ideas.

I will love you do a video applying this in a code

man j really wish these kinda of videos existed when I was in school. I would have reached my math potential instead of getting bored and losing interest because my teachers didn’t know how to teach.

Parts of this borrow too heavily from Tony Padilla's exploration of Graham's Number

How would this function do agains Rayos(N)?

Man, I hate that this has nothing to do with neurons, or anything biologically inspired. great explanation to see what is really going on. but this has nothing to do with intelligence.

What a terrific video and channel. Great work! Subbed.

To be fair, back in Kelly's time, there was significant misinformation on the dangers of smoking. It didnt really become common knowledge of how bad it is for us until decades after he passed away.

I came here to learn something, and after watching the video, I learned that I should've paid more attention to the maths because I couldn't understand it. 🙃

I don't understand how Fortran is still relevant. Is it?!? Is cuBLAS used by any Fortran compiler?!?

The matrix shapes you’re discussing here are common in machine learning but uncommon in the scientific computing programs that you started the video with. We do almost no dense computation in modern scientific computing. For instance the finite element algorithm you rendered is dominated by sparse krylov solves.

Fortune’s Formula…great book on the subject

This. Is. Beautiful. Intuition 100.

I can compute all these numbers for humanity if we made me immortal. We should do that

Math? = Meth Amazing and engaging description, thank you!

I subscribed, but if you fix the backgrounds on your monitors to actually align continuously, I'll share your channel with a few friends. As it stands, I have to keep this between us, I can't be associated with this kind of thing.

Amazing!!

This is really interesting - I spend all day solving linear algebra problems for my work (quantum physics research), and this involves matrices the scale exponentially with the size of the problem. Accuracy does matter - in what I’m working on now, machine precision is a genuinely annoying limit I’m butting up against! - but a controlled approximation that brings down the cost of these problems by an entire complexity class would be incredible. However, we’re almost always working with square matrices, not tall and skinny ones, but if this is as big a game changer as this video presents, that hurdle should be overcome. We also frequently employ low-rank approximations so that’s not a problem if some of these techniques only work on those. I’d be most interested to see how this can help solve eigenvalue problems - that’s a special case of SVD, and also I believe finding the QR decomposition essentially solves it too.

I didn't understand a lot of things said in this video, but I know one thing for sure: I want to see games that use this with a relatively low accuracy. I just want to see what that looks like! (I'm guessing geometry would jiggle)

Great video. There is a lot of utility for this, as well as prior art related to it, dating back over 30 years ago. Good stuff. BTW, linear does not mean what you asserted, and actually only limit cases are objects like straight lines, flat planes and such that you asserted.

I wonder if we are just now discovering what intuition is from a computational point of view. From the point of view of how the brain works, this is really fascinating stuff.

So by how much must you shrink them?

I realize that YT comments are not the best place to explain "complex" ideas, but here it goes anyway: The head bending relative difference piece reply is "just" a coordinate transformation. At 29:45, you lay ellipses atop each other and show the absolute approximation difference between the full sample and the sketch. The "trick" is to realize that this happens in the common (base) coordinate system and that nothing stops you from changing this coordinate system. For example, you can (a) move the origin to the centroid of the sketch, (b) rotate so that X and Y align with the semi-axis of the sketch, and (c) scale (asymmetrically) so that the sketches semi-axis have length 1. What happens to the ellipsoid of the full sample in this "sketch space"? Two things happen when plotting in the new coordinate system: (1) the ellipsoid of the sketch becomes a circle around the origin (semi-axes are all 1) by construction. (2) the ellipsoid of the full sample becomes an "almost" circle depending on the quality of the approximation of the full sample by the sketch. As sample size increases, centroids converges, semi-axes start aligning, and (importantly) semi-axes get stretched/squashed until they reach length 1. Again, this is for the full sample - the sketch is already "a perfect circle by construction". In other words, as we increase the sample size of the sketch the full sample looks more and more like a unit circle in "sketch space". We can now quantify the quality of the approximation using the ratio of the full sample's semi-axis in "sketch space". If there are no relative errors (perfect approximation), these become the ratio of radii of a circle which is always 1. Any other number is due to (relative) approximation error, lower is better, and it can't be less than 1. The claim now is that, even for small samples, this ratio is already low enough for practical use, i.e., sketches with just 10 points already yield good results.

6:08 I'd like to point out that there's a "shortcut" Collatz fn, which is even more similar to the BB one!

Why can't you change the estimate of the probability as you get new data about the bet? More data equals better estimate. Continuus improvement.

Great video. Subscribed!

We can feel how much you need to control yourself to not ask " class, are you understanding?" 😂 Thanks to reminds me of Montecarlo tho

Great vid bruv

大概捋了一下整体思路，用很多浅显易懂的描述代替了很多复杂的数学公式，让我至少明白了他的原理。感谢！

Finally, after watching this 3 times I got the intuition of this method. Thank you for uploading a great series!

Jesus Christ 2:23 "What is linear algebra?" is akin to "At first there was nothing... then there was the big bang". Like come on, dont waste time and just drop a LA textbook in the description and get to the point.

This video is incredible. I'm really looking forward in watching other videos of yours