Data Analysis 9: Data Regression - Computerphile
Vložit
- čas přidán 12. 09. 2024
- Real life doesn't fit into neat categories - Dr Mike Pound on some different ways to regress your data. This is part 9 of the Data Analysis Learning Playlist: • Data Analysis with Dr ...
This Learning Playlist was designed by Dr Mercedes Torres-Torres & Dr Michael Pound of the University of Nottingham Computer Science Department. Find out more about Computer Science at Nottingham here: bit.ly/2IqwtNg
This series was made possible by sponsorship from by Google.
/ computerphile
/ computer_phile
This video was filmed and edited by Sean Riley.
Computer Science at the University of Nottingham: bit.ly/nottsco...
Computerphile is a sister project to Brady Haran's Numberphile. More at www.bradyharan.com
Check out the full Data Analysis Learning Playlist: czcams.com/play/PLzH6n4zXuckpfMu_4Ff8E7Z1behQks5ba.html
I'm ckecking Full list...
Do You have data of it?
#اطمن_انت_مش_لوحدك
#ducks
@@gxsoft not funny
Wow,I admire your work,thanks for sharing
I binge watched the whole series and I would like to take of my hat off! Thank you for putting together this concise, well thought-out course that has helped me to practice and deepen my knowledge in the area. What a world we live in where we not only have these powerful technologies at our disposal, but where we can also get to know them so easily.
I am a programmer for years and recently got into data sciences for work related stuff. I read a few books and articles and took a course on the topic. But i have to say: You condensed the knowledge so effectively and presented it so clearly and easy to understand that i probably learned more from this playlist, than from all the other resources. Thank you very much.
Agreed! Well said
Dr. Mike performed a data analysis on the topic "data analysis" by cleaning and reducing data, clustering and visualising the topics so we could gain all the knowledge from data available.
Agree, best lecturer on CZcams, I watch a lot of them just for fun.
This has been some of the best computerphile content ever. Thank you very much for this! I‘m sure i‘ll go back and watch them many more times.
Much appreciated, and very well made! This kind of subject area overview crash course with concrete examples must have taken some real effort behind the scenes, but be aware that it shows! I, for one, would definitely welcome similar courses in other topics in the future!
What I didn't like was the number of episodes. I want MOAR
Exactly!
I love the format and how the content was presented. Examples were realistic, modern and relevant. Having the content organized in a coherent progressive series makes the understanding of the topics easier and more friendly for the beginner and more accesible for the advanced to skip or review concepts already understood.
Learned many things from your series, including that y=mx+c is used in the UK while y=mx+b is used in the US.
while we in europe go with the y=kx+n
I had no idea that anything other than y=mx+c was used.
y=kx+m
in the US I've always seen it presented as y = mx + b but y = mx + c makes much more sense considering c as the conventional constant of integration
In Poland, we have y = ax + b
OK this short series has been invaluable to me as someone trying to understand and then apply machine learning to a problem. The work up to practical applications without dumbing it down has been really helpful to me. A really valuable overview.
Please more of this! Maybe deeper dive into some Network designs
Yes! The brief into to neural networks was great, would be great to see it implemented and an example of it in practice
Regression is so much more than just a predictive tool. It's a powerful hypothesis testing framework as well. Would love some content along those lines one of these days.
More deep dives like this please. This was an excellent introduction to the techniques used in data analysis
I really enjoyed this series - splitting it up into separate episodes in order to focus on different topics makes a lot of sense. I'd definitely be interested in seeing more series of this kind of format.
if there was a like button for a playlist, i would've used it for this series! really accessible and there is something to clear misconceptions of those who have learnt to do data analysis through web tutorials. more of this please
We want more Mike Pound series. Thanks a million for this one.
Would be nice more series from Mike Pound.....maybe on RL or DL
As a "mature" student, I found this series amazingly valuable. Your enthusiasm for the subject shines through and is the sign of a great teacher. Thanks!
This series is by far the best introduction to data science and ML I've ever seen. Can't thank you enough gentlemen.
Thank you for the great course. I feel fortunate to have this well desinged knowledge curve at my disposal for free. Hats off to Dr. Mike and the course creation team and big thank you to the best channel ever computerphile.
What an amazing introduction. You make this all sound so simple, which is a really helpful place to start!
Loved these videos, some of my favorite videos from numberphile. Watched the whole series in 2 sittings
Your videos helped me to pick up R. Before this I did transformations of excels provided to me in Power BI (newbie data analyst here). Well it works, but.. It's not pretty. And sometimes painfully slow.
While going through these series, I learned how powerful R is, and the concepts You tell about - I didn't even know about anything of that.
These series are invaluable for those, who pick up data analysis with no background education - just learning on their own.
Now I implement R scrips in my own solutions and clients are happy :) Thank You for the outstanding work. Very, very highly appreciated.
Hands down the best data analysis video content i had watched
Pretty cool series, covers quickly some of the most important elements of data analysis. How about a dedicated machine learning series?
As someone who is currently completing a pH.D. in AI for cyber security, you did a good job explaining the transition from linear to non-linear functions. Most people probably will still not get it, but at least some people can visualize the non-linear hyperplane that becomes the decision boundary.
In what ways might machine learning be used in artificial intelligence?
@@Jupiter__001_ It's used to teach the A.I. whatever you want it to do. That A.I. is a "neural" net, machine learning is setting the weights in the network so it produces an output you want.
@@myothersoul1953 Whoops, that was not the question I intended to ask. I must have been half-asleep. I meant to ask "In what ways might machine learning and artificial intelligence be used in cyber security?"
I watched this whole series in one morning (on 2x speed, Dr. Pound, your voice is incredible when it's sped up!). I've taken some university level statistics and computer science courses, currently taking an introductory linear algebra course. It was fascinating to see how these seemingly disparate disciplines come together to form data science. I am so excited by this content and I can't wait to learn more about machine learning! As usual, thank you for your charismatic and approachable teaching style, and I look forward to many more videos from you in the future!
In my stats 2 class we used the iris dataset too so it was very funny to see it pop up again in a completely different context
Love this series! I wonder, will you cover ANOVA (analysis of variance)?
Many papers use this kind of statistical analysis to infer features of importance. But I never wrapped my head around it.
Thanks for the wonderful content.
I really love this series. You managed to write something so concise but useful. I was already learning R and it made that much more enjoyable. I would love a few recommendations for next steps. What books etc?
I just wanted to say thank you very much for this great series!! I am starting at a new students job tomorrow where I will be working in data analysis, which is a completely new field for me coming from mechanical engineering. But thanks to your great series and some other practical programming examples I did I feel more comfortable in starting at that new position and I am looking forward to implementing my newly learned knowledge in a practical field (I will be working with wind turbine data, mostly offshore, to predict malfunctions and so on!)
Wow this brought me straight back to not understanding Linear Regression at all in my Economic Statistics classes and making it in Excel to wrap my head around the Y's and Hat Y's, R's, Errors and even implementing an accuracy value that changed the absolute prediction of Y into a range of Y-probable error ~ Y+probable error. This video made a lot more sense than my teacher all those years ago.
I loved the series! Thank you so much dr. mike and computerphile!
Amazing series! Now please make another one just on predictive modeling!
Machine learning and Data Science videos are so relevant these days. This playlist was amazing! Thanks for the incredibly well made content. Perfect timing aswell, as I'm studying this right now :p
I loved the series. It is such a great introduction with some nice examples. Excellent job Computerphile and Dr. Pound
Guy I can’t thank you enough for this series. Please keep making them!
Favourite series on Computerphile! I would love to see more.
Thanks for the serie, loved watching it!
Where was this video when I was taking data analytics in University?! Great stuff :)
This is truly excellent. Thank you for this amazing playlist.
This was amazing! Thank you for putting this playlist together and offering the datasets. I will be revisiting the series again and again.
Hi Sean and Mike! Awesome series!! I really enjoyed he deep dive into this topic, showing practical applications (using code and data sets, and using programs and methods that are really being used today). Single videos often are great for introducing topics, but for real world discussions, it often requires longer series like this where the ideas are built up with the assumption we have seen the previous episodes (and have that understanding) so more abstract ideas can be discussed without repeating too much.
Take away points:
- awesome work in this series. I really learned a lot and honestly believe this will help me use matlab/r/etc. in my work in more ways that I could previously.
- the deep dive series format is awesome and should be continued on other topics. (Cryptography, data structures, programming methods...)
- showing actual code running is always great to see. Drawings on paper are great at conveying ideas, but code solidified that it is really usable.
- thanks as always for the great work!!
Absolutely brilliant. Both interesting and useful, I'll be coming back to the videos again and again. I would love to see more too.
Loved this series. I would love more videos deepening on each of these topics. What I mean is that I want more of everything.
This was a fantastic series, was tough to get a grasp on the theory applied on example datasets but was worth the effort
I heard someone who worked with deep/machine learning and AI that she thought a better term for all three (especially DL/ML) would be 'applied statistics'.
But that doesn't sound sexy, so that wouldn't get published.
Absolutely loved this series! Great primer on data analysis and data science, very eager to dig deeper on some of these topics
thank you for your service dr. mike pound!!!!
Simplest explanation of neural network out there
This is the most didactic content I've ever seen on the channel. Great stuff!
thanks guys, cool stuff. great to find out the "boring" math ;-) behind the meaningless buzzwords.
might be cool to have some challenges or exercises for practicing the techniques. I realize at that point we're verging into MOOC territory, but nonetheless I always find trying the techniques out for myself will 1) assist retention, 2) deepen understanding by challenging your weak points, and 3) prompt new questions.
thanks for the series, was great! Personally I would love to see more on PCA and how to use the result to predict values (since the values are in a complex plane, I've found it hard to see what we can actually predict with PCA)
What a great series!
A great series! You managed to really describe the data analysis process very well without overloading on details. I will definitely recommend this as an intro to my students next semester.
A follow up series that goes into more detail on neural networks would be amazing.
Thank you for this series!
Loved the series!
You will also find Root Mean Square Error (RMSE) in Global Navigation Satellite Systems (GNSS) or GPS. The better the RMSE rating the better the accuracy and usually also correlates to a higher price.
Oh No it's the 9th, now I gotta watch the whole series
These most recent episodes look so nice! Great work.
Excellent series, thank you! please do one that explain the pros and cons of different machine learning algorithms and how to compare the performance
Critical temperature in kelvin is a "ratio quantity", as per the Noir classification from video 0 of the series, meaning absolute zero is well defined. Furthermore, all temperatures are positive and span several orders of magnitude (from tens of milikelvin to tens of kelvin). Thus, it would make sense to use the logarithm as the fitted variable. Similarly, for all positive definite ratio quantities in the dataset, take the logarithm and only then perform a linear regression. I would be very much interested to see the results of that regression.
I like this series so much! I watched it all and it was incredible!
As always, Dr Pound is awesome.
Awesome series! Thank you very much!
Thank you Dr Mike Pound. Awesome series!
Thank you very much. This is very helpful for scientific research
Very good series, congratulations. Hopefully we will see more like it
Great series, thank you very very much! BTW, the first time I heard about Data mining was near the release of sql server 2005, that came with data mining tools on their analysis services. The term may be older than what you expect.
Freaking awesome. It just clicked now to me that MSE is just a correlation of the predicted outcome to the expected outcome.
So Tom Scott made a series about How to Build an App sponsored by Google, and Computerphile made one about Data Analysis. Also sponsored by Google. Both were fantastic. Are there any more? Is there a list of all the channels that made a sponsored series?
Thanks for this playlist! Has been very interesting.
Keep making this idea come trought. Great videos! Go for some real usage project lessions
This was a great series. Very useful
How would you perform regression with a NN on a two dimensional function?
What about Bayesian techniques? Can you cover them?
Yeah, give us some Maximum Entropy, doc.
Thanks for the great series!
In Data Regression using a neural network, is it generally better to use principal components of a set of attributes as an input to the network, rather than the attributes themselves? What are the scenarios when this is not the case? thank you, N
Great series!
This series is great for people who have a little bit of background in programming and/or statistics, but I think some people would be put off by all the terminology used.
Yes, much has already been excluded to keep things simple and easy to approach, but there are still some instances where Dr Pound just throws a new term at the viewer which sounds complicated and scary.
I guess it just depends on who the target audience is - I think a few small changes would make it more approachable to even high school students, but that might take away from the experience for others.
Hi Dr. Mike,
Your data analysis series is really good, but I've got one question, suppose if my R2 is 0.3 and my RMSE is very low, should I consider this model to be good, please let me know.
great series!
Excellent. I had never played with R. This is no longer the case 🖖🏻
make more of these series!!
Love the video, how can we analize data to guess what is good/bad based on the potential outcomes of a decision being taken. I'm trying to figure out if "good for me at the same time as everyone else" can be coded. The Internet is good, because it covers a basic human need such as communication, it's the people who use it in a wrong way, what make it look bad sometimes.
sir you're best
wait.. how do u turn "he does/doesn't own a car" into a numerical data to get the weighted sum????? do u just use 0 & 1?
This bro is classy, great interpretation
When I see this guy, I Pound the like button!
mrs major 2.0 is best comuter course ever
In a world drowning in numbers... he is our last hope for making sense of it... starring Mike Pound as himself... *The Data Doctor*. Coming this October, plus or minus a month...
God bless you!
VERY GOOD CONTENT
is it possible to use levenberg-marquardt vs gradient decent regression?
Could anyone explain me how I adjust the weights of a neuronal network with 3 inputs 2 outputs and the desired output is also known. But how I have to calculate weights in the hidden layer.
Check out the series Neural Networks Demystified from Welch Labs. Episodes 3 and 4 should answer your question but I definitely recommend the whole series.
Quick and easy answer is we use calculus to determine the direction each weight should move in order to make the actual output move closer to the desired output. In practice, use something like Tensorflow and it does all this for you. It is still useful to understand though. Hope this helps :D
Watch 3b1b series of 4 videos on Neural Networks
Move over Passive Aggression, we now have Linear Aggression ;)
Thanks for the series
😂
Dr. Mike "Take Me To" Pound "Town"
Great Dr. Mike Pound
"Multivariate regression" is not synonymous with "multiple regression". The former requires multiple predicted variables while the latter requires multiple predictor variables.
These are great. riiiiiiight?
That's quite difficult to grasp for me: a neural network (in the computing sense, not biological) is a set of linear equations? I must have missed something.
I just finished binge watching each of the nine episodes. Superbly done. The episodes 1 through 8 shook some latent knowledge of statistics out of the dank recesses in my brain. But you lost me on episode 9. What are those hidden attributes between the input attributes and the output? Is that a representation or other perspective on the decision tree you discussed in a previous episode?
The hidden attributes usually don't have any intrinsic meaning. Rather than calling them attributes, perhaps something like "temporary outputs" might be more appropriate.
An interesting point to mention, is that although you don't assign any explicit meaning to these hidden outputs, it is surprisingly common to see the network **naturally** converge in such a way, that these non-constrained intermediate outputs actually seem to strongly correlate with some high level attributes (not included in the input data) that a human might think about, when answering the questions, that the network is trained for.
For example, a convolutional neural network might have hidden layer outputs (also called features), that strongly correlate with presence of shapes and textures, that might be related to the important objects in the image.
@@ruroruro thanks for your response. I watched the episode again, I'm still stuck. I'm wondering how you'd label the hidden neurons? In order to put a linear relation between a node of the hidden neurons and the input attributes they must have been derived from the original data. That's my thought. The input attributes may be a result of the PCA, what data are represented in the hidden neurons.
@@ruroruro It took a while. I hope I got it now. Using Dr Pound's example with the input attributes a, b, and c, the hidden neurons y1,...y4 I get these equations
y1=w11a+w12b+w13c
y2=w21a+w22b+w23c
y3=w31a+w32b+w33c
y4=w41a+w42b+w43c
Let the output node be r, w1,...w4 be the weights applied to the hidden neurons. Then you could use two equations, either
r=w1y1+w2y2+w3y3+w4y4
or
r=w1y1*w2y2*w3y3*w4y4.
Ok, I understand the hidden neurons now as a set of possible outcomes, depending on the weights applied to the input attributes. So they are not fixated on a result. The result is determined by the weights applied to the possible outcomes y1,...y4.
RuRo, thanks for your patience. Have a nice day.
@@isabellabihy8631 Not exactly. Mike has covered this briefly but probably it was a bit confusing. Each neuron in the network has the activation function which transforms a linear sum of inputs into an output. If you just make activation function linear, there is indeed no point in having more than one layer in the network. But generally, what a neuron outputs (and feeds to the neurons in the next layer) overgoes a transformation using some kind of a sigmoid function. There's a bunch of those to choose from, Mike has mentioned tanh; that along with softplus and ReLU are probably the most popular ones used now. The activation function is the very thing making multilayer networks making sense. Each neuron has it and it is introducing nonlinearity.
@@Lodinn Thank you for the response. Yes, it would be too simple with just linear equations. Then more questions boil up. But let's just stop it here.
My wish is: Dr Pound, please do a more detailed look at neural networks, covering also how come up with the activation equations. That would be fantastic.
Liked the series