Cheap vs Expensive MacBook Machine Learning | M3 Max
Vložit
- čas přidán 2. 06. 2024
- The power and performance of the cheapest M1 MacBook Air up to the newest expensive option - M3 Max MacBook Pro, and a few machines in between. This is an in-depth comparison of their machine learning capabilities, from CPU and GPU performance to the Apple neural engine.
Run Windows on a Mac: prf.hn/click/camref:1100libNI (affiliate)
Use COUPON: ZISKIND10
🛒 Gear Links 🛒
* 🍏💥 New MacBook Air M1 Deal: amzn.to/3S59ID8
* 💻🔄 Renewed MacBook Air M1 Deal: amzn.to/45K1Gmk
* 🎧⚡ Great 40Gbps T4 enclosure: amzn.to/3JNwBGW
* 🛠️🚀 My nvme ssd: amzn.to/3YLEySo
* 📦🎮 My gear: www.amazon.com/shop/alexziskind
🎥 Related Videos 🎥
* 🤖 INSANE Machine Learning on Neural Engine - • INSANE Machine Learnin...
* 👨💻 M1 DESTROYS a RTX card for ML - • When M1 DESTROYS a RTX...
* 🌗 RAM torture test on Mac - • TRUTH about RAM vs SSD...
* 👨💻 M1 Max VS RTX3070 - • M1 Max VS RTX3070 (Ten...
* 💰 This is what spending more on a MacBook Pro gets you - • Spend MORE on a MacBoo...
* 🛠️ Developer productivity Playlist - • Developer Productivity
📚 Blog post: tlkh.dev/benchmarking-the-app...
(Looks like Timothy's domain expired, so you can use archive.org to view the site: web.archive.org/web/202309221...)
📙 Code CPU: github.com/alexziskind1/m1-cp...
📙 Code GPU/ANE: github.com/alexziskind1/tf-me...
💻 MacBook models in this video
M1 MacBook Air 8GB
M1 Max MacBook Pro 32GB (Timothy's)
M2 Max MacBook Pro 64GB
M3 Max MacBook Pro 64GB
- - - - - - - - -
❤️ SUBSCRIBE TO MY CZcams CHANNEL 📺
Click here to subscribe: www.youtube.com/@azisk?sub_co...
- - - - - - - - -
📱LET'S CONNECT ON SOCIAL MEDIA
ALEX ON TWITTER: / digitalix
- - - - - - - - -
#m3max #m2max #machinelearning - Věda a technologie
JOIN: youtube.com/@azisk/join
Most people cannot really understand how impressive it is to have 128GB of GPU memory at that price. On a laptop. Consuming 50W. From a deep learning perspective, even the most expensive M3 Max is a bargain compared to what you would need to spend to build an Nvidia based computer.
exactly
Both are expensive
I got this spec just to run LLMs. 128GB VRAM is insane with 400GB/s, and compute doesn't matter as much as memory bandwidth for LLMs.
Definitely, I appreciate it; and I use it. However, in case of AI the speed (in concrete the bandwidth between VRAM and GPU cores) is unfortunately far behind the state of the art, which leads to lame training performance and lame inference. My M2 Ultra is for sure great, but not ideal. It feels like a SuperNES emulator on an NES. Technically anyhow possible, but lagging.
Can you train on it though? Lol. For normies who simply just LLMs, sure. For actual researchers like myself, it sucks like your mom
I have a MacBook Pro with an M3 Max 128GB and a workstation equipped with an Nvidia A6000 48GB. When training the Stable Diffusion model, my MacBook Pro delivered performance that was, on average, about 50% of the A6000. Considering the workstation has a 2000-watt power supply, this certainly makes for an interesting performance comparison.
Hey. just a question, in your opinion what would be the optimum level for the chip and unified memory if i'm doing most of my deep learning with cloud gpus? new macbook pro or something else?
@@undrgrndrecords Sorry for the delay in checking your comment. In my case, I often go on business trips to places where the internet environment is not good, and I just used a high-spec Macbook so that I can work ML/DL tasks at such times. When I work in the office or at home, I work almost 100% on the cloud, so if most of your work is done on the cloud and you are not a special case like me, I think you will be satisfied with a Macbook air series with 16GB of unified memory and the latest chip (maybe M3 now?).
The A6000 48GB is rated to max 300W, what r u on about?
@@alien2327 thank you!
Thank you Alex really appreciate your attention to detail 👍
Hi Alex, I imagine that you put in a ton of work to make your videos because what you produce is excellent. Well thought out, informative, entertaining, and with humor. Thank you.
thank you 🙏
Great.
I really hope that the bandwidth will be significantly increased with the introduction of M6.
Thanks….! I’ve been looking into why all the gpus and ml cores, this really helps illuminate the various options and perfromance differences.
Video packed with good information, and supper fun to watch, your stuff is getting really good! Congrats on the video.
Glad you enjoyed it!
You should do comparisons on how the different Mac processors are able to run different huggingface llms. Tons of cool models to test with different sizes and complexity. From language models and text embedders to text to image generators. Would be a nice showcase of what modern Macs can do.
Thanks for sharing!
Thanks really helpful info. I am still learning about ML, but gave D Bourke's ML benchmark a try and added in some improvements. I will try Timothy's benchmark. Good to learn as I go, there are still lots of stuff learn.
Joined. Thanks for the quality content.
welcome 🙏
I just ordered the M3 Max 16" version with 128GB and 8TB SSD for deep learning and it's gonna be a powerhouse for what I do, I come from a 15" Base MBA and the main upgrades are the SSD, CPU and GPU power and the unified memory for me, with the Magic Keyboard and Trackpad and essentially a Pro Display XDR Built into the MBP, it's the best computer I can ever ask for
Great quality vid!
Thanks!
SVD is the abbreviation for something called the Singular Value Decomposition of a matrix. It's extensively used in data related tasks.
Thank you!!
SVD stands for a matrix's Singular Value Decomposition, it's basic/advanced linear algebra stuff, depending where you learnt it from :)
traumatic flashbacks from vector spaces class in 2nd semester : ((
Hey great vid. just a question, in your opinion what would be the optimum level for the chip and unified memory if i'm doing most of my deep learning with cloud gpus? new macbook pro or something else?
Very interesting! Have you tried Mixtral quantisation though? 4bit quantisation worked on my 32GB RAM + 12Gb vram just fine.
Super great channel!!!!!!!!! I met you because I'm waiting for my m3 max for artificial intelligence... and it's just the content I was looking for!!! It would be great if you could talk about the new apple framework, the mlx!!!
thanks! and congrats
I do not have the skills nor am I planing on acquiring them to do what you do, but I find your videos informative none the less as they show who will benefit from buying which version of the m series, and which like me will not benefit from an upgrade based on my current use.
would you recommend the 16in M2 or M3 MAX over the 14in? is the difference in battery life and performance /heat dissipation that different? thinking of switching from windows laptops to mac for machine learning and data science
What external monitor do you recommend us, except the Apple Display?
i thought i was missing something earlier but i think Mixtral 8x7b can actually be ran on your machine. it maxes out at 52gb of ram in the blokes q8. Correct me if I'm wrong.
thx for yoru videos
Hi, a quick question: if we have to perform non deep-learning tasks like parsing large data, may be plotting 3d plots, how can we use GPU/ANE?
The mac mini with 256gb of ssd and 8 of ram worth it to run docker and developing with spring boot?
Thanks for making this video... I think you considered my comment on your previous video... Thanks man... Thanks for the video...
I think the better solution for software developer is (MB 15 Air, 8gb RAM) + (mini itx, 96 or 128 RAM, 16 core AMD (32 thread), 1-4 TB ssd) setup. Then setup all dev environment (docker, VMs, compilers and etc) on mini-pc and use the cheapest 15 Air as just remote dev console with excellent screen for this powerful dev mini-pc.
And What is the GPU for Mini ITX?
From your experience, starting out in programming, would the 15” MacBook Air 16/512 ($1,699) suffice or recommend just to get the 14” M3 Pro 16/512 ($1,999)?
My only concern is does the CPU usage require fan to keep cool during Xcode builds?
Very interesting results on the ANE with CoreML models! Looks like the ANE smashes lower batch size inference.
Thanks Daniel. If anyone missed Daniel’s deep dive on the M3 series, check it out: czcams.com/video/cpYqED1q6ro/video.htmlsi=m_qAvCZuZHpEYp9n
i like this. Weak vs strong. Not the usual strong vs strong !
Am getting into Apple it will be my first time owning Apple Laptop ..... this was a good review
i still have the m1 air even tho its old its still works pretty good for me and its fast enough for most of my tasks.
how much RAM do you have?
@@AZisk 16 gb ram and 512 gb ssd
Thanks!
🤩 thank you much!
I personally wouldn't splash out on a M3 Max because I expect the Neural Engine to grow in importance in future iterations of Apple Silicon. The current GPUs are impressive but with local LLMs on the horizon, larger onboard NPUs would be "cheaper" to train and run those models as they are expected to be used more frequently. For the time being, a cheaper M3 Pro or a refurb M2 Max can complete the job anyway, just a little slower.... to allow you to have a tea break.
Glad to find neuronetwork tests, it’s not easy to benchmark that NE, for iPhone prolly harder
can u make a video on tensorflow-metal n configuring them in a conda environment and the performance difference
Can you compare Nvidia rapids cuDF and classic pandas 2.0 on apple macbook m3 max?
I just upgraded from M1 Max 10-core w/ 64gb ram to M3 Max with 16 cores and 128gb ram. I edit 8k video and 50 megapixel stills in Final Cut X and Capture One and did not see a significant speed increase in export speeds and general usablity. Maybe this is a testament to how good the M1 is. I have both of them on my desk and while I have not run comprehensive tests with a stopwatch I only notice the improved speed in loading web pages in chrome with lot of ads. I am running 2 studio monitors and will be getting a pro xdr so the m3 was necessary for this to work.
Nobody cares, this is a video about ML
Yeah a lot of apps use machine learning eg Final Cut “Note: Because Final Cut Pro uses an enhanced machine-learning model on Mac computers with Apple silicon, tracking results may differ between Mac computers with Apple silicon and Intel-based Mac computers.”
@@timjenkins8943 that has nothing to do with training and testing MLMs
Hi Alex, I need your help deciding between two options for a MacBook Pro - the M3 Pro with 18GB or 36GB of RAM. I want to use the MacBook for learning to code and I don't want to use swap because I heard it could shorten the lifespan of the SSD. I want to choose a device that will work well for me now and in the future. Can you please advise which option is better for my needs?
Always go with more RAM. The more the better.
Can you make a video comparing the M3 Pro MacBook Pro 18gb with 36gb regards to developers' needs, also including VM?
@@sreeharisreelal be aware, that Windows/Linux are a bit castrated in Apple Silicon VMs. No GPU/ANE-access, no nested hypervisors,... at least for now..
I do not know about ML, but I wonder the feasibility of starting a model development on M1/M2/M3 and latter on move to a NVIDIA server somewhere in the cloud. Would it require any modifications?
Moving from one architecture to the other is easy. The Apple Silicon Machines really shine for local single-user prompt-processing. E.g. for on-site/local Retrieval Augmented Generation (RAG). Apple RAM (even on the Ultra) looks cheap, if you compare it to buying your own workstation-class NVIDIA GPUs. To me NVIDIA shines, if you can rent it in the cloud, I'm happily playing around with RAG,... on a M2 Max 96GB. It even runs Mixtral 8x7B reasonably well locally (Q5k_m quantized = compressed with a good tradeoff between size and quality-loss), but I still hope for some performance tweaks for it.
depends what tech you use. if it’s cross platform stuff like pytorch, then sure (as long as you account for mps vs cud). but if you use the new mpx framework, that will only work on apple
Very nice, keep up the great work. Apple is on to something with the unified memory and MLX. Looks like all cylinders will be firing with the release of M4.
Hi ! Thanks a lot for your very instructive videos. It really makes me thinking about purchasing a mac. I’m ML engineer and i usually debug locally and send job to cluster like azureML. One think that i like for debugging is using blobfuse to mount on my local filesystem (wsl) a cloud hosted storage. It’s basically a apt package... is there a way (maybe with rosetta) to use apt x64 package easily in a terminal without using any x64 vm or things like that on OSX ?
Thanks !
We are waiting your test on MLX 😁
3:03 I have the M1 Max Ultra gluten free top spec whatever MacBook Pro 15” and am kinda relieved, that the gap is not so huge. Hope it lasts until the M5 uperduper
This is whats stopping me from getting new pc/laptop.. I needed something that can train Mistral model on consumer computers.
I saw another creator talk about how the m3 pro and max line of chips have lower gpu memory bandwidth then similar m2 or M1 chips and that he got better video editing performance from M2 Max then m3 max. Have you done a video about how memory badwidth affects coding? Or is this a thing that really only matters for editing vids?
MLX and Ferret are out so hope to see a video on that
How good are macs for ML compared to PC with NVIDIA GPUs that run CUDA?
NVIDIA CUDA is still king for any ML task right now. Most the frameworks were developed for CUDA, on CUDA with extensive documentation for it. Apple chips are still in their infancy in regard to ML/AI, so there isn’t much documentation or online help if you run into any issues.
Hi Alex can you make an comparision M3 Max 48 GB RAM vs 64 GB RAM?
Nice apples to apples comparison
I have a bit of a dilemma. Should I go for a Mac Pro M2 Max or a Base Model MBP M3?
Both are coming in around the same price. Looking to use them for running open source LLMs.
I do research in local llms and would recommend a sationary device
Would be nice to revisit Microsoft Volterra to see how it evolved in this aspect.
it hasn’t. there isn’t anything new with it yet. wondering if microsoft will continue it with a v2. i sure hope so. but then again, since qualcomm is pushing new arm chips next year, devs could get a better machine from other vendors potentially
Can U do a video on Stable Diffusion on all the Macbook u have ??
Can you make a video comparing the M3 Pro MacBook Pro 18gb with 36gb regards to developers' needs, also including VM?
Answer is 36gb I wouldn’t do the 18 at these prices
Haven’t watched the whole thing but I bet the M3 Max will SMASH the Air
Compare the money spent on: [MBP M3 MAX] VS [MBA M1 (with rest of the money spent on cloud compute units)]
I run Q3 Mixtral on my 32GB, again using Ollama
will M3 pro base 14 inch model be suitable for a university student doing UnrealEngine and c++ OpenGL, also note that this is my specialization and what i study.
Game development with Unreal you’re probably going to want Windows with an Nvidia GPU. Mac gets the short end of the stick in software support from epic games, having to wait to get many new features in Unreal months later, if they get them at all. Most other types of development are better on Mac, but game dev with Unreal specifically you’ll have a much better time on Windows.
can M3 max with 30 core gpu and 96 gb memory can run all kinds of LLM models like llama 2 -70b and also fin
etune stable diffusion ?
Llama 2 70b-hf in inference precision fp16/bf16 need 133GB VRAM/ fp32 263GB
Nvidia is still the standard in machine learning. I personally would not go that route, but if you know what you are doing, it may differ.
I’d like to see this test with the regular M3 pro. I’ve been using my M3 pro MBP and I want to get into machine learning so I’m wondering how it would do.
Performs close to the m1 max
@@blackice214 CPU stuff is equal to M2 max, but the graphics are not even close to M1 max. They blow every non-max chip out of the water, and the difference isn't even funny.
Similar to M2 Max
Which ram varient???
@@sreeharisreelal 18gb
At 8:04... Why is you M3 MPB named MacBook-Pro-2 ?
Where you really running on the M3?
yes
you can actually name your machine whatever you want
Awh cute dog. 🐕
How to run LLM on ANE?
Could you make a video going through Fedora Asahi please?
Which cheaper Windows/Wintel machines are faster?
Buying the M1 MBA was the best decision this year
I truly believe that you should not spend so much money on top Macbook since next year it will be almost obsolete, instead, you should buy one that allows you have a great battery, durable components and use cloud services as google colab, AWS or so on, that will save you a lot of money and you will pay only for what you are really using.
Ollama tests would actually be relevant information...
The question is, does the Performance depend on MacOS or can it perform on Linux the same?
Is m1 air 16gb enough for masters of artificial intelligence (I am going to study in 2024) or should I get a windows laptop
It has 2 handicaps, the slow RAM (68GB/s vs 100 on M2,M3) and 16GB is too low to run a non-quantized 7B (e.g. llama, mistral) model, you still can probably run all tutorials. As for getting a PC laptop (with probably Linux or Windows' WSL2 for AI), I think Macs are very competitive for mobile AI workloads. I run some things on my M2 MacBook Air with 10GPUs/24GB RAM (had a M1/16GB before, but liked the new design and could sell my old machine with not to much loss). But I think you might be better off with moving to cloud-based solutions together with your existing machine than buying a PC Laptop or a new MacBook.
@@andikunar7183yes but I can't buy a PC cuz I will be flying abroad for masters
I thought they do matrix multiplication would be do e in the npu not cpu…
Alex, I am getting 16” Macbook Pro M1 Max 32 GB RAM and 1TB SSD for $2099 with $1400 off. Should i buy it right now or 14” Macbook Pro M3 Pro ($2000) with 16 GB RAM and 512 SSD?
nice! congrats , that sounds like a great machine right there (and a really good deal). there are people in the comments that are skeptical about this. I’ve bought plenty of things from B&H, but never a laptop, so let us know how it goes
@@AZisk i live in india and electronics here are extremely expensive . I can give you an example. The MacBook pro , i mean just pro costs here 750 dollars more than the cost You get in USA. And the Max variants start from $4200, So when i saw this deal, i called my cousin in USA and he ordered this for me.
He received it yesterday. And guess what! it’s beast. I was using Macbook Air (Early 2015) which is still working till now without any complaint but i didn’t want to skip this deal so i got this.
Now i am waiting for him to come to india as he coming in next month so that i can enjoy my new mac. Btw it’s new and sealed. it’s warranty started from today. i am saying this because so many people are questioning the authenticity of this deal going on. It’s NOT refurbished ⭐️
@@SaqibFayazPatigarooNice! Thanks for sharing
@@AZisk You didn’t tell me, was my purchase worth enough? i mean i could have purchased m3 pro in lesser amount, did i did the right thing? i am asking this because i am not technically as sound as you.
I think so. You've got a good machine there@@SaqibFayazPatigaroo
Did you beat Covid yet Alex?
You look well.
thanks. getting there
M1 is amazing giving the price tag
yep
Could anyone give an idea of whether a MacBook m2 pro with 16gb ram is worth it over a M1 Max with 32gb ram for a similar price?
It depends. If you run AI things locally (the video title), 16GB is on the very low side. And the M1 Max has 2x the memory-bandwidth of the M2 pro. Also maxing out the number of GPUs helps with performance. However the M1 might have some future limits, because it is an older CPU (obsoleted earlier).
@@andikunar7183 I appreciate the reply! I’m not planning to run AI locally anytime soon however the M1 Max does have 30gpu cores that I’m looking at as opposed to the 19 on the m2. Are there any other advantages the m2 pro would have over the M1 Max? I know it has a better battery life but for the same price I could get an M1 Max with double the ram and ssd.
@@majorpeg8534 I don‘t know. Apple has not yet done anything that I know of, where they have a M2/M3 chip feature, not available on the M1. One general issue with Apple silicon is, that in virtual machines, they limit the chip (e.g no GPU, WSL2,… under Windows). The folks from Asahi Linux say, that is due to limitations on the M1 chip, and that some things could work with the M2. But nothing seems to have been implemented yet.
What do you think is a better deal, M2 Pro 32gb/1tb for 2099$, or M3 Pro 18gb/512gb for 1999$?
m2 pro
for MS office M1 just enough
i like how you palm
Looking forward to someone running transformers on ANE
Can you imagine if, overnight, Apple took the entire AI/Cloud processing business by releasing a power efficient chip, purposed for such tasks!? That would be INSANE. (and also fantastic for the environment!)
I'm sure they already do that for their own use in the back end. Their hardware under development is always 3 to 4 generations ahead, just not ready for the market, but maybe usable for their own use, if they deal with the issues.
No… not really. Most of their work with LLMs are on Nvidia GPUs. Remember their focus is on the consumer market with their silicon.
Can you do a battery test on the M3 pro chip vs M3 max
Fanless M1 MBA still have many Fans, Notch less too!🥰
Fine, then unless someone is issuing me a Max soc and needs me to be mobile then I can grab a used Mac Studio with an M1 ultra soc down the road.
Otherwise gpu instances from the public cloud provider I get budgeted will still be the play.
Hey bruh what u think about air m1 in 2024
Base varient
it’s a great machine
@@AZisk❤
Sir plz make a dedicated video on air M1 base varient after 3+years🙇🏻 because I have it😅
And the most imp thing is that sir windows laptops with 12th gen still lags on normal usage why is it?
my subscription is worthful...
happy to hear :)
@@AZisk The quick replier in youtube...
Asitop shows NE performance wow
Hey manager where’s have you been ❤ we waiting you for two weeks they no one’s telling us about this machine in the field it just you
sick
Can somebody tell Timothy his domain expired?
You know which thumbnail won the vote…
Apple's Mac certainly excels in operating system quality, design, and efficiency, enabling most users to perform exceptionally well. Nonetheless, for tasks involving artificial intelligence, machine learning, and deep learning, the high-powered chips of the Apple M series fall short. As of now, running TensorFlow natively on an M series is not possible, and while PyTorch can be installed on M series chips, attempting to generate a 15-second AI animation-something an NVIDIA RTX 4090 can accomplish in few minutes-can be an arduous 15-hour endeavor on my Mac M1 Pro max, with similar inefficiencies for facefusion, deepfake, and other AI generative tasks. After purchasing an M1 Pro Max in 2021, and facing challenges in adapting it for machine learning, I ultimately acquired a Linux machine with an NVIDIA GPU for professional projects, while the M1 now handles my daily tasks. I remain hopeful that Apple will address these significant limitations. Thank you for the amazing videos Cheers.
That is true. But we are three years into the Apple Silicon revolution, and Intel and AMD have a massive head start. Apple Silicon is improving the software and hardware integration much faster than I expected, but for some tasks, Intel/AMD remain the go-to devices...for the moment.
Base models are far more better than max model considering the price.
Describe better. Does better mean cheaper or does better mean faster or does better mean based on a value/performance metric. I'm confused.
@@jimtipton8888 Value. The 40 core GPU is of course better than an 7 core gpu.
@@jimtipton8888 Base model gives you around 70-80% performance as compared with maxed out model which basically means they are better and almost fits in everything on basis of that price range.
@@sushantatimalsina9064 Thank you!
Value its called. Yes best value base models but that doesn’t fit everyone’s needs
But GPU is much more in use while dealing with AI tasks compared to typical use due to numpy and other similar libraries dealing with matrixes, for which GPUs are basically designed, considering how graphics are being processed on our computers. So mind that this does not address typical use cases for most people and even programmers, who do not deal with AI. I'm not an AI expert but data engineer so I might be wrong here, but pretty sure I'm not, just wanted to clarify so people not working with AI or graphics won't accidentally choose their laptop based on NE/GPU performance. That being said I still consider M3 Max for python / sql programming, as I flooded my old good 2016 Macbook Pro with hellish keyboard. :P I will go with 16'' M3 Pro probably though or just max out regular laptop and go with Ubuntu. Also heard that M3 max generates more fan noise than M3 Pro under similar tasks, not sure if that's true.
Merhaba . Phyton gibi yapay zeka programları kullanmak , veri analizleri yapmak için MacBook uygun mudur ? Yoksa Windows tabanlı intel grafik kartlı bir laptop mu önerirsiniz ? Teşekkürler
Well that's not the thumnail which won
Yes, it is. For people that showed they cared about actually helping with the thumbnail :)
Still don't get why someone want to train a serious model with apple silicon.
For inference tho, pay your openai bill already. It is really a thing for datacenter, but I have little idea why you want to run this locally on a daily basis.
Some even compares this thing to A100? Oh, boy, let's talk about the training speed.
If you are relying on a laptop for training, that means you are probably not really into the game.
The link to blog post is full of security issues. The certificate is bogus.
it is like comparing a dead horse to a cheetah
which is faster? depends
@@AZisk The M3 max is cheetah and the m1 air is the dead horse. The M1 air is good for people that do web browsing and basic web dev. If you try to go deeper like react native in andriod studio the laptop will struggle. I bought a 16 gig m1 air last year and I had to return it because of extremly slow build times. Then i bought the m2 pro with 16 gig and it worked great!
are any of them any good at ironing t shirts?
Hm, twice as fast for 6 times the price? But yes, LOCAL AI goes really slow right now. Hope that the Snapdragon X Elite improves. Probably only at 64GB+. But I would not want to spend $5K on a laptop. Especially from Apple.