Notes on AI Hardware - Benjamin Spector | Stanford MLSys #88

Episode 88 of the Stanford MLSys Seminar Series!
Notes on AI Hardware
Speaker: Ben Spector
Abstract:
This week, one of our hosts -- Ben Spector -- is subbing in at last minute to deliver some thoughts on AI hardware, and why we should probably be investing more time in learning about it
Bio:
Benjamin Spector is a second-year PhD student in computer science at Stanford. His interests center on how systems can make AI faster and more open. Before coming to Stanford, he received both his bachelor’s in computer science and mathematics and master’s in computer science at MIT. He also started a not-for-profit startup accelerator, prod.so, cofounded cofactory.ai, and researched computational models of fusion while at MIT. In his free time, Benjamin enjoys cello, ping-pong, history, cooking, and hiking.
--
Stanford MLSys Seminar hosts: Avanika Narayan, Benjamin Spector, Michael Zhang
Twitter:
/ avanika15​
/ bfspector
/ mzhangio
--
Check out our website for the schedule: mlsys.stanford.edu
Join our mailing list to get weekly updates: groups.google.com/forum/#!for...
#machinelearning #ai #artificialintelligence #systems #mlsys #computerscience #stanford

[@rfernand2]

This is a presentation that Ben "threw together" at the last minute? Amazingly well done!

[@420_gunna]

Ben continues to be a stud 💪💪💪
Thanks Stanford students/faculty for putting these online, they're among the beast learning opportunities for people on the sidelines 😄

[@lpang]

I am glad you talked about inverse lithography technology (ILT), which I named twenty years ago, and I am still working on it using GPU acceleration. BTW, I also got my PhD from Stanford

[@radicalrodriguez5912]

great presentation. thanks

[@andrewm4894]

Love this! Thanks!

[@maximliu]

Great presentation! Wondering if there is any literatures or papers, tutorials on the similar topics? The talk was kind of quick, need read more specifics from literatures. Any pointer would be appriciated. Thanks!

[@BenjaminFSpector]

I blew through a ton of different topics in the course of the talk, so it really depends what you're looking for.
If you want more on making the most of an H100, NVIDIA has fairly good docs on both the CUDA programming model as well as the specific features of the H100, but actually using them can be tricky, so your best bet is probably to read the CUTLASS repo and see how they do things.
If you want more on hardware design, I'm not sure there are great alternatives to taking a class. Hardware design seems to me like an awful lot of work -- writing good RTL is hard enough, but the whole EDA stack is a bit of a nightmare.
If you want more on semiconductor manufacturing, I'd highly recommend the Asianometry YT channel, which has a lot of really excellent content.
Otherwise, some of my main sources for this talk were SemiAnalysis ($500/yr, but I like it enough that I pay for it even from a grad student stipend), Bill Dally's HC2023 talk, and various coursework, particularly 6.172 from MIT for performance engineering. (It's on OCW at ocw.mit.edu/courses/6-172-performance-engineering-of-software-systems-fall-2018/video_galleries/lecture-videos/ and while it's focused on CPU performance engineering many of the principles apply across both.)
Hope this helps!

[@prasannaprabhakar1323]

@@BenjaminFSpector Thanks a ton man! What you have shared here is gold. I really appreciate it.