DeepMind and trying to fairly hear out both AI doomers and doubters | Rohin Shah (2023)

Originally released June 2023. Can there be a more exciting and strange place to work today than a leading AI lab? Your CEO has said they’re worried your research could cause human extinction. The government is setting up meetings to discuss how this outcome can be avoided. Some of your colleagues think this is all overblown; others are more anxious still.
Today’s guest - machine learning researcher Rohin Shah - goes into the Google DeepMind offices each day with that peculiar backdrop to his work. He’s on the team dedicated to maintaining ‘technical AI safety’ as these models approach and exceed human capabilities: basically that the models help humanity accomplish its goals without flipping out in some dangerous way. This work has never seemed more important.
For years Rohin has been on a mission to fairly hear out people across the full spectrum of opinion about risks from artificial intelligence - from doomers to doubters - and properly understand their point of view. That makes him unusually well placed to give an overview of what we do and don’t understand. He has landed somewhere in the middle - troubled by ways things could go wrong, but not convinced there are very strong reasons to expect a terrible outcome.
In this episode:
• Rob's intro [00:00:00]
• The mood at DeepMind [00:06:43]
• Common misconceptions [00:15:24]
• Rohin’s disagreements with other ML researchers [00:29:40]
• Ways we might fail [00:40:10]
• Optimism vs pessimism [00:55:49]
• Specialisation vs generalisation [01:09:01]
• Why solving the technical side might not be enough [01:16:39]
• Barriers to coordination between AI labs [01:22:15]
• Could we end up in a terrifying world even if we mostly succeed? [01:27:57]
• Is it time to slow down? [01:33:25]
• Public discourse on AI [01:47:12]
• The candidness of AI labs [01:59:27]
• Visualising and analogising AI [02:02:33]
• Scalable oversight [02:16:37]
• Mechanistic interpretability [02:25:56]
• Dangerous capability evaluations [02:33:41]
• The work at other AI labs [02:38:12]
• Deciding between different categories of safety work [02:46:29]
• Approaches that Rohin disagrees with [02:53:27]
• Careers [03:01:12]
----
The 80,000 Hours Podcast features unusually in-depth conversations about the world’s most pressing problems and what you can do to solve them.
Learn more, read the summary and find the full transcript on the 80,000 Hours website:
80000hours.org/podcast/episod...

[@SirCreepyPastaBlack]

I had an interesting moment with Copilot recently.
Asked it to analyze the movie Metropolis (2001) in the style of Bing and then Copilot. (Results weren't great because i didn't use the correct wording also, doubt it would be able to capture the difference in opinion well due to the lack of context, different weights, etc.)
When it replied in the copilot style, i saw some uniquely saucy wording that my style of prompting doesnt normally bring. So I told it "ya know this isn't a test, right?" I meant it as the normal meaning as well as it isnt being trained now.
The reply took 3-5 seconds. I normally only get brief pauses, 1-2 seconds.
Id appreciate someone more familiar with LLM's to either explain why that happened or validate it was strange if ya can. Danke

[@michelleelsom6827]

So my worry is that AGI & ASI will simply use the energy they need. They won't worry about leaving some energy for us unless they are aligned to do so 😢

[@aisle_of_view]

We're already seeing accounts of people losing their jobs to AI, and upon the trend you can depend.

[@RasielSuarez]

It's as if all the protocols are being designed around the premise of the good actor. Consider the scenario of a suicidal or psychotic developer who gets their hands on AGI then goes about training it as a perfect weapon. What then?

[@nicholascurran1734]

I have liked the discussions I've listened to on this channel, and am wondering where to find them that isn't a year later than the original release date?

[@eightythousandhours]

We've only recently started uploading new podcast episodes regularly to our CZcams channel, so you'll find new episodes uploaded here but we're still catching up on uploading the full archive.
You can get new audio episodes the moment they're released by subscribing to the 80,000 Hours podcast on either Apple Podcasts or Spotify.

[@bobtarmac1828]

Laid off by Ai then human extinction? An Ai new world order? With swell robotics everywhere, Ai jobloss is the only thing I worry about anymore. Anyone else feel the same? Should we cease Ai?