Academic AI researchers have never had the compute hardware to contribute to AI research since 2012, except some who worked at corporate giants (mostly deepmind) and went back into academia.
They are getting more hardware now, but the hardware required to be relevant and to develop a capability that commercial models don’t already have keeps increasing. Table stakes are now something like 10,000 H100s, or about 250-500 million in hardware.
I am not sure MIRI tried any meaningful computational experiments. They came up with unrunnable algorithms that theoretically might work but would need nearly infinite compute.
As you were being pedantic, allow me to be pedantic in return.
Admittedly, you might know something I don’t, but I would describe Andrew Ng as an academic. These kinds of industry partnerships, like the one in that article you referred to, are really, really common in academia. In fact, it’s how a lot of our research gets done. We can’t do research if we don’t have funding, and so a big part of being an academic is persuading companies to work with you.
Sometimes companies really, really want to work with you, and sometimes you’ve got to provide them with a decent value proposition. This isn’t just AI research either, but very common in statistics, as well as biological sciences, physics, chemistry, well, you get the idea. Not quite the same situation in humanities, but eh, I’m in STEM.
Now, in terms of universities having the hardware, certainly these days there is no way a university will have even close to the same compute power that a large company like Google has access to. Though, “even back in” 2012, (and well before) universities had supercomputers. It was pretty common to have a resident supercomputer that you’d use. For me, and my background’s orginally in physics, back then we had a supercomputer in our department, the only one at the university, and people from other departments would occasionally ask to run stuff on it. A simpler time.
It’s less that universities don’t have access to that compute power. It’s more that they just don’t run server farms. So we pay for it from Google or Amazon and so on, like everyone in the corporate world—except of course the companies that run those servers (they still have to pay costs and lost revenue). Sometimes that’s subsidized by working with a big tech company, but it isn’t always.
I’m not even going to get into the history of AI/ML algorithms and the role of academic contributions there, and I don’t claim that the industry played no role; but the narrative that all these advancements are corporate just ain’t true, compute power or no. We just don’t shout so loud or build as many “products.”
Yeah, you’re absolutely right that MIRI didn’t try any meaningful computation experiments that I’ve seen. As far as I can tell, their research record is… well, staring at ceilings and thinking up vacuous problems. I actually once (when I flirted with the cult) went to a seminar that the big Yud himself delivered, and he spent the whole time talking about qualia, and then when someone asked him if he could describe a research project he was actively working on, he refused to, on the basis that it was “too important to share.”
“Too important to share”! I’ve honestly never met an academic who doesn’t want to talk about their work. Big Yud is a big let down.
Just to be pedantic, it wasn’t academic AI researchers. The current era of AI began here : https://www.npr.org/2012/06/26/155792609/a-massive-google-network-learns-to-identify
Academic AI researchers have never had the compute hardware to contribute to AI research since 2012, except some who worked at corporate giants (mostly deepmind) and went back into academia.
They are getting more hardware now, but the hardware required to be relevant and to develop a capability that commercial models don’t already have keeps increasing. Table stakes are now something like 10,000 H100s, or about 250-500 million in hardware.
https://www.semianalysis.com/p/google-gemini-eats-the-world-gemini
I am not sure MIRI tried any meaningful computational experiments. They came up with unrunnable algorithms that theoretically might work but would need nearly infinite compute.
As you were being pedantic, allow me to be pedantic in return.
Admittedly, you might know something I don’t, but I would describe Andrew Ng as an academic. These kinds of industry partnerships, like the one in that article you referred to, are really, really common in academia. In fact, it’s how a lot of our research gets done. We can’t do research if we don’t have funding, and so a big part of being an academic is persuading companies to work with you.
Sometimes companies really, really want to work with you, and sometimes you’ve got to provide them with a decent value proposition. This isn’t just AI research either, but very common in statistics, as well as biological sciences, physics, chemistry, well, you get the idea. Not quite the same situation in humanities, but eh, I’m in STEM.
Now, in terms of universities having the hardware, certainly these days there is no way a university will have even close to the same compute power that a large company like Google has access to. Though, “even back in” 2012, (and well before) universities had supercomputers. It was pretty common to have a resident supercomputer that you’d use. For me, and my background’s orginally in physics, back then we had a supercomputer in our department, the only one at the university, and people from other departments would occasionally ask to run stuff on it. A simpler time.
It’s less that universities don’t have access to that compute power. It’s more that they just don’t run server farms. So we pay for it from Google or Amazon and so on, like everyone in the corporate world—except of course the companies that run those servers (they still have to pay costs and lost revenue). Sometimes that’s subsidized by working with a big tech company, but it isn’t always.
I’m not even going to get into the history of AI/ML algorithms and the role of academic contributions there, and I don’t claim that the industry played no role; but the narrative that all these advancements are corporate just ain’t true, compute power or no. We just don’t shout so loud or build as many “products.”
Yeah, you’re absolutely right that MIRI didn’t try any meaningful computation experiments that I’ve seen. As far as I can tell, their research record is… well, staring at ceilings and thinking up vacuous problems. I actually once (when I flirted with the cult) went to a seminar that the big Yud himself delivered, and he spent the whole time talking about qualia, and then when someone asked him if he could describe a research project he was actively working on, he refused to, on the basis that it was “too important to share.”
“Too important to share”! I’ve honestly never met an academic who doesn’t want to talk about their work. Big Yud is a big let down.
A joke I heard in the last century: Give a professor a nickel and they’ll talk for an hour. Give 'em a quarter and you’ll be in real trouble.
It’s true, I’m terrible for it myself 😅