Is AI Really Taking Our Jobs? Humanoids, AGI, and the Future of Work with Ben Armstrong
Publicado no IA Dots ·
Agentes de IAAGIAI Signaldot-agidot-ai_agentsdot-artificial_intelligencedot-humanoid_robotsidioma-enInglêsRobótica e humanoidesyt-8DTjwiYEw-k
Vídeo em Inglês, publicado pelo canal Open Data Science and AI Conference.
About this content: The summary, key points and topics below are generated automatically from the video transcript. They reflect the content, opinions, claims and predictions presented in the video and do not represent a position, validation or fact-check by IA Dots.
Video summary
This episode of the podcast features a discussion with Ben Armstrong, executive director and research scientist at MIT's Industrial Performance Center, regarding the impact of artificial intelligence and large language models on the future of work, job losses, corporate adoption, and industrial competition.
Key points
- Research from MIT's Industrial Performance Center shows that core technical tasks vulnerable to algorithms often represent a fraction of a larger job role, creating human inertia in the economy.
- Historical data on automation and robotics indicates that predictions of massive job displacement have often been exaggerated, with organizations focusing on augmenting tasks rather than wholesale replacement.
- China has built significant competitive advantages in robotics and manufacturing through scale, industry policy, and targeted regional support.
- Studies on large language models in the workplace show that while code generation outputs may increase, overall productivity gains and production-ready outputs do not always scale proportionally.
- Corporate layoffs are frequently attributed to AI for branding and share price boosts, while actual hiring data reveals a flattening or reduction primarily in junior-level roles due to uncertainty.
Topics
Artificial IntelligenceFuture of WorkRoboticsIndustrial PolicyLarge Language ModelsLabor MarketProductivityAutomation
Video transcript
Transcript from available captions; it may contain errors.
Some of our job, certainly relies on tasks that even AGI can be imagined to do, or maybe tasks that even a successful humanoid could do. But that's not all of our job. And I look at the studies of radiologists whose main core critical task is now done equally well by an algorithm, right? And these are very highly skilled high-wage people. But even that core task is only 20% of their job, right? When we do task studies of radiologists. So, I do take comfort in a lot of this that our society and our economy is distinctly human in a lot of ways. So, let's say that we have algorithms that can do the technical parts of our job just as well as humans tomorrow. I still don't see the dire implications or the dire speculation of AGI folks coming to pass given just how much inertia, you know, like a good inertia, you know, human inertia there is in the economy in the world. >> Welcome to the AI X podcast. This episode is sponsored by O'Reilly AI West, the premier AI builders conference coming up this October 27th [music] to 29th in San Francisco and virtually. Learn more at odsc.ai/west and use the code podcast for an exclusive 10% discount. And if you're on the East Coast, join us for ODSC AI New York, our AI for work summit taking place December 2nd and 3rd in Manhattan. It's a hands-on event focused on building and applying AI agents, automation, and AI-powered workflows to real workplace challenges. Learn more at summit.ai and use the same code podcast [music] for 10% off. >> Hello listeners and welcome to the ODSC AI X podcast. Now, technology leaders increasingly talk as if AGI is close or even already here. But what would that actually mean for jobs? My guest today is Ben Armstrong, executive director and research scientist at MIT's Industrial Performance Center and co-lead of its Work of the Future initiative. For the past 3 years, Ben and his colleagues have studied how more than 50 companies and organizations are adopting AI and how it's changing work. We discuss what the evidence shows about AI and job losses, whether recent layoffs have really been driven by AI, and why junior hiring may be the first real pressure point. We also look at China's advantages in robotics and manufacturing, whether supervising AI will become a lasting form of work, what decades of automation can teach us about humanoids and AI agents, and whether AGI is even a useful way to think about the future of work. So, let's get into it. Ben, welcome to the podcast. >> Thanks for having me. >> Delighted to have you, Ben. You spoke at ODSC AI East back in April, and great to have you on the podcast cuz you have been doing a lot of work and research that I think will be of a ton of interest to our audience, especially your work of the Future initiative. Before we get into that, Ben, could you give our listeners a little bit of color on your background and what you're working on right now? >> I lead a group at MIT called the Industrial Performance Center, which has been around for 35 years. It's been focused, as the name would suggest, even though the name might be a little Soviet-sounding at this point. Around the time when it was founded, the industrial performance was a key objective for the United States relative to the rise of Japan in particular. And there was concern that with competitive Japanese steel and automobiles, that the US needed a plan to come out and compete again. And a group of faculty at MIT started a commission around how to improve American productivity. They came out with the book called Made in America that was sort of a blueprint for policy of how the US could compete. And the IPC, Industrial Performance Center that I now lead, came out of that book. And in the three-plus decades since, we've been focused on similar research around how you compete in a changing economy, an economy that's changing in terms of people's work preferences, in terms of how technology is changing. A lot of our work historically has been around globalization and why companies want to produce things in Asia instead of the United States or in Mexico instead of the US. So, that has been the core set of questions that we've been interested in around industrial competition, economic competition. And around 2018 or so, these questions around technology and work really started to rise to the surface. Some very exciting robot demos, concerns that robots were going to displace a large share of workers. There was a very popular paper, while it was still a working paper, around 2014-2015 from Frey and Osborne, and they predicted that a percent or so, or up to 47% of jobs were vulnerable to automation. That paper became a favorite punching bag for certain academics cuz they pointed, well, after 2014, a decade later, a lot of the jobs they pointed out as being highly vulnerable, there were more of those jobs than there were in 2014. So, these jobs didn't seem to be declining at all. And it turns out a lot of these exercises by economists to predict the future and which jobs are going to stay and which jobs are going to go end up being a fool's errand. But anyway, our research, my colleagues at that time, I wasn't part of the organization in 2018, they set out to answer that core question of how should we think about the role that technology is going to play in reshaping work. And it was a question of is technology going to affect the number of jobs available? Are we going to have a jobless future, which is what some people were predicting? Or maybe more importantly, was technology going to affect the quality of jobs? So, my colleague David Autor has a long history of doing work on the hollowing out of the middle class and the role of automation technology, new technologies in really increasing demand for low-skill, low-wage jobs and increasing demand for high-skill, high-wage jobs, essentially at at poles of the economy, but reducing demand for jobs in the middle, the classic middle-class jobs that were very helpful for people without a college degree to live the so-called American dream. That was a real interest of the groups was how do we develop policies or organizational practices that maintain good high-quality middle-class jobs as technology might put pressures on the labor market to polarize and really help jobs at the extremes. And that was all before the drop of GPT-3, GPT-3.5, and this new LLM boom. As that work wrapped up and I moved into my role, we thought we really want to apply what we've learned from studying robots and studying traditional industrial automation and see if those same lessons apply with LLMs. In 2023, we started a study that we tried to look at the earliest use cases of LLMs in established companies. Every company early on had a blank check to experiment with these technologies. An overstatement every company, but a lot of the companies we studied were so eager to figure out ways to use LLMs productively within their organizations. And we just started going out to organizations that we knew either through MIT connections or through our past work and trying to understand what they were doing. What were the use cases? Were they valuable? And as they experimented, we stayed in touch with the companies to see what worked and what didn't. And that became a three-year study where we would convene these companies. Over 50 companies joined our working group. 50 organizations, some were non-profit, some were companies, joined our working group over time. And we got a nice snapshot of the early days of how large organizations were using these tools. I would say that the headline was that a lot of the historical lessons of how automation works within big complex organizations still apply, that LLMs weren't rewriting how organizations use technology or how work is affected by technology. >> Yeah, Ben, there's a ton to unpack there. Thanks for giving us the background story on your group at MIT's Industrial Performance Center. And it's interesting how it originally started with concerns around Japan. And I do remember that even though I'm from Ireland myself, but I remember growing up in Ireland and it was kind of fascinating to us how obsessed the US was with Japan. And it was all about how Japan is beating the US because of unfair competition. Everyone's buying Japanese cars, something to do with exchange rates, trade policy. People are complaining about intellectual property issues. And now, of course, that concern has completely migrated to China and the AI race. >> So, now from the Irish perspective, or maybe just your individual perspective, do you think that's silly that we're concerned about Chinese competition now? >> I don't >> that we were concerned about Japanese competition. >> I don't think it's silly cuz I have a bit of a personal anecdote there. I was in Boston doing my masters in computer science in the mid to late 90s. And I was working for a manufacturing plant that used to be in Hyde Park. And eventually that just moved away. The jobs went away, the factory went away, and they were making electrical materials. And that was when I forget when China ascended to the World Trade Organization, but a lot of people were complaining about it. >> 2000. >> 2000, yeah, just around then. So, so yeah, I saw firsthand how a factory that had employed five or six hundred people in Hyde Park in suburb of Boston just kind of disappeared, you know, slowly but surely over the course of three or four years. Very sad to see. I think China has built real competitive advantages through scale, manufacturing strength, and has to be said, industry policy. While some of those, I think while some of that advantage also comes from practices we would reasonably view as unfair, maybe. So, on the robotic side, they are definitely pulling ahead, but is it unfair competition? I'm not qualified to give a definitive answer to that, but when you look at Boston Dynamics selling a $200,000 robot dog, and contrast that to Uh, we bought a Chinese Unitree robot dog for I think we paid $4,000 for instance. Not quite as performative, >> Yeah. >> but very good for the price points. And I was also looking at the Robot Olympics China just hosted, which is pretty hilarious. Like very, very janky and a lot of robots were controlled by wire, >> Yeah. >> but still quite impressive. But it's interesting you asked me that question because from an outside perspective and having lived here quite a while now, not so much an outsider, but the Japan debate seemed to be like, "Hey, the US invents all this great technology, but we can't turn it into jobs and production." Which by the way is what everyone in the UK and England have been complaining about for years. Same problem, right? Great inventors. So, I think it's a very nuanced debate, right? >> It is it's related. So, my core work is on manufacturing still. I look at this as a way of understanding the industrialization and commercialization of technology might vary in an office environment or software versus hardware, which is the majority of what I focus on, but it's so parallel to what you're saying that you can invent a technology in the early readiness levels, but not have the muscle to harden that technology and commercialize it. And with so many of our electronics inventions are now more and more our medicines, we don't really have the muscle to commercialize and scale up our technologies. I think this is a really interesting evolution of the open weights that kind of open versus closed models debate. China seems to have that commercialization and scale up muscle that's been very well developed in their manufacturing economy. And it's now becoming more and more ruggedized with their software, particularly software as a services companies. So, that's something that I think we really need to learn in the United States more than that we just already know or can lean back on. >> Yeah, it's incredibly interesting because there's fair advantages and unfair advantages, right? Because China undoubtedly has the manufacturing scale and look what they're doing with clean energy. It's fantastic that so many clean energy products like solar panels and solar storage batteries and they've really dropped the price of that down and it's become so accessible, especially to third world countries, helping reduce pollution. But there is a lot of state support around that and they're fostering competition and in AI robots it's very interesting because like you, I've followed the robots very closely. There's my robot arm. I'm trying to famous guy. I probably got a couple of them. Um >> Why do you focus on fairness? I guess is my cuz the state support, I've never really accepted that argument. Like that >> All right, so let's let's let's try this again. Could you push back in that? So let's talk about one area that I have a little knowledge about is let's look at the field of robotics, right? So Unitree just IPO'd. Their stock went up 600%. But when I looked at that prospectus and you look around is you see that each of these robotics centers in China, what they do is they set up these centers of excellence around robotics, right? >> Yes. >> And just like in what Nvidia is doing in the US with this circular financing, they're doing something very similar in China where these robot research centers, where they train robots, are buying from the robot manufacturing companies. The robot manufacturers are selling these robots to these research centers. The centers are using these robots to generate training data, which they sell back to the robot manufacturing companies and then use that money to buy yet the more robots. So you can see this very strong local states government support creating this virtual cycle of innovation and development. Now, is that good industry policy? Probably yes. Is it good local policy for jobs in future? Yes. Is it also slightly unfair that these same states are basically funding these centers with government money? Whereas in certain rules in the US and the UK, or sorry, European Union, that may not be allowed. I think that's one example around robotics. Do you see that as well? >> So I I totally agree that that's what's happening. There's concentrated regional industrial policy in China. There's extreme levels of competition the state has cultivated, and there's a lot of supporting suppliers by you know, weird buyer supplier relationship. Your use of a circular model that makes perfect sense. Now, from my perspective, there that isn't unfair necessarily. It's just we're jealous that works so well. So I do think that a lot of these unfair trade claims, like dumping practices, I get why we need to have enforceability around dumping and other established anti-competitive practices in the WTO. I'm with that. I also think that fair labor standards and environmental standards are things that we should try to enforce at a global level through diplomacy. I'm all in support of that. But that they've developed these really thriving local innovation ecosystems, good on them. And we got to try to compete. It's almost like complaining that your competitive team recruited a bunch of star athletes and are now kicking your butt. It's like, whoa, you got to go find your own athletes to compete. That's some of my frustration where I work with manufacturers that really are trying to figure it out. What's our alternative to the Chinese model, which we can't replicate in the US in the same way that China does it. But they want to throw up their hands and say, well, we can't compete. China's 10 cents on the dollar labor wages. It's not the only differentiating factor, and we really don't have a choice but to try to compete in a lot of these domains. >> And we can pivot that to the job market because it's very interesting because if you look at the advantages that China has right now, open weight AI models, they've advantages in that. They've computer advantages. They definitely have data center advantages because they can build anywhere. Unlike in the US, you can get all this pushback as it rightly should be. They have >> energy, you know, energy to supply the data centers, construction to build the data centers. Yeah, keep going. >> Robotics, they're way way ahead in robotics. And even things like procurement and supply chain, like it's so far advanced at this point. But at the same time, like everyone's worried about the jobs, but right now in China, if you read what's going on there, there's a lot of unemployment there as well. That's a big problem. And with so much government control, it's interesting I don't hear about this too much, but they're barreling towards the same problem themselves. Like China has over 150 humanoid robotics companies, and if humanoid robots become better and cheaper than human labor, like we saw Figure 8, they released a new model that's quite good, you know, it went into the untrained environment for the first time. I was able to do laundry, do cleaning, and things like that. So, it's interesting because they're building these very sophisticated humanoid robots that may exacerbate their unemployment. That's one issue. Secondly, they're in a state-controlled environment where AI can do a lot for you. So, eventually these AI models, if they become too powerful, then obviously China will shut them down. Just like they shut down the sale of Mana to Meta, for example. So, I find it curious that we're so concerned about competing with China in AI because at some point, as the models get better, China may want to just freeze the open weights and models are not releasing new ones or prevent their use overseas. So, I don't think it's as simple as people are painting it to be. >> Yeah, and I don't think we know So, what we know from the last 10 years or so, that the Chinese industrial economy has had a big shift. It's a shift from lower value-added products and electronics to higher value-added products like the bioeconomy and automotive in particular. We've seen this explosion in the automotive industry in China that now serves a huge market in the rest of the world. And their growth in robotics, this is the biggest growth market for industrial robots in the world is largely, I think, attributable to changing labor market and the unit economics for robots making more sense because wage rates have gone up in China, but also because you just use a lot more robots to make cars than you do to make devices that are assembled by hand. So, that's I think the easy explanation. Now, the question is that they made this big bet on humanoids, which I think is partly motivated by the demographic cliff in China that they just don't think they're going to have enough workers to expand the economy and they need humanoid workers. I'm not convinced that that's the right bet. So, I actually think it's going to be much harder to make these robots flexible in practice than it is in principle. We took us a long, long time to make very rigid industrial robots work at scale. So, I think there might be some growing pains from this imagined deployment of humanoids in in China that remains to be seen. >> Oh, yeah, but it is fair to say that China is now the leader in caged robots, we say those robots that are fixed in place at car manufacturing plants. >> For sure. >> Versus say Japan, which was the leader. >> Yeah, I mean, still by like as a share of their industrial economy, places like Japan, South Korea are still doing well. But you're right, like since the mid 2010s, places like Japan and the US and Germany, we've been relatively flat in our robot adoption of traditional industrial robots. China is up into the right. >> Oh, yeah. >> Um the question for me and our work around LLMs is not about can we build the AGI model or can we build a higher and higher performing model? It's like, could we actually extract productivity benefits from the models that have already been developed? And I had been under the operating assumption that this was going to follow a similar trajectory to search, which I think I've been wrong about many times over. With search, you get a big user behavior change, which is a lot of people go from or you pre-internet search methods to starting to use a tool like Google. And then, once they lock in their behavior around Google, you see marginal improvements in Google search algorithm and you you still for many years had the 10 blue links, but the 10 blue links become irrelevant, directs you to other apps, et cetera. I thought there was a possibility that the trajectory of our LMUs would see one big bang where you see enterprise users and individual users change their behavior from traditional information acquisition or research tasks. A lot of information acquisition tasks remain the modal use for AI. And then you would see marginal improvements in how accurate these tools were and how useful they were in acquiring information. I think we still have sort of a a dearth of evidence around where these tools give us productivity advantage and where they don't. We have a little bit of evidence that in customer service applications they improve the learning curve for customer service workers. This is Daniel Lee and Erik Brynjolfsson's work. We have pretty scant evidence actually that they improve the average productivity for software engineers despite the widespread use of these tools for computer programming. We have one paper that I think is the best evidence I've seen that shows under certain conditions you see a big output in code but not a huge gain in the number of projects completed. And when you see high improvements in output of code, you actually see more bugs that you need to fix. So you see a shift in work from programming to bug squashing. Which again, not surprising given the experience that many of us have had or have observed with these tools. But are we going to see an aggregate level productivity gain that lines up with other general purpose technologies of the past? We just haven't seen it yet and I don't know what the timeline is to see that. I'm not sure either that pushing the frontier of the models is what we need to get there. I think it's more around organizational transformation and figuring out how to line up the use of these tools with human teams versus trying to push organizations to being all lights out run by autonomous agents. >> Those are very solid points. I've seen that first hand. For example, there was another report out of maybe a month ago and we actually wrote an article in our blog site about Meta. They found exactly the same issue around productivity you talked about. Whereas they saw code generation outputs increase by some ridiculous number. It might have been 300%, it might have been 600%, but it was pretty large. But, also exactly that. The actual code that made it into production was only something like 25% if I recall correctly. It was a 40% increase in bugs. So, it was either a wash or maybe slightly forwards, exactly as you said. But, one caveat I would say to that is working and coding in a a brownfield code base, an existing code base is much less productive versus if I coded an app from brand new code base, which I think is very productive. So, it depends on what kind of a code base you have. But, yeah, one thing that I keep an eye on and I kind of trust this even though it is obviously biased, but it's the Tropic Economic Index. Even there, they're saying something like they're expecting, and this is from a company that's IPO'ing in a few months with a $2 trillion valuation. I think if I got the number right, was it something like 0.1% or 0.2% productivity, which is nothing for economy size of the US. And then, interestingly enough, they had a third report where they released this radar chart and it looked across different job profiles like lawyers, legal, HR, software engineers, people working in knowledge work with exposure to AI and it showed a gap between their productivity right now and the potential productivity. And they were saying that developers are right now reaching 10% productivity where the actual potential is 80%. And to be honest, I don't believe that. I work with AI all the time. I'm not getting anywhere near those kind of numbers. But, really I'm kind of getting to the question here is you see more realistic what's going on cuz you talk to a lot of companies. Like, how are these companies really laying people off based on the capabilities that AI has right now? Cuz I don't see it. I don't see these models at their present being that capable that they can replace someone in their job, although that can change quickly. >> No, they're Yeah, they're not. We don't see that at all. Uh I think there have been a few somewhat isolated cases where a new technology, new software tool comes in. This is even pre-LLM where people have been doing routine jobs that now the tool can do for you. You imagine RPA, too. What a lot of these agentic companies used to be were RPA companies. Um and RPA could do a series of routine administrative tasks that might make someone's job irrelevant. What we had seen in practice when we were studying RPA in the hospital setting was that if an RPA could come in and do 80% of someone's job, they would reshuffle that person cuz there was excess demand for work within the hospital system. So, that person then became someone who did the non-routine work that the RPA couldn't do. So, as long as you have a growing system or organization, these technology tools come in and take work ideally that people don't want to do, and then people can be reallocated. The worry about these tools and historically the worry about robots was, let's say you have an organization that's producing a pretty stagnant output. Once you have better tools, you're just going to need fewer people to provide the input. We haven't seen that yet. We haven't seen organizations, existing mature organizations figure out how to do the same with far fewer people using AI. Now, they're not going to go out and scream that from the rooftops. That doesn't look great. Some organizations have hinted at it in their earnings calls, but we haven't found much deeper evidence that they say, "Oh, we're using these tools and we're laying off a bunch of people." But, there seem to be other reasons for those layoffs in most cases. I think the best evidence of any change in the overall demand of labor is around new hires. And I think you see a flattening or a slight reduction in new hires in computer-related professions. And I chalk a lot of that up to uncertainty. That we don't really know what we need given that these tools are still filtering into the workforce. And that uncertainty about how to best use these tools seems to be affecting hiring. And when I talk to companies, I wonder is like wouldn't you want to scoop up all of these 20-something eager workers who know how to use these tools well and aren't being hired by your competitors? Because if you don't hire during this time period, you're going to have a big gap in your workforce of middle-skilled people eventually if you start hiring up again. Because that's what we saw manufacturing with companies like Boeing. When they went 20 years without hiring, then they had a bunch of senior people with a lot of knowledge and a bunch of junior people who weren't quite ready to absorb that very sophisticated knowledge. And that's a big problem for these more mature organizations. >> I want to get back to that pen, but >> Again, it's a big question is AI taking jobs in its present form because a lot of people are making decisions on future career moves based on the premises that AI is taking jobs. So Look, because again, we run new sites, we see the announcements from Block laid off 4,000, Amazon I think it was 20 or 30,000, Atlassian cuts 10% of their workforce PayPal Alliance Cloudflare. And look, it's much easier to blame these layoffs on AI instead of saying, you know, we want to increase the share price cuz as soon as you announce layoffs, everyone knows the share price going up. But another thing people are saying as well, a lot of these companies over-hired during the >> Yeah, >> Yeah, so what are some of the excuses you're hearing out there? >> Well, I'm not I'm not sure I'm not sure what's actually going on. I do think there could legitimately be an improvement in productivity in some of these firms such that they need a smaller, more nimble software engineering team. That's certainly a possibility. I don't think that's the whole story for a lot of these layoffs. If you look at the Block example, clearly Block is, you know, we're reverting toward COVID-era hiring levels even though it's it looks like a big layoff now. It would be consistent with a story of over-hiring or it's consistent with a real significant increase in their productivity of their software engineering workforce. Why I don't completely buy the productivity argument is that we don't see that in the research done, the granular-level research done on productivity. And I don't hear that from companies that I'm working with. Now, the companies that I'm talking to and they come to MIT could be way behind the curve. A lot of them Fortune 500 companies, pretty impressive, very profitable, but they could be way behind the curve. But you know, it could be a greenfield versus brownfield issues. These are mature companies with layers and layers of technology that might be hard to change. Whereas if you have a younger, more nimble company, then it has an easier time deploying these tools. But what I can say from those legacy companies, the ones that are more established, I don't think that they've seen extraordinary benefits from these tools so far, and they're still trying to figure out how best to use them. I'd say they're very much in the early days of a multi-year deployment. I imagine that these tools will filter in to their organizations over time, similarly to how enterprise software filtered in these organizations many years after the internet became very popular. My sense, and I've been trying to think about this, if I zoom out, the rise of LLMs and the progress of frontier models being similar to the consumer internet. That the tools and capabilities of the internet became widely apparent before they ever had a significant impact on companies, particularly established companies. And these companies had really poor websites or any internal use of the internet and software for at least 10 years after these the technologies were available. So they were multiple generations behind the frontier for a significant period of time. And I don't see any reason why this time is going to be different as they figure out how to use frontier models. And I should say, you know, we have good evidence from more traditional machine learning models that a lot of those models are could be more helpful to big research-based companies like pharma companies, manufacturing companies, and they still haven't figured out how to use them at any scale. >> Yeah, I think your insights are spot-on there because look at the rise of e-commerce and so I think comparing it to the internet and web is quite spot-on. And yeah, look, my final point in that debate is when companies announce layoffs due to AI, it's it's a win-win for them because number one, they're broadcasting that they're AI forward. >> Yep. >> going to boost their share price. And they're also assigning blame to AI as opposed to an over-hiring mistake or whatever it is they did. But as you mentioned earlier, and I want to go back to your earlier point, you're right. You said there's more evidence pointing to the lack of junior hiring people out of college and junior employment. And you see that. Again, we just published a story in our Open Data Science blog site. And there's actually increase in some of these companies for hiring of senior employment. But the junior one is definitely in negative territory. And so there's big evidence for that. And when I talk to startups who are just starting, two or three people, they're hiring in a much different way now. And they're kind of canaries in the coal mine. They're doing more with less people and getting big productivity gains with AI. For more mature companies, do you think it's more of like they're trying to roll out their AI initiatives and they're trying to force the current workforce to say, "Do more with AI before I'll give you another hiring wreck, for example." You think that's going on? Is the potential of AI causing lots of companies to slow junior-level hiring because of the potential of AI versus what AI can actually do? >> So, I'm going to give the caveat of not wanting to generalize, but then I'm going to generalize. Um so, >> Okay. >> I do see some of these cases. This idea that LLMs in particular LLMs pretty small subset of this broader universe of AI research and and applications, etc. But LLMs, I think the the amazing thing about LLMs is the uptake in the CEO community. The CEOs just love LLMs. They love these tools. They see all these ways that these tools can transform their organizations. So, almost to a company, you see some C-level executive being a huge champion of diffusing the use of these tools throughout the organization. And it's been this interesting sort of top-down evolution. So, we see a lot of companies pressuring or incentivizing their workers to adopt. And when I teach automated systems, we teach start with a problem. Start with a task that you feel like needs to be improved within your organization. Don't start with the technology. That's backward. But, with LLMs, they've almost [laughter] to a company started with the technology. What can this hammer What kind of nail can this hammer hit? And let's go out and search for that. So, it's been backward for a lot of organizations. And the the minority of organizations that have made really great progress with these tools, where they've started, is with a problem that's been really just a thorn in their side for a long time. It's a problem that they can quantify the value of. And if they're lucky, LLMs or another new technological breakthrough can help them solve this problem in a way that past technologies could not. And one of my favorite examples is in medicine. When you have ambient note-taking, it was a long-established that doctors hate spending all their nighttime hours catching up on notes and doing administrative work, their pajama time work. Hospitals knew this. Survey after survey, this was leading to burnout. They had tried previous technologies, like they had hired medical professionals halfway around the world to listen to recordings and annotate these notes. And now LLMs come along and help transcribe and summarize the notes for doctors. Feels like a slam dunk. What's interesting is that the early studies of these tools, although they get very high marks from doctors and seem to allow doctors to focus more on their patients in the session, it's not necessarily depends on the study, but not necessarily reducing their after-hours work that much. There are also frequent medically relevant errors in the summaries. But nonetheless, that's the type of thing that achieved really remarkable scale very quickly in an environment that affects patients and doctors, probably because it was speaking to a long-standing problem. When you just go out and search for LLMs, how can we do this to increase our sales throughput or whatever else, it's not like these tools are a solution to a long-standing problem. >> Yeah, I think that's that's very true. And also related to that, I want to talk about the research you've done from performing a task to supervising an AI system doing it. And the company I was thinking about earlier, I'm pretty sure it was Figure AI. I'm not sure if you've seen this, but they have this thing called Index, which is a kind of a gig economy marketplace. They pay people to wear these cameras on their heads and it shows them doing laundry and everything like that. And that's how they were able to release last week a very, very impressive model where a robot can go into an unseen environment and start picking up clothes, doing the laundry, and stuff like that. Yeah. Again, I only saw a video demo, but it's interesting though because a lot of complaints of that focus around basically training your replacement. And those jobs are temporary, not durable jobs. And now this transfers from the physical AI world to the digital AI world online. Because people are asking themselves, is supervising AI, is that going to be a durable form of work? Is human in the loop going to be a durable form of work? Or is it just a transition phase for 6 months or a couple of years? Or eventually, I've trained it so well it becomes reliable. >> Yeah. I'd like to take you to a factory where the factory should be running perfectly all the time. It should be fully automated, fully autonomous, sometimes relatively simple tasks. And it's not it's never. One of my favorite quotes from an automation integrator that's been very successful helping factories become more productive is there's nothing more manual than automation. That over time when we try to have machines take over certain tasks, they need constant supervision and intervention. And the challenge really is is designing that job for what the most appropriate supervision will be. Cuz I So I'll I'll step back for a second. So, I'm teaching this semester MIT graduate students in engineering a course on automated systems. And my co-teacher is with David Mindell, who's a terrific He's many things. He's a pilot, he's a venture capitalist, he's a science historian. And And he wrote this great book called Our Robots Ourselves. And yesterday we worked with the students on three concepts. So, there's humans, there's remote operation, and then there's autonomy. And we had we worked through as a class kind of like, "Okay, so what is a manual task? What's a remote task? And then what's an autonomous task?" And we had the students, "Could you come up with a fully autonomous task?" And we just couldn't do it. Cuz even tasks that are imagined as autonomous still need some kind of human tasking, some kind of supervision. And when we think in a lot of the examples of the class are from the the theater of of war, drones, you know, remotely piloted vehicles, the hope is that you would have a fully autonomous agent maybe doing surveillance or something. But no, you need tasks you need tasking and you need supervisions and you need a reward function. And all of that requires a human loop and some kind of supervision. And I would say, in any organization or task I was part of, I wouldn't want a human to be out of the loop. That isn't a desirable sense, cuz you don't want to have an autonomous agent detached from human reward functions. So, I think that sometimes we get seduced into this language of autonomy, that things will just take care of themselves, when almost always in our daily lives, things that look autonomous really aren't. >> Yeah, and then I guess that begs the question, cuz I'm very much of the opinion that human judgment is the ultimate skill in the stage of AI. But then sometimes people push back to me on that and say, "Well, what makes AI supervision, whether it's physical AI or digital AI, what makes AI supervision a skilled engaging job rather than endlessly checking a machine output?" Which is a fair question. I'm like, "Well, you're not just endlessly checking and looking at it. You're You're using your judgment to apply there." >> Well, it's what's it's what the task is. I wonder about this sometimes with I find it easier to think about this in a physical environment versus an office environment where you look at your bay of monitors. I understand that modern software engineering could be you're checking on your agents and you're redirecting them and you kind of have this team of minions. But in a manufacturing environment, we've seen a lot of jobs go from manually loading and unloading a machine, doing visual inspection quality checks, pushing a cart over to the next stage to really overseeing, looking at a monitor to understand the health check of your more automated cell. You have a robot that automatically loads and unloads. You're still doing some visual inspection, but you're also maybe reprogramming what the next job is going to be. You're looking at the overall flow of the factory and you're thinking about how to improve what you're doing. So, it's less stressful cuz you're not under the same time crunch. And it's more thoughtful. So, it's more of a job like mine where we're trying to look at a system and figure out how it can be improved. I'm biased, but I think that's an improvement from a stressful, you know, very physically taxing job. So, not for everyone. Some people actually like the physically taxing jobs and I think they're going to be plenty of them available. But the transition to a more cognitive job from a more routine one, I think is good and it often times comes with higher wages. >> That's that's a very fair point. And then, if you don't mind, I'd love to talk about AGI a bit because I'm not a huge fan of talking about AGI to be honest because I don't believe it to be like a singular point in time when AGI arrives. It's more like a multi-dimensional matrix that kind of gets diffused out. But look, people tell me, "Well, AGI, if it does arrive, that's when the labor market is going to change completely." And people talk [clears throat] about universal income and other ideas like this. Which again makes no sense to me. But what's your insight on that? Is like completely not worth thinking about or if these models do get very good, do we really need to see AGI before we see significant replacement because even if we do achieve a version of AGI and leaving aside Greg Brockman's statement last week that AGI is all right, I don't believe so. Maybe it's the beginning I don't know. I don't believe that. Do you think AGI if it's achieved in some form could see significant displacement? I see an increase in employment if anything. I don't see even that displacing that many jobs. Maybe I'm delusional, I don't know. >> No. I try as much as possible in our work at least to understand the present or the recent past, in some cases the distant distant past, rather than predict the future cuz it particularly with this that that is so hard. We don't have any precedent in the past for mass displacement related to new technologies. We've had it over time a shift in the demand for labor, increased demand for certain jobs, decreased demand for other, the elimination of entire categories of jobs. Normally that happens over long periods of time. AGI from what I can understand seems to be a marketing term that gives certain people power to shape a narrative and to inspire fear or maybe from some people excitement. I don't necessarily know what it means. I don't know what general intelligence is. Like do you have general intelligence? Do our students have general I I don't know. I've seen it defined as the ability to do just about any task as well as a human. What human? Why would we want this? So there's a bunch of questions that all come back to what's the problem that we're trying to solve. And in my work, I'm again, as we started out talking about, I'm really interested in economic competition and how we make the things that are really essential to our society and making our society more prosperous. And what's required to make our society more prosperous? Well, we need to innovate, we need to make new things and make them in more productive ways. We need to learn how to make essentials. We need to cure disease, create better medicines, we need to create tools for our defense. I'm open to various tools and tactics to make all those things better and make them in better ways, but I'm more interested in what's the best tool to solve the problem versus let's build this tool that could solve every problem, but we don't really know how to define it or have any clear vision there. So, I think that we're actually by if we directed too much toward this big ticket kind of big bang AGI effort, we would miss all the important problems that there are to solve that are more concrete. And I think actually if we get back to the difference between China's AI strategy and ours, China has a very applied view of how to use these AI far more than we are in the United States. I think we could benefit a more grounded problem-solving approach to how to use these tools. >> Well, let me reframe this idea then because this is the way I like to think about it. So, putting aside the whole AGI thing, which again is very difficult to understand what does that mean, but I think there's a more useful argument here and that is just as you said, most of these changes take time. Adoption takes time. Companies got to redesign workflows, build out the infrastructure, change regulations, but it undoubtedly AI capabilities are changing much faster than previous technologies. And look, I've been doing this stuff for 14 years and I remember the attention is all you need paper the first version of ChatGPT and all of a sudden on November 2022, everything changed overnight. Like it really did have big impacts in the area of the economy that I'm working with startups and you know, software engineering and all that kind of stuff. So, really the question is that the things you're saying that you need to focus on realistically, it's the speed of transition I think is going to be the big challenge here because if the technology is running so far ahead because in my view, that's It's been one of the big bottlenecks. It doesn't matter how fast technology moves, it'll only move at the pace people will allow it to move, right? It can only affect, like you said, like a one-person startup can implement AI much faster than a Fortune 500 company, undoubtedly, right? That's stating the obvious. So, my question is like, what happens if it speeds up more and begins changing faster than companies can keep up, like let alone workers, even the education system? >> So, I'm with you that the speed of development and let's say solving frontier math problems generates a lot of uncertainty because you're wondering, you know, what's possible if in only a few years we've seen this dramatic improvement in performance. And I guess I take comfort in two things. One is that these tools can solve all the complex math problems they want, but if we don't actually have the next generation of mathematicians to understand what the hell the tools are doing, they're not going to be very useful. And the second is that I want to imagine in practice, kind of in our daily lives, the things that we interact with that are really important to us, that we get food from the grocery store, we spend time with friends and family, a lot of us have jobs that are doing something in the real world that provides value to a community or a society, maybe we make something physical, maybe we provide a service. >> Yeah, and and and think back to the peak of deep learning hype, which I think it was 2015, and you already referenced this, but Geoffrey Hinton said, "If you're a radiologist, you're better become a plumber or something like that. The radiology jobs are all going to go away." And the opposite happens. There's tons of radiology jobs. So, last question, Ben, so you are at MIT, as we talked about, you're teaching students, and of course, you got some very accomplished students over there. But I'm sure they're concerned that they're entering a labor market that's being reshaped by AI, whatever the narrative is. When they ask you, "What should I learn and how should I prepare for a world where increasingly capable AI is going to do more of the cognitive work, what do you tell them? >> I would say we're not fully on board with some of the assumptions that the true cognitive work is going to be delegated to AI. And we have different patterns in our students of how they are interacting with these tools. So some students, this is their interface to do all their work. All their work is going through some kind of filtering process with AI tools. And then some students just don't want to touch it. They feel like it is somehow corrupting to their original academic work. And I think that's okay. I think that variety of engagement with these technologies is okay. But I think the main feeling that we want to imbue within our students is experimentation and creativity. That we want students to figure out what's the best approach to solving a given problem and where are these tools can help them and where they can't, which is how myself and colleagues have tried to help students approach new technologies in the past. And just a change in how we taught. Now we're trying to gear our teaching toward helping students learn things that they couldn't learn, that no LLM would tell them. So getting out into the world and observing things, taking them to the factory floor, taking them to talk to people to get them to share knowledge that no AI company has stolen and used to train their model. >> Yeah, I think it's excellent you're doing that. So listen, Matt, this has been fantastic and I love this conversation because the AGI debate especially is more hype than reality and it's good to talk to somebody who's had a more grounded conversation. So I think this is going to be very valuable to our audience. So thank you so much for taking the time to do this. >> Thanks for having me. Really appreciate it. >> This episode is sponsored by ODSC AI West, coming up October 27th to 29th in [music] San Francisco and virtually. And ODSC AI New York, the AI for [music] Work Summit, taking place December 2nd and 3rd in Manhattan. Learn more at ODSC.AI/West and summit.ai and use the code podcast for an exclusive 10% discount at either event. And if you enjoy the podcast, please subscribe. It helps support our small dedicated team and allows us to keep bringing you conversations with the people building and shaping what comes next in AI. Thanks for listening and we'll see you on the next episode.
Assistir ao vídeo original no YouTube
Assuntos: Agentes de IA, Robótica e humanoides, AGI. Publicado no YouTube em 2026-10-15.
Seleção automática por assunto e idioma. O vídeo pertence ao canal de origem.