The people filming their chores to train humanoid robots
Publicado no IA Dots ·
AI Signaldot-artificial_intelligencedot-humanoid_robotsidioma-enInglêsRobótica e humanoidesyt-HzZPxVKO7rE
Vídeo em Inglês, publicado pelo canal The Globe and Mail.
About this content: The summary, key points and topics below are generated automatically from the video transcript. They reflect the content, opinions, claims and predictions presented in the video and do not represent a position, validation or fact-check by IA Dots.
Video summary
AI companies are hiring workers, often in developing countries, to film themselves performing household chores and daily tasks while strapping a phone to their foreheads. These first-person videos are used to train humanoid robots to move like humans and perform physical labor. Tech and AI reporter Joe Castaldo discusses the mechanics of this data collection, the low pay and precarity for workers, and the implications of teaching robots physical labor.
Key points
- AI companies are collecting first-person video footage of humans performing household chores and tasks to train humanoid robots.
- Workers strap smartphones to their foreheads to record activities such as cleaning, folding laundry, and going to the grocery store, following strict app guidelines.
- Many contributors are low-paid workers located in developing countries like the Philippines, Indonesia, and Vietnam, participating primarily for financial reasons.
- Videos are often reviewed by automated processes and must meet specific framing and movement criteria to be approved for small financial payouts.
- Separate groups of workers in regions like Nigeria and Kenya are hired to manually label the video clips with detailed descriptions of hand movements.
- Experts and industry observers discuss the inevitability of automation and propose that workers or unions might need to capture and monetize their physical movements as intellectual property.
Topics
Humanoid RobotsAI Training DataFirst-Person VideoHousehold ChoresGig WorkLabor and PayAutomation
Video transcript
Transcript from available captions; it may contain errors.
Some AI companies are hiring people to strap a phone onto their foreheads and film themselves doing household chores. Things like vacuuming, cleaning a toilet, setting a table, even going grocery shopping. These videos are to help train humanoid robots, machines that are built to look kind of like a human with a head, torso, arms, and sometimes even legs. Just as chatbots are trained on vast amounts of information, humanoid robots need videos of humans to understand how our bodies move. But these videos are usually made by low-paid workers, often in developing countries. And by training robots to essentially move like humans, they could one day replace the work of humans. Today, Joe Castaldo is here. He's the Globe's AI and tech reporter. He tried this technology firsthand and spoke with people that are doing these tasks. He'll tell us [music] what it's like and what it means for humanity when we teach robots to do physical labor. I'm Manica Raman Welms and this is the Decibel from the Globe and Mail. Joe, thank you so much for being here. >> Thanks for having me. >> So tell me, you downloaded one of these apps called Atlas Capture. You strapped your phone to your forehead and you tried to do some of these tasks. What was this like? >> It was awkward, to say the least. Uh you you know, first you got to put on a headband, attach your iPhone, and adjust it so that your hands and your forearms are in the frame, which is tricky because when it's on your head, obviously you can't see where it's pointed. So, there's a lot of calibration you have to do it first. And then, >> You got to have to have your hands like out in front of you then almost a little bit to be in the frame. >> It makes sense to stick your hands out which means you end up kind of looking like a robot >> [laughter] >> and also the guidelines on the app tell you that you should always look at your hands and like follow your head should follow your hands again to make sure that they're in the frame so you end up moving very stiffly. It just compounds the robotic effect. You're not supposed to sit down. You're always supposed to keep moving. Nobody else can enter the frame. And what's kind of funny is before you submit a video you have to certify that you're moving naturally. But of course like you can't. Like there is no possible way you can do this and move naturally. >> Okay, so I think you had to do this like you had to do a test video and then if you pass on the app you get to do these other tasks for money. Tell me how this works. >> Yeah, anybody can download this app or others like it and then to make sure that you're kind of suitable enough to help train robots you have to pass the test first. The example that that I did was I had to clean my living room for 3 minutes and it did take me a couple tries because my hands the first time were just out of the frame most of the time. And then I think a day or two later the video was approved and then there were a whole bunch of tasks on the app that I I could do for money. I mean I experimented with it to be clear I did not accept any while I was doing this. But there was a whole range of things that you could do there. >> What what kind of tasks? >> So cleaning and household stuff is the big one so mopping, vacuuming, cleaning the bathtub, you know, taking out garbage, stuff like that. Organizing seemed to be another category so taking everything out of your kitchen cupboard, grouping items together and putting them back. There were some that weren't so much practical, but I think we're more about just capturing precise hand movements. So, one example was filming yourself taking apart an electric fan and putting it back together again. Um I checked the app this morning actually and one of the tasks was to turn on and off every single light switch in your house. And then there were just ones that were odd, like filming your entire trip to the grocery store was one. The The strangest that I saw was a task to volunteer at an old age home. So, go there, ask for permission first, and then help the staff, you know, distribute food and games and stuff like that. Um you know, and be careful not to film anybody's faces or or their medication bottles or anything like that. >> a task. >> But I just cannot imagine a world in which that is practical or advisable. Uh yet it was there. >> And Joe, I understand I think you also were folding laundry for one of these tasks as well at home. >> Yes, not very well apparently. I saw some people commenting online about my poor folding skills. But I'm just trying to sabotage the robots to protect [laughter] humanity. >> You're actually doing it for all of humanity. Thank you. >> Yes, exactly. >> [laughter] >> Okay, let's get to the the point behind all of this because there's some really strange stuff that you're you're talking about here, like you know, the intricacies of taking apart a fan and filming your your hands while doing that. Joe, why would some AI companies want people to film themselves doing doing these kind of tasks? >> So, an AI model has to learn from something. It has to learn from data. So, if you want whatever you want AI to do, you have to show it lots of examples first. So, if you think about chatbots, uh like chat GPT, um they have ingested just huge amounts of text, right? A lot of it pulls from the internet in order to figure out language, you know, definitions, how words relate to one another, how to create a response that humans will like. It's all based on data. And the same is true in robotics. So, if you want a general-purpose robot to clean your house, the AI model powering that robot needs to see lots of examples of people cleaning their houses first. Um ideally from the first-person perspective because that will also be the perspective of a robot. >> Okay, so is the point of this then to specifically develop robots like who can do these specific household tasks or is it broader than, you know, these tasks that are listed out here? >> It's It's both. So, there are companies uh like 1X Technologies, for example, it's based in Silicon Valley. It is building a humanoid robot for your house. It's like a domestic helper robot that is supposed to do things like tidy up, put away dishes, fold your laundry. Uh so, that is one avenue, but also part of it is is just to build like a very capable foundation model for robots. >> What does that mean? >> So, if you have a model that can do lots of things, you can then sort of train it on a a little bit more data for something specific. So, let's say there's a very capable model that's been trained on household chores. So, the robot is fairly capable, but you want to take that robot and put it into a factory to do a you know, a few specific jobs. So, you could collect data uh from the factory and fine-tune that model on that data so that robot can then work in the factory. And the whole point of having like a capable model to start with is you won't need that much more data to get it to do something new, something that it hasn't been trained on. >> Cuz it's interesting cuz we have robots now who like work in a a car factory or something, right? But they're doing it like a very specific repeated task, but it sounds like, you know, if we're looking at different people in different environments, they're being trained on kind of a a broader range of things then. >> Yeah, it's a good point. So, if you think about your your typical robotic arm in a car plant or something, it's not necessarily a sophisticated AI model that's powering it. It could be, you know, hardcoded or programmed to do that same task over and over again in the same environment. Uh when you talk about a general purpose human humanoid robot, like it has to do tasks in different environments. Like think, you know, my home does not look like your home, for example. So, it needs to be able to figure out, you know, this is what a this is all the different variations of what a mug looks like, and this is all the different variations of what a table looks like, and what it means to move a mug to a table. So, that's where AI comes in, right? It has to help the robot interpret the world and what is in it, and figure out what to do next. >> Mhm. >> So, how many of these first-person videos are needed then to to train this kind of humanoid robot to understand what a mug looks like in every situation? >> Well, a lot. So, um like thousands and thousands of hours of video. So, a couple years ago, um there's a company called Physical Intelligence uh out of California, and they built an AI model for robots that was trained on, I think, about 10,000 hours of video. Um just back in August, there's another company called Dyna Robotics that made one based on a million hours of video. And someone told me next year companies are looking at 10 million hours of this first-person video. So that's the equivalent of like over a thousand years worth of footage. And like what researchers have found is the more first-person or egocentric video as it's called, the more of that you use train an AI model, the more accurate the model will be, the better performance of the robot down the line. So that explains why there is this rush on one to collect video. >> Okay, so the scale of this is is really huge. They need a lot of people shooting a lot of video then. And we're going to talk about the people who are making these videos in a moment here, Joe. But first let's just talk about the companies behind these apps. What do we know about these companies? >> Yeah, there's there are startups first of all that are just getting into this space. So one that I wrote about in the story is called Mecca. It's Canadian, also has offices in the US. It was only incorporated last year and it appears to be growing really quickly. They've said they're on track for something like a $300 revenue run rate by the end of the year. And you know, that's over a very short time period. There are more established companies. There's one called Scale AI. Meta owns a big chunk of it. It's like a more established AI training company. Robotics is one thing that it it does. And then robotics companies themselves also collect this data. There's a company called Figure out of the US that is making a humanoid robot and it has its own data collection platform. So there's there's a variety of approaches. >> Do we know then what the goal is for these companies in the end? Like why they're they're doing this with trying to get people to to capture all this video? >> Well, the the video is kind of the root of everything in a way in terms of building an AI model for a robot. So traditionally what companies have tried to do is is build an AI model through teleoperation. So, that's literally where you have somebody control a pair of robotic arms and to do, you know, a bunch of tasks. And that gets you really high-quality data, perhaps from multiple camera angles, from motion sensors, stuff like that. So, that's that's valuable. But the the problem is it's time-consuming and it's expensive, right? You need a bunch of robotic arm rigs. You need to pay people to do it. It's operationally complex. So, that's why companies, as a complement, have turned to first-person video because it's abundant, it's cheap, it's stuff people are doing in their daily lives anyway. And so, if you get a lot of that and then you add some like a smaller amount of teleoperations data, researchers found that that gets you pretty far to having a capable AI model. >> [music] >> We'll be right back. All right. So, Joe, we've talked about the companies behind these apps and what they're trying to do here. Now, let's talk about the people who are actually making these videos. So, these are the people who are strapping their phones to their heads like you did and completing these basic household tasks with their hands in the frame. So, who are the people who are actually doing these videos? >> Yeah, it's not surprising that a lot of the people who sign up to these platforms are living in lower-income countries. From what I could tell, there are a lot of people in the Philippines, Vietnam, and Indonesia who sign up for these platforms. And one executive I spoke to said the Philippines is kind of a sweet spot for this because it's wealthy enough for a lot of people living there to own a high-quality smartphone to collect data in the first place, but it's not so wealthy that they might turn up their noses at the pay for doing this. >> And how much would they be getting paid? >> So for some companies, they pay $3 an hour for footage that is accepted. So if you film an hour of footage, you get $3 for it. And so you have to be doing a lot of chores for this to be worthwhile. >> Yeah, that's not that's not a lot of money and it is, you know, pretty time-consuming it sounds like from from what you're saying. You are doing tasks for a good stretch of time. Do we know how many people are actually making these videos? >> So companies say they have tens of thousands of contributors on their platform. Now that might not be, you know, active, but it could be cumulative, but I think it's fair to say that there are thousands and thousands of people who have contributed video. >> And Joe, I know that you spoke to a number of people who are making these videos. What did they tell you about why they do this work? >> Purely for financial reasons. >> [clears throat] >> Nobody >> who don't have a lot of other options then? >> Yeah, exactly. Nobody was doing it because they were really interested in AI and, you know, necessarily wanted to help train a robot. They they just needed money. Some had jobs that didn't pay well. Some couldn't find work. They just saw it as an economic opportunity. But even then, you know, people I spoke to were were questioning whether it was really worth their time. They did have frustrations with it. For example, you know, if you could spend a lot of time filming a video, but if it doesn't meet the guidelines on the app, it could be rejected or part of it could be rejected. And and you don't get paid. So again, like if your hands were out of the frame too much, if it was too shaky, there's some kind of automated review process and if that determines that you weren't actually doing the chore, if you were faking it, for example, or they they deemed you to be faking it, your video could rejected could get rejected for that reason. So, that was a common complaint. There were other complaints about payments being delayed at times as well. What happened with Atlas Capture is sometime in August, the company implemented a limit of 50 hours. So, once you've submitted 50 hours of footage, that was it. You couldn't contribute to the platform anymore and so some people were surprised by that because they had been earning some money and then now all of a sudden they couldn't do that anymore. >> Yeah, it sounds like it it could be kind of difficult, right? You end up doing lots of work and if your video doesn't get accepted, you're not getting any money for that. I imagine that's a kind of a precarious situation for people like to to not know if, you know, the work that you're doing is actually going to give you some income. >> Yeah, it's uncertain. There's kind of no like formal working relationship. These are not independent contractors, for example, that are working with the company. It's it's literally, you know, sign up, yes, you sign a privacy policy, you submit a video and you can get paid for it. It's it's it's not even I wouldn't even call it gig work. It's more crowd sourcing in a way. But, people are depending on this for part of their income. >> So, Joe, we've been talking about people who are making these videos, doing that filming, but there's also another group of people who are hired to label videos like this. Explain that to me. What are what are they doing? >> Yeah, so Atlas Capture, for example, maintained sort of a separate labeling business. So, for this video to be more useful for AI training, it has to be paired with language. So, there are people who write descriptions of every movement and every part of every video. So, you know, pick up screwdriver with left hand, for example, or something like that. And people who sign up to do that, from what I could tell, seem to be based in Nigeria and Kenya. And the way it works is they're paid by the clip. So, if they label a clip that's, you know, a minute and 30 seconds, 2 minutes, they might get 50 cents for doing that. And how long it takes to label really depends on how complex the movements are in the video, how many movements there are to label. Um people told me it could take them anywhere from 7 minutes to 30 minutes to label a clip. Um so, the pay is not uh like predictable. And the other issue is they have to follow guidelines to make sure they're labeling everything correctly. And if they make a mistake, if they deviate from the guidelines, it hurts their quality score. Everybody has uh like an accuracy score. And if that dips below 70%, then the company told laborers they won't be compensated until they get their quality score uh quality score back up. But companies are also exploring using AI models to label video, so that they don't have to uh pay human laborers anymore. It could be cheaper and more efficient to uh do that with AI, although they are still struggling with accuracy. Uh but it it does seem like some companies could move in that direction. >> Mhm. This all sounds similar, Joe, to how humans have helped train other forms of AI, like chatbots, right? Like this This kind of work seems to use these these low-paid workers to to train these systems. Why do AI companies seem to rely on this form of human labor in order to train their systems? >> Yeah, there's there's a history of it in AI and and tech. Um like think about self-driving cars. Um all that road footage and data, like it had to be labeled at some point. You needed to pay people to draw boxes around, you know, a pedestrian in the scene and label that as a pedestrian and label that as a fire hydrant. Um and you have to do that at scale. Like there's just a lot of a lot of data and uh it's costly. And so they have moved to lower cost jurisdictions like Nigeria and Kenya um where they can do that. Uh it's the same with chatbots. Like the way a chatbot is trained, you know, people have been paid to rate the responses of chatbots, right? To teach it what what humans like or fact-check or edit responses for accuracy. Um and again, that that work has been in sort of lower-income jurisdictions. And it's changing now that we want uh AI models with specialized knowledge about say law or biology. So if you have a a law degree, you can sign up to one of these um AI training companies and actually get paid at like a a very uh generous hourly rate to help teach AI models about law or whatever your your specialized discipline is. >> Oh. Huh. So we're seeing this for other forms of labor, too? Like I guess I wonder we're talking about household chores primarily with these these robots that we're talking about today. Um but I wonder about other jobs where people are also being asked to help train AI. Is this something kind of in in other uh in other jobs as well, then? >> Yeah, so for robotics specifically, we've been talking about um collecting like household chore data. But some of these companies, uh like Atlas Capture for example, they will partner with businesses. Uh say restaurant workers or baristas, that's skilled trades, mechanics, Um to get workers there to wear cameras actually on the job and collect data that way. And you know, they'll pay the company for this and the people doing the recording can also get, you know, a little extra income by recording themselves. Which is like, you know, on the surface maybe a bit uncomfortable. Like are these people, you know, recording themselves to train robots that will one day put them out of work? It it's actually hard to say. So, like there could be a company out there trying to build a very capable carpenter robot that, you know, will build houses or whatever someday. But it could also just be that they they just need that data just to get more data on hands and how they move just to build that capable foundation model again that can do lots lots of things. >> But I'm sure this is something that people are thinking about though, right? Because it especially if you say, you know, you know, putting cameras on baristas and seeing how they do their job, like it does seem like people are training robots to do the job of humans. The people that you talked to Joe, like how do they contend with that aspect of this? >> So, one of the most interesting conversation I had for this story was with a guy named Daniel Wigdor who runs a venture studio focused on AI in Toronto, like launching AI companies. And he was making the point that like all of this like forces us to rethink the value of of ourselves in a way. Like we haven't really considered that our physical movement, how we interact with the world, is valuable. But it is. Like companies want this data. And so he was making the point that like if you're in the skilled trades, your your physical movements are part of your intellectual property. And you should be able to capture that somehow. And so he's made the proposal to some unions that they should be equipping some members with cameras to record themselves on the job and his company will partner with the labor union to form a company that will build the AI model and the workers will have equity in that company and that's a way for them to capitalize on their value. And they might think well that's crazy. Like they would why would a union do that to their members? And his argument is first of all like you know automation is kind of inevitable. Like this is already First of all like you know automation is kind of inevitable. Like this is already happening as we know. They're already companies collecting data from workplaces to build AI models for robots and if unions don't get involved and take ownership then you know perhaps they're just rendered completely irrelevant someday. In his argument it's it's short-sighted to try to stop robotics. It's like there's a way for unions to participate and have ownership and actually control how robots enter work sites someday and this is one way of doing it. So it might still sound crazy but I think it sounds a little less crazy when you consider that. >> [music] >> An interesting place I think to end this conversation Joe. Thank you so much for taking the time to be here today. >> Anytime. >> [music] >> That was Joe Castaldo the Globe's AI and tech reporter. >> [music] >> That's it for today. I'm Manica Ramon Wilms. Our producers are Madeline White Cheryl Sutherland [music] and Rachel Levy McLaughlin. Our editor is David Crosby. >> [music] >> Adrian Chung is our senior producer and Angela Pacenza is our executive editor. Thanks so much for listening and I'll talk to [music] you tomorrow. >> Mhm.
Assistir ao vídeo original no YouTube
Assuntos: Robótica e humanoides. Publicado no YouTube em 2026-10-20.
Seleção automática por assunto e idioma. O vídeo pertence ao canal de origem.