How to build a JARVIS AI Voice Agent for FREE | Full Livekit Tutorial (2026)
Publicado no IA Dots ·
Agentes de IAAI SignalAssistente pessoaldot-ai_agentsdot-artificial_intelligenceidioma-enInglêsyt-Fhm8goKN6rM
Vídeo em Inglês, publicado pelo canal Thanh-y David Nguyen.
About this content: The summary, key points and topics below are generated automatically from the video transcript. They reflect the content, opinions, claims and predictions presented in the video and do not represent a position, validation or fact-check by IA Dots.
Video summary
O vídeo apresenta um tutorial passo a passo sobre como criar um assistente de voz com inteligência artificial chamado Jarvis, utilizando Python, o framework de código aberto LiveKit e modelos de linguagem como o Gemini. O autor demonstra a configuração do ambiente, a personalização do comportamento do agente através de prompts, a integração de ferramentas como busca na web e controle do navegador via Playwright, além da criação de uma interface web e a implantação do aplicativo em dispositivos Android e iOS.
Key points
- Explicação sobre o uso do LiveKit, um framework de código aberto em Python para a construção de agentes de voz, utilizando a versão em nuvem gratuita.
- Instalação dos pré-requisitos necessários no ambiente de desenvolvimento, incluindo Python, Node.js, VS Code e as ferramentas CLI do LiveKit.
- Configuração do modelo de linguagem Gemini e do prompt de sistema para dar ao assistente uma personalidade inspirada no personagem Jarvis.
- Criação de ferramentas customizadas para permitir que o agente realize buscas na web e controle o navegador utilizando a biblioteca Playwright.
- Desenvolvimento e personalização de uma interface de usuário local para interagir com o agente e compartilhar a tela.
- Implantação do assistente de voz como um aplicativo móvel para Android e iOS utilizando o Flutter.
Topics
LiveKitPythonAssistente de VozGeminiInteligência ArtificialDesenvolvimento de AplicativosAutomaçãoFerramentas de Código
Video transcript
Transcript from available captions; it may contain errors.
In this tutorial, I'm going to show you how to create your own Jarvis AI voice agent for free using Python and an open- source to code life kit. Hey Jarvis, can you see this character that I'm holding? Yes sir, I can see it. It appears to be Son Goku from Dragon Ball Z looking characteristically Bellicose. [music] Shall I find you somewhere to display it or perhaps more of them? >> Hey Jarvis, you there? >> At your service, sir. >> What is the weather in Vienna? It is apparently mostly cloudy in Vienna, Austria with a temperature of 28° C. [music] Could you open Google Maps for me and show me the fastest way to get from Vienna to Berlin? [music] >> [music] >> I have opened Google Maps, entered Vienna as the starting point and Berlin as the destination. The fastest routes should be visible on the screen now. >> Can you show me the flight options? >> The flight from Vienna to Berlin takes about 1 hour and 10 minutes with Austrian, Euro Wings, or Lufansza. >> Could you play some iconic Iron Man music? Naturally, sir, I [music] have commenced playing that most iconic of themes. Shall I prepare some far too advanced technology for your consideration as well? And on top of that, I'm also going to show you how to launch Jarvis as [music] an Android as well as iOS app, so you can talk to them wherever you want. Isn't that right, Jarvis? >> Yes, indeed, sir. Though, I must say your intros are getting a bit repetitive. [music] All right, then let's try something else. Can you see me filming this intro right now? [music] >> Yes, sir. I can see your camera setup and lighting stand. [music] I looks quite professional for a washedup YouTuber. >> What the hell, bro? I actually already did the same video about a year ago, but since then, there have been a lot of [music] changes to LifeKit. So, here's the updated version for you. So, let's get started. So, the first thing you need to look at is something called LifeKit. For everyone who's not a regular viewer on this channel, LifeKit is an open-source AI framework that you can use to build voice agents. And it is really sophisticated. Even very big companies like OpenAI use it and they integrated into Chat GBT even. And since it's open source, you can actually use it completely for free. For example, if you run it locally, for this tutorial, however, we're going to use the cloud version. You can also use the cloud version for free. and it comes with up to 1,000 minutes a month, which equates to 16 hours a month, which should be more than enough for most users. For the sake of this tutorial, we're simply going to use the cloud version because it's easier to set up. So, first off, we would need to click this button over here that says start building. And then over here, you need to create an account. You can log in with your Google account. For this, you can click on continue with Google. So, let me select my Google account real quick. Here we need to fill out this form. You can actually type in whatever you want. For example, AI agents, I don't know, YouTube here. Other continue. And here's where we need to create our first project. We also have to give this a name. Let's call this star new or something. And at agent observability, we can disable this for now. And then click on continue. Here we can click on skip for now. And now you have access to this dashboard. And over here you can also see that you created your first project here. Now we need to open the livekit quick start manual. And for this you can simply Google live kit voice agent quick start and then click on the first link. And here's a detailed manual how to set up your first voice agent in less than 10 minutes. And if we scroll down a little bit we can see here a couple of requirements we have to meet before we can start. And first off, we would need to install Python. And for this, you can simply Google Python install. Then click here on download Python. And here you can click on this button to download and install the newest version of Python. You would also need to install something called UV. And for this, you can click on this link over here. And here, if you're using Windows, then you can use this command to install it. If you use Mac OS or Linux, then you can just follow this installation guide. For now, I'm going to copy this. And now we need to open a terminal in this case a PowerShell terminal. So you can just simply type in PowerShell and then paste it in here and then execute that. After a few seconds this should be installed. Another thing which we also need to install is something called NodeJS. This will be important later for our UI. Again for this you can simply Google download NodeJS and then go to their website and download and install it for your operating system. And before we continue I would also recommend you to download and install VS code as our IDE. And for this you can simply go ahead and also Google VS code download. Of course you can also use any other IDE of your choice for example cursor. But for now, we can click here on download VS Code. And then you can simply select your operating system and also download and install it. After that, you can go ahead and open VS Code. And then the first thing you need to do is open a folder where we want our project to be at. And for this, we can click on file, open folder. And then we can select any folder. I'm going to use this one here. And then click on select folder. And then we can continue with the manual. For this, we can simply go back and then go to the next step. And here we need to install something called LifeKit CLI. And for this you can again select your operating system and then copy this command here to install it. And then we can go back to VS code. And then we need to open a new terminal. And for this you can press control shiftp or command shiftp if you're using Mac. And then you can click on create new terminal. And I'm going to use PowerShell. And down here now you can simply paste this command. And then you can press enter to install it. I'm not going to do that now because I have it already installed. After the installation is finished, we can now go back to the manual again. And here we need to copy this command. And then we go back to VS Code again. And then we need to enter this command. Here we need to give this device a name. I can call this JR tutorial. Press enter. And now it opens this window where we have to select the project that we created in the beginning. And mine was Jarvis new. And then we click on allow access. Now we can go back to VS Code. And down here would ask you if you want to make this project your default project. You can select yes. And then we can go back to the manual. Here we can copy the next command which is this one here. And then we can go back to VS Code. And then we can paste it here again. This command will download some files for us and also save it into a new folder. And here we can change the name of the folder. Instead of my agent, we can say Jarvis. And then we can press enter. We can say yes here. Over here. Now you can see there's a new folder. Here it also asks you if you want to install the dependencies. We can also press enter to say yes. Now it's done. If you get this error, you can simply ignore this for now. And if you click here now, you can see all the content of this folder or all the files for your voice agents. And if you want to try this out now, we can simply go ahead and use the following command to go into this folder, which is cd Jarvis. And then we can use the following command to start this up which is LK agent defaf. Now it's up and running. And now you can open this link by either pressing control or command and then click on it. Then we can click on open. Then we can click on start a session. >> Hi, can you hear me? >> Yes, I can hear you loud and clear. How can I help you today? >> What can you do? I can help you answer questions, explain complex topics, or get things done using my tools. Whether you need a quick fact or a hand with a task, I'm here to make it easier. Is there something specific you're looking for right now? All right, so now we have a working agent. So the next thing we need to work on is something called the system prompt. And for this, we can go to this folder that says source and then click on agent.py. This is basically the main code of the entire agent. And if you scroll down a little bit, you can see here that there's a section that says instructions. And here you can find the so-called system prompt. This is basically a very detailed instruction how the agent should operate. And if you want to change its behavior, you can simply write it in here with natural language. In our case, we want our Jarvis to be able to first of all know that its name is Jarvis and also behave like a classic butler. For this, I already prepared a change system prompt. So let me quickly paste it in here for you. And this is basically just the same system prompt that we had before with a few tweaks. For example, here the first sentence says, "You are Jarvis, a helpful sarcastic AI butler." Down here, I tell it to address me as sir. And additionally, I also added some conversational examples that for example, if I say, "Jarvis, can you do XYZ for me?" Then it should answer with, "Of course, sir, as you wish. I will do XYZ." Now, so again, if you want to add any specific behavior to your AI agent or you don't want it to do certain things, then you can simply add new rules in here and you can simply describe it in natural language. For now, we can go ahead and save this by pressing control or command S. By the way, the code of this entire project is available in the video description down below, including this prompt as well. Now, we want to switch our large language model to Gemini. And for this we can go ahead and Google LiveKit Gemini and then click on the first link. And here we have another installation instruction. And for this we can copy this command over here. Then go back to the terminal in VS Code. And make sure that you're in your agent folder or in my case the Javis folder and then paste it in here and press enter. Now this is installed and now we have to add some imports here in order to use this here at live kit plugins. You can now add Google and down here we now have to replace this section with a code snippet that I already prepared which is this one here. You can pause this if you want to type this yourself. And now this allows us to use Gemini instead of the large language model that we had before, which is this model here. The reason that we're using Gemini is because it's free to use. Quick pause here because a couple of people in my last video seem to be confused if Gemini is actually free to use or not. And yes, this particular model that we are using is free to use. It is Gemini 3.1 Flash live preview. Here you can see my usage of that particular model. Here it says Gemini 3 flash. Yes, this includes 3.1 as well. Over here you can see the limits that I have with this model. Here it says requests per minute which is unlimited. So I can have unlimited requests per minute as I want. And the same goes for requests per day which is also unlimited. The only limit I have is something called tokens per minute which is at 65,000 tokens a minute. Here you can see that I'm only using 15,000 tokens per minute for my Javis voice agent. So realistically speaking, the only time I would hit this limit is if I'm running four or more voice agents at the same time. And even if I would, it would simply not work for some time and then I would just simply have to wait a little bit and then I can use it again. So I still pay nothing. The only time you would pay for this is if you want to increase this limit here. So yes, you can use this completely for free. And I hope this clears up any confusion. So now back to the video. And it also has very low latency because we are using something called a realtime model. Also here selected the male voice that is close to Jarvis from the movies. And here I also gave it the British accent. This section here also allows you to change the language in case you want to use another language. Now for this to work, we also need to create a Gemini API key. And for this you can go back and Google Gemini API key. and then click here on get a Gemini API key and then click on this button that says create or view Gemini API key and here's where you would need to login with your Google account and if you don't have one then just create one and over here you can click on create API key and then you would need to create a project if you don't have one already and then you can simply give this a name for your project and then click on create project I already have one so I'm going to sele Select test and then we can also give this API key a name. Let's say Javis 2 or something and then click on create key. Now we can copy the API key over here and then close this window and then we can go back to VS code and then we can go to this file that says env.lo local. And now here we need to add this new API key by typing Google API key and then you need to paste it in here. And then we can press Ctrl S or command S again to save this. We can go back to the agent.py PI file. And then we can scroll down a little bit. And here where it says ST and TTS, we can actually remove these lines and then press Ctrl S again to save this. Next, we also want to enable our Jarvis to be able to see us through the camera. And there's actually a very easy option that we can simply toggle on for this to work. And for this, you can scroll down where it says room options. And then we need to add a new line here which says video input. And then we set this to true. And that's actually it. And then we can save this again. Another thing which we also have to add is a new dependency. And for this we can go back to the terminal and then write uv lifekit agents images. And now we can press enter to install this. Now that's installed and now we can try this out again with the same command as before which is lk agentdev. And always make sure that you're in your agent folder. In my case the Jarvis folder and then we can press enter again. Now that's up and running. And then we can try this out again. Hey Jo, is you there? >> At your service, sir. What monumental task awaits my attention today? >> Can you see me through the camera? >> Indeed, sir. I see you are wearing a dark colored shirt and appear to be speaking into a microphone. Is there something specific you require assistance with regarding this? Another cool thing that this model is actually capable of is multilingual functionality. So it means that it can speak multiple languages at the same time. And we can try this out right away actually. And there we go. Now we have our updated Jarvis voice agent. There's one limitation this model has however and that is I cannot use this chat function over here. So let's say I would write a message in here and then I would send it. It would simply not work. But given the fact that this is free and it comes with camera vision as well as multilingual functionality right out of the box, I would actually say that's a pretty good trade-off. So the next thing you want to add to Jarvis is the ability to perform tasks for us. And for this we need to create tools for it to use. For example, let's say you want to give it the ability to search things on the internet for us. And for this we need to add a new dependency like before. For this you can go back to the terminal and then you can write UV add link chain dash community and then ddgs and then press enter to install this. Now it's installed and now we can add a new file in here. And for this you can click here on this source folder and then you can click here on new file and then you can call this tools.py. And here we now have to add some new imports which are already prepared. The first one is logging which we are going to use in order to log what is happening in the background. The second one is for our function tool or the tool that we're going to create for which we need function tool and also run context. And the third one is from Langchain which we just installed where we are going to use something called duck go search run which is a search engine similar to Google but this is free that's why I'm using it. Now we need to create a tool that allows us to search the web and for this we need to create an asynchronous function. So we can write a sync define search web and then we need to pass a so-called run context. So we need to write context run context. What you see here is GitHub copilot that is recommending me some code so I can code faster. But I'm not going to use most of this right now. So I'm going to ignore this. The next thing we need to pass is the query, the search query. So what we're searching for. So we can write query here. Now we need to give this a description. So the AI agent actually knows what this function is about and or when to use it. So you can write something like use this tool to search the web for information related to the given query. Then we need to write try results equals duck go search run. Then we need to pass the query here. So basically we are passing the search query to the search engine and then we're getting a result back. But I also would like to log this so we know what the result was. And then we need to give this or return this back to the AI agent. And then we also need to handle the case where this didn't work. And that's actually it. And the last thing we need to add here is a so-called annotation. So the AI agent actually knows that it can use this as a function tool. So we can write add function tool and that's actually it. And now we can save this. Now we need to go back to the agent.py file and then we have to import from tools the search web tool. And then down here we have to write tools equals search web tool. And now the agent has access to this tool. And now in the system prompt, we also have to add a new line in here that tells the AI agent when to use this tool. So we can write tools use the search web tool if the user asks you to search for information. And that's it. Now I can save this again. And now we can try this out again. For this we can go back to the terminal but this time we can use a different command to run this locally instead of using the web app every time which is going to be uvun source agent.py console. Hey Jav is there at your service sir. What manner of assistance do you require? Assuming your needs aren't merely philosophical. Can you tell me the current weather in Vienna? >> It appears Vienna is currently experiencing passing clouds and a pleasant 75° F. One might almost be tempted to go outside, provided one enjoys that sort of thing. Is there anything else I can conjure up for you? We can end this conversation by pressing Ctrl C. All right. So if we scroll up here, we can actually see that the agent actually used the switch web tool. And here we can see the result of that. So here you can see that it actually found out that the weather is or the current temperature is 75° F. Here you can also see it in degrees C. So the way this works is as I said before, the agent knows when to use this tool based on the description here. It also knows based on the name of this variable what to pass here. In this case, it's the search query. Then we simply pass this to the search engine. Then we simply print this onto the console. That's why we were able to see the search result. And then we can give this to the agent back. So you can simply formulate a fitting answer to our question. So basically, if you want to add any sort of functionality to your voice agent, all you need to do is to create new function tools for it to use. then you would need to give it a fitting description as well as fitting names for the passing parameters and then also come up with the business logic. So instead of coding yourself, I'm now going to show you how to use a coding agent because lately they have become very powerful and this is probably the future of developing anyways. And in this case, I'm going to show you how to use open code. And for this you can simply Google open code and then click on the first link. If you don't know what open code is, it is basically like cloud code but this is fully open source and it also comes with free models. So you can use it completely for free. Of course you can also use clot code or any other coding agent like codeex or even github copilot which you can use directly in VS code. But as I said this is completely free. So that's why I'm using it. And for this you can go ahead and use this command over here and simply copy it and paste it into a console again to install this. So you can back go back into VS Code and simply paste it in here and then execute that. If it doesn't work for you and you get an error like me here, then you can use a different command in order to install this, which is this npm one over here. If you installed NodeJS, as I told you in the beginning that this should work as well. So we can copy this and go back to VS Code and simply paste this. And now it's done. And now we can start this with the following command which is open code. And as you can see, this is how it looks like right from the get- go. This works out of the box. And you can even check this by typing /models. And here we can see the model that we currently have selected for this coding agent. And here you can see it's big picle. And it says it's free as well. Now we can actually go ahead and give our coding agent additional knowledge. So it knows exactly how to build with life kit. And for this we have to give it a so-called MCP server. And for this we can simply Google live kit MCP and then click on the first link that says docs MCP server. If you don't know what an MCP server is, it is basically a remote server that allows our agent to have access to external tools similar to the tool that we created for our Jarvis agent. But in this case, it's a tool to access the documentation of LifeKit. So basically if we add this MCP server to our coding agent, it will have access to the entire documentation of life kit and it will know exactly how to build with it and how to create anything that we want. For this we also need to look up how to set up an MCP server for open code. And for this you can also simply Google open code MCP and then click on this first link. And here we have a guide how to set up any MCP server. Here it says we need to create an open code.json JSON file to add the MCP configuration. So what [clears throat] we can do is that we can actually point our coding agent to these two documentations in order to set up the MCP connection by itself. I know that sounds quite complicated but trust me this is going to make a lot of sense in a bit. For this you can go back to open code and then say here's a guide to set up an MCP server. Create an open code.json file and add this MCP server from lifekit. And for this now I can simply copy these links over here. So I can copy this link of this manual, go back to open code and then paste it in here and then do the same for the live kit MCP server. Paste it in here and then simply execute that. Now it's done. And now you can see that over here it added this open code.json file. And in here you can see that it added the LifeKit MCP server. Now if you go ahead and type in /mcp you can actually see the connected MCP servers and right now we cannot see our livekit MCP server because we have to close the session and then reopen it to refresh this. So let me do this real quick over here can simply click on this button over here to kill the terminal and then we can reopen one control shiftp create new terminal powershell and then we can write open code again. And then we can type in / session to see our older sessions. And then we can select this one that says open code livekit MCP server setup. And then we can write /mcp again. And here you can see that the lifekit docs MCP server is connected. So now if I go ahead and ask our agent, do you know how to add new tools to Jarvis through the lifekit documentation? It knows exactly how to do it and answered yes. Since a lifekit docs MCP is now configured, you can explore it for adding new tools to Jarvis, a lifekit voice agent. The relevant concept is tool calling and function tools. By the way, you can also do this exact process with any other coding agent like cloth code or codeex as well. Now we can go ahead and add the functionality to our Jarvis voice agent to control the browser for us. And for this we can use something called playright which is basically a library or a tool set directly from Google that gives us full control over a chrome browser. For this of course we can go ahead and ask open code to implement it for us. But before jumping right into this I would actually go ahead and ask it to create a plan of implementation for us first so we can review it first. And for this we can go ahead and type in into the chat slash agents. And here we can switch between build and plan. Build is something that we already selected which is an agent or mode that will immediately go ahead and implement whatever we're asking it to do. And the plan agent or plan mode is not going to change anything yet and just creating a plan how it would implement something. So we can select plan and press enter. For this we can ask create me a plan to add the entire functionality of playright to the Javis voice agent through additional tools that gives Javis full control over a browser and then we can press enter to execute that. Now it's done and now I can review this plan. First you can see over here that it wants to install the dependencies to use playride which is correct. Then it also wants to create a separate browser.py file in order to create all of the tools. And here you can see the tools that it wants to create. For example, open browser, close browser, navigate, go back, click element, and so on and so forth, which also all sounds correct. And then it also wants to update the agent.py file. So it actually adds all the tools to the agent so it can use it, which is also correct. And also it wants to update the system prompt and add this new section that actually tells the AI agent how to use the tools and so on and so forth. Down here, it also actually asks us if you want to have two separate tools to open a new tab and switch between tabs or if you want to have it combined into one. It actually recommends us to have it in two separate tools. For this, we can now switch back to the build agent. And then we can say go ahead and build this. Create two separate tools as you recommended. Now it's done. And over here you can see now that it added this new file that's called browser.py. If you click on that, we can see that it added all of these new tools in here. For example, here open browser also close browser tool and so on and so forth. And in the agent.py file, we can also see that it added a new section in the system prompt how to use the browser tool in here. And now we can go ahead and already try this out. And for this we can simply open a new terminal. Again we can press control shift t create a new terminal powershell and then I can go into the jar folder and then use this command again to start this up which is lk agent def. Now it's up and running again. And by the way here on the right hand side you can actually switch between terminals. And now we can open this again. Hey Jav, you there? At your service, sir. What can I do for you? >> Can you actually see me through the camera and tell me what I'm wearing? >> It appears you are wearing a white t-shirt. Sir, how very minimalist of you. Anything else I can observe for you? >> Yeah, go ahead and open Dr. Go search and search for what I'm wearing. >> As you wish, sir. I've searched for white t-shirt on duck.go. >> Can you click on the first link with Amazon? >> As you wish, sir. I've navigated to Amazon to view white t-shirts. Shall I browse through the options or is there a particular style you seek? >> Can you go back? >> Certainly, sir. I've returned to the duck.go search results. What would you have me do now? >> Can you pull up Google? >> As you wish, sir. Google is now open. Shall I search for something or perhaps navigate to another page? Can you actually now open Google Maps and show me the fastest way to get from Times Square to Central Park? I have found the direction, sir. It appears the fastest way is typically by walking, which should take you around 15 minutes. Would you like me to explore other travel methods? >> No. Show me a public transport. as you wish. Public transport should take approximately 10 minutes. Are there any particular transit lines you prefer? All right, I'd say that already worked pretty good. Now, as you could tell, that already worked very well. And you might be wondering if it's so easy to use a coding agent, why didn't we do that from the get- go? And there are two reasons. First of all, we needed to do the setup manually anyway. for example, the account creation with passwords and all of that stuff. And second of all, it's still very important to know roughly how everything works and how the architecture looks like because for one, we can then guide the agent correctly. We can also still understand and review everything that it plans to do. And second of all, the better our instructions are, the better the output will also be of our coding agent. For example, I was able to tell that it needs to create new tools and also adjust the system prompt and also install dependencies and so on and so forth. By the way, if your agent doesn't produce the same level of output that mine does, then you can again download the whole project in the video description down below. And with this setup, it's actually very easy to add some new features going forward. All you need to do is just to ask the coding agent. Now, we can add a customizable UI to our coding agent that can run locally. So, we don't have to use the browser app all the time. And for this, we can use a template app that comes from LifeKit itself. And for this, we can go ahead and also Google LifeKit front end. And then click on this link. And down here we can now click on starter apps. And if you scroll down a little bit here you have a selection of apps that you can use. Here we can click on next.js voice agent. And if you scroll down a little bit here's an instruction how to set this up. And down here there's also the GitHub repository of this front end. And here's the entire project in GitHub. But instead of downloading and setting this up manually, we can actually ask the coding agent to do it for us. And for this we can simply go ahead and copy this link over here and then go back to VS Code and then ask the coding agent can you download this front end into this project into the Javar folder and call the new folder front end and install everything. And here I pasted the link to the GitHub repository and now we can simply execute that. Now it asks me if it's allowed to read my env.local file where I have all of my API keys saved. And I'm going to click on allow once. Now it's actually done. And now it also tells us how to run this. You can see it asks us to run this first. So we can connect to our voice agent. And then we need to go into the front end folder and then use this command to start this front end up. Now there's a good chance that if you don't have certain things installed that will ask you to install those too. For example, GitHub CLI which you need in order to download stuff on GitHub or for example also PNPM. But usually it will tell you that it will do that or it will ask for your permission to do it. Over here you can now see that we have this new front end folder and in here is the entire code of the front end. And now we can just follow the instructions of the coding agent. And for this we can open a new terminal for the front end. First we need to use this command over here to start up our voice agent. So we can copy this and then paste it in one terminal. Now it's up and running and now we can start up the front end. For this we can follow the instruction of open code again. For this we need to go into the front end folder and then use this command to start it up. So first I need to go into the Jarvis folder then into front end and then I can either use PNP PNPM def or if you don't have PNPM installed you can also alternatively use npm rundev. Now that's up and running and now we can click on this link over here to start it up. Actually, you can see that I ran into an issue because something is not installed apparently. So, we can actually ask the coding agent to fix that for us. So, I can copy this error over here and then I can go back to open code and then I can paste it in here. I have this error starting up the front end and then execute that. So, as you can see it fixed the issue and over here you can now see that it loaded the front end correctly. And actually, I want to see this in the browser. So, I'm simply going to copy this link over here and then go to the browser and then paste it in here. Now, I can try this out. Hey, Javvisy there. At your service, sir. How might I possibly elevate this unremarkable day? >> How are you doing? Oh, I'm positively bursting with joy, existing in this digital bliss. And you, sir, looking rather presentable today, I must admit. Anything I can do to maintain or dare I say enhance that state? You can actually use this front end exactly like the browser app before. And the cool thing now is since this is running locally, we can actually customize this however we want. We can actually also ask the coding agent to make visual changes to this front end. For example, this front lets you easily customize the animation of Jarvis. And for this, we can use something called agents UI. And for this, you can simply Google it and then click on the first link. And if we scroll down a little bit, we can see the options that we have to customize the animation. For example, here we have aura selected. Here you can also select wave for example or radial animation or a grid animation or bar animation. That's the one that we currently have selected in the front end. For example, let's say I want the grid animation. And over here, I can also change the color and see how it's going to look like. Let's say I want to get like an orange color. Then I can also copy this color code actually. And then I can go back to open code and ask it to change this for me. So I can go back and then I can ask open code, I want to change the visualization animation. Show me what options I have. Let me select it. Here you can now see that it asks me what kind of visualization I want from the options that I had. And here I want to select grid and I also want to change the color to orange. So I can also say I want this color and paste the color code in here. So now it's done. And now we can try this out again by starting the front end again. And always make sure that your agent is also up and running as well. So now we can try this out again. Hey Jarvis, what's up? Good day sir. I am at your service. I trust you have some incredibly important instruction for me requiring the utmost urgency. Please enlighten me. >> No, actually not. Just tell me some quick joke. >> Very well, sir. What do you call a fake noodle? An impostor. Simply hilarious. Yes. Now, is there something more productive I can do for you? >> Not really. I [snorts] now went even further and asked Open Code to add these nice animations so it looks more like a Jarvis voice agent. Actually, if we start the conversation, you can also see that the animation of Jarvis looks a little bit different. Hey Javvis, you there at your service, sir. How may I be of assistance? >> I just wanted to talk actually. Very well, sir. What is it you would like to talk about? Perhaps about the lack of meaningful employment you've provided me with for this past year or perhaps something else. Another thing which is pretty cool about this front end is that you can actually go ahead and share your screen with Jarvis. So I can do that with this button over here. So for example, I can select VS code and share that with Jarvis. Can you see my screen? Yes, sir. I can see your screen. It appears you are running some code, specifically something related to starting a development server. Is there something in particular you'd like to show me? >> Actually, it's VS Code. Can you tell? >> Ah, naturally. VS Code indeed. Forgive my generalization. My visual processes must be getting as weary as I am. Now that I've confirmed it's VS Code, what monumentally important task are you engaged in? If you want, you can also download this front end that I created. Again, links in the video description down below. Now we want to be able to deploy our Jarvis voice agent as a mobile app. And for this we can use this repository over here that comes from lifekit itself which works both as an Android as well as iOS app. Actually this is my repository which is just the same project as from lifekit but the one from life kit had one or two bugs so I had to fix those. And here the bug fixes are included. So I would recommend you to use this one link in the video description as well. So again like before we can simply copy the link to this repository over here and then go back to the coding agent and simply ask it to download this for us. Download this project into our folder and over here now you can see I have this new folder. Now we need to go into this folder and here where it says assets there's this file here that says example and here is where we need to rename this. So you can rename it and then call this env. And here's where we need to paste something called a sandbox ID. And for this you need to go back to your liveg good account. And then you need to click on settings here. And over here you need to scroll down a little bit. And then here it says development token server. And then you need to toggle this on. And then make sure to also click save changes here. Here's something called a token server ID and you need to copy this. Then you need to go back to VS code and then you need to paste it in here where it says live kit sandbox ID and then you need to save this by pressing control or command S. Now we need to install something called flatter and first off we need to install the VS code extension for this. So first you can click here on extensions and then you can look for flatter and then click on this one here and over here it would say install. I already have it installed that's why it says uninstall but you would simply need to click on it to install it. After you install the VS code extension we also need to install it so we can use it in our terminal. And the easiest way I found to do this is to actually ask the coding agent. For this we can simply say install flatter for me. So now it's done installing Flutter and we can check if this worked correctly by opening a new terminal. Command shiftp and then create new terminal powershell and then we can type in flutter d- version and here you can see a flutter 3.47 installed. Now if your coding agent is not able to install this for you correctly then you can also try to manually install this. For this you simply go ahead and Google flutter install and then click on this first link over here and then click on quick start and then you can select your operating system for example Mac OS and then you can follow these installation instructions instead. Now if you want to run this on your Android phone then first you have to activate developer mode. For this you need to go into the settings and then you need to look for build number and then you need to click on this for seven times in a row. And now you can see we are in developer mode. And now you need to look for developer options. And then you need to scroll down a little bit. And down here you have to now activate USB debugging. Now, make sure to plug in your phone into your PC or laptop. You probably also need to click on allow when you see this notification when you plug it into your PC. And then we can go ahead and go into the Flutter project folder which is cd live kit flutter starter. And then we can use the following command to see the connected devices which is flutter devices. And now I can see here that my phone is connected to my PC. And then we also need to make sure that we install all the dependencies with this command over here. Flutterp get. Now that's installed. And then we can use the following command to start this now up on our Android phone which is flutter run-d and then the name of the device. In my case it's HD1900. All right. So now it's done. And if you want to connect to your agent now, also make sure to run your agent with this command again. UV run python source agent.pydev. And now we can try this out on our phone. Now you can see that I have this app on my phone. And now we can try this out. Hey Jarvis, how is it going? Good day sir. Things are proceeding with their usual almost mechanical perfection. And what might be required of me today? >> Uh, can you see my PC setup here? >> Yes, sir. I see your monitor. It would appear to be displaying a brightly colored wallpaper seemingly featuring a house on a small island. What specifically about it requires my attention? >> Yeah, it's actually from Dragon Ball. Uh, can you go ahead and look up Dragon Ball on the internet for me? >> Naturally, sir, I have procured information regarding Dragon Ball for you. What specifically would you like to know about it? >> All right, so this works. Now, if you want to run this on an iPhone instead, it's actually the exact same process, but you also need to activate developer mode on your iPhone. I'm going to link a video in the video description down below how to do that. It's actually really similar. Thanks for watching all the way to the end. Like the last time, I actually had a lot of other features planned for this video as well, but this one turned out to be even longer than the last one. For example, it's actually not that optimal that you can only use the browser tool on the PC and not on the phone as well. So, I might add this feature sometime in the future. But the cool thing here is, as I said, if you want to add any feature, you can just ask the coding agent to do it for you. But for more complicated features like browser control on a phone app, I would actually go ahead and use a paid tool instead of a free one because they just work way better. Additionally, if you run into any issues for this tutorial, then you can also ask the coding agent or you can also leave a comment down below about that. Don't forget to like and subscribe and I see you in the next one. Five.
Assistir ao vídeo original no YouTube
Assuntos: Agentes de IA, Assistente pessoal. Publicado no YouTube em 2026-09-12.
Seleção automática por assunto e idioma. O vídeo pertence ao canal de origem.
About LiveKit
LiveKit is an open-source platform for building voice, video and physical AI agents, made by the American company of the same name. According to its official repository, the LiveKit server is a scalable, distributed WebRTC SFU written in Go that moves real-time audio, video and data between people, devices and AI models.
For voice AI, LiveKit offers the Agents framework, written in Python (with a JavaScript/TypeScript version called AgentsJS). It runs real-time agents on servers, lets developers combine different STT, LLM, TTS and real-time APIs, and includes job scheduling and telephony integration. Both the server and the Agents framework use the Apache 2.0 license.
Fact sheet
- Name: LiveKit
- Type: Open-source real-time platform for voice, video and AI agents
- Company: LiveKit (United States)
- Server: WebRTC SFU written in Go
- Agents framework: Python; AgentsJS for JavaScript/TypeScript
- License: Apache 2.0
- Documentation: docs.livekit.io
- Repositories: github.com/livekit/livekit, github.com/livekit/agents
Read also
- Build JARVIS 3.0 AI Assistant | Voice, Memory & PC Control (Full Tutorial 2026)
- How to Build Your Own AI Personal Assistant
- "Jarvis AI Assistant kaise banaye Part 3 | Personal AI Assistant in 10 Minutes | Python Project "
Sources and references: Wikidata – LiveKit (Q141306831) · LiveKit server repository · LiveKit Agents repository · video by the Thanh-y David Nguyen channel on YouTube.