AutoGen Studio Tutorial - NO CODE AI Agent Builder (100% Local)
Vložit
- čas přidán 24. 07. 2024
- AutoGen Studio was released, and it makes creating AI Agent teams super simple; basically, there is no code. I'll show you how to install it, how to use it, and show you some examples.
Join My Newsletter for Regular AI Updates 👇🏼
www.matthewberman.com
Need AI Consulting? ✅
forwardfuture.ai/
Rent a GPU (MassedCompute) 🚀
bit.ly/matthew-berman-youtube
USE CODE "MatthewBerman" for 50% discount
My Links 🔗
👉🏻 Subscribe: / @matthew_berman
👉🏻 Twitter: / matthewberman
👉🏻 Discord: / discord
👉🏻 Patreon: / matthewberman
Media/Sponsorship Inquiries 📈
bit.ly/44TC45V
Links:
AutoGen Advanced Tutorial - • AutoGen Advanced Tutor...
AutoGen Studio - microsoft.github.io/autogen/b...
LiteLLM - github.com/BerriAI/litellm
Conda - conda.io/projects/conda/en/la...
Ollama - ollama.ai/
Chapters:
0:00 - Intro
0:40 - Installation
2:30 - How to Use AutoGen
7:21 - Testing a Team
11:58 - Local LLM Setup - Věda a technologie
What tutorial do you want next about AutoGen Studio?
Set up a workflow for a specific department of a traditional company. Ie Human Resources, or billing department.
Build a SaaS
It would be workflow, a lot of agents and more examples.
Use it with lmstudio, memgpt and a rag like chroma DB 🙏🏿
Autogen Workflows need to be a site after this video, this will be the coming thing, Autogen templates and workflows… so maybe building a platform or infrastructure for that 😅 this is the new gpt store, Autogen workflows, said it here first ☝🏽😂 a civitai of sorts
For Windows users: set OPENAI_API_KEY= (Instead of 'export OPENAI_API_KEY=' )
Thank you🥰🥰
THANK YOU!!
Thank you this was driving me bonkers! XD
How to set API URL base?
@@AngusLou If you're using conda like in the tutorial and the above isn't working, try the following:
-----------------------------------------------------------
conda env config vars set OPENAI_API_KEY=[YOUR API KEY] -n ag
-----------------------------------------------------------
This will permanently set the OPENAI_API_KEY environment variable for the conda environment "ag" (obviously change that at the end of the command if you named your conda environment something different).
ALSO NOTE: I noticed there's a bug in AutogenStudio - if you change an agent to use a different LLM, it does NOT update that agent in the Workflows. You either have to create a new Workflow with the updated agent, or modify the agent in the Workflow itself.
I only realized this when I noticed Mistral wasn't outputting anything in the console, and then realized AutogenStudio was actually using GPT4 instead of Mistral like I told it to. Only after a lot of digging did I find the Workflows didn't update after I changed the agents to use Mistral. Hopefully they fix that bug, but that might also be why its calling for the OPENAI_API_KEY even if you're "using" local LLMs.
I'm so grateful for how quickly you figure this stuff out and then articulate it. Thank you for the hundreds of hours you save me.
+100% agree
For real
Very cool. Amazing how over the span of 3 months we went from an all command line version to this interactive webui solution. Can't wait to try it out.
Yesss! I've been waiting on this video from you! great job. Keep up the amazing work, your videos are very helpful even for the more "experienced" people working within AI
Matthew I am going to revoke this API key Berman
To be fair, he'd get spammed with folks saying he should revoke if he didn't
He used to get half the comments giving him "advice" to revoke the key 😂
This comment had me rolling with laughter. Thank you.
Haha I started saying it with him. It just feels right now.
@@MikeKasprzak exactly lol
Thank you for how quick and clear and straight you move through steps. Usually one has to skip parts and listen to rest on high speed to get info
Matthew, you are amazing, a blessed man!!! I am an engineer that wants to learn about AI and I enjoy your videos so much!!! Thank you so much for the high quality videos!!!!!
The speed at which innovation is happening is staggering and the speed at which you are making the videos is amazing
Have you seen the new RABBIT device ? It runs of the first
Large Action Model
Now the Rabbit could literally be working for you 24/7 and making you money while you sleep 🛌!!
Holy moley.. There is not enough time in the day to experiment with all these new toys! What a exciting time to be alive!. Thank you for easy to understand video
Yes this is exactly the selling point for Brain computer interfaces.... the rate of acceleration is slowly and steadily increasing
Yea but you still have the same rate of interaction with the data, which I guess is not a real problem yet. But there is no groundbreaking shift in the rate at which information can be transmitted from computer systems to biological systems, then I don't think we will be maximizing value extraction from these systems and other more concerning issues as monitoring behavior for deviancy and rebellion and other potential "internal" issues potentially buried within the weights of the models. For the incredible amount of money invested, it would be prudent to consider.
This is great. I would love to see a comparison breaking down CrewAI vs Autogen and theirs pros/cons/use cases.
Second this!
Third this
4th this!
Can someone please tell me what the fuck just happened here?? Not laughing at all . As a business owner how would this help me ? What real world applications?
PS. When he says AGENT is that similar to say a A.I. Companion aka Large Action Model like the new 🐇 RABBIT tech???
@@zacharywalker9102that’s what everyone is trying to figure out. It all depends on tasks you do everyday that takes hours of your word day. For instance, you keep sending emails to clients, or u do blogging for the company, or you need to get some data from internet. U have people to do these tasks, right? Why don’t u fire them and leave one guy who is going to use AI, for completing these tasks.
Amazing. Could you do a video for coding? A developer team agents (backend, front end, quality assurance, testing team, buisness consultant, project requirements drafter, project planner, etc) and have this produce a ready product.
This is awesome I started working with Ai after seeing your AutoGen video months ago. But my coding skills aren't strong and I've moved on to other things like LM Studio, Faradev, Coze, MindStudio, LOLLM, etc. Every time I scroll past my autogen folder with the python code I felt kinda sad for it. This is amazing I can't wait to delve back into it. FYI Gemini API keys are available for free now too.
Matthew you save me so much time figuring stuff out ! I appreciate you and your channel so much
Great video. It would be cool if you could demonstrate how AutoGen projects are deployed in saas production environments - perhaps do a complete case study from ideation, development and final deployment - thanks 🙂
Thank you. Could you please create a tutorial on how to use different options in memgpt like functional calling, custom instructions, RAG with local files work together?
A++ love your work matty! lots of love form Perth, Western Australia - keep up the good work!
Great vid Matt. Would be great if you could do a more in depth vid on skills, e.g. web search with google or other similar things that limit agents at the moment.
Nice work, your videos are much appreciated! You've caught me up so quickly and now I'm experimenting with my own agents. thank you!
You are the only useful channel about AI, others talk about theory, but you show us how to accomplish real life projects. Bravo.
the 'only' is strong word :) but i is loco :)
Can someone please tell me what the fuck just happened here?? Not laughing at all . As a business owner how would this help me ? What real world applications?
PS. When he says AGENT is that similar to say a A.I. Companion aka Large Action Model like the new 🐇 RABBIT tech???
@@zacharywalker9102 I don't use this information for business, I used it for my pleasure:) < IBM Watson , and Google maps calendar and Gmail < i use < IBM is strong in business industry :) I love EV and Solar but Unix and Linux and more are my first lovers < have fun:)
The biggest benefit it is that new Autogen is using GPT4 turbo (now the cost is ok to play withit) - the old Autogen used old expensive GPT4. Thank you for the video.
I like that you always add the local llm twist. Thank you
if you cant run ollama because youre on windows like me, you can use lm studio to do the same. there is a local server function as well
Love these tutorial videos.
Hopefully Ollama will have a release for Windows soon.
Thanks a mil, Matthew. Your tutorial is always top-notch
This is great. Thanks Matthew! It would be helpful to do a multi-agent programming workflow.
Awesome. Really awesome. Thank you so much for this. Can you do a real world application tutorial on how to use Autogen? Sorta a case study.
I really appreciate this. I don't pay for the monthly sub to GPT- I only have the API access, so I don't have access to the image generator. This allows me to use their image generator without paying for the monthly sub!
Awesome. Thanks for posting. This will exponentially improve productivity Matt🤓
Note: If your doing this in a Windows Powershell Conda Enviroment, the command is $env:OPENAI_API_KEY = "sk-youropenaiapikeyhere"
Thanks for all the great work Mr.Berman 🙏
just tried this and its not working for me.
@@zerohcrows Yeah, I am having issues too connecting Litellm with the WSL Ubuntu as well. Trying to install the conda env on the linux side but running into issues with that as well. I will get back to you if I can find a fix.
That’s cool. Will be great to try to setup a team of developers, like project manager, frontend, backend, qa, devops, ui/ux. With local models, and give them simple project to accomplish as for instance a business owner. I’m really wondering how good it can be for something related to a simple business case, like landing page or promo product page.
Everything is ready, the whole team
I am a small business owner here for the same thing. Been using these similar models including ChatDev for a while bit I haven't found many good uses yet, it's mainly been me spending time making the outputs viable or troubleshooting the output scripts etc... I know some folks are automating the video lead process though
We are still limited by context windows are we not? Having this many layers would overflow the buffer.
I've been doing this with chatdev, crew ai and gpt pilot, it's pretty neat but looks like autogen studio might have em beat.
@@r34ct4still limited yes, but think of each agent having its own token limit. In a multi-agent setup, the token limit constraint becomes less of a bottleneck, especially in workflows involving multiple, smaller tasks
Great video! I'm going to try this. Is there any way to connect this locally to stable diffusion models to create images in the way that you created images using DALL-E3 with gpt?
I watch nearly every video you create Matthew! I really like the way you teach these topics. I always leave with something that I can actually start using right away! I had toyed with AutoGen Studio for a few minutes and you clarified several important parts that hadn't been clear to me. Please do continue to include steps for running with local LLMs. I think that part is super important.
I freaking love your videos. Can I make one request? Will you consider not quite doing your screencap to the very bottom of the screen? Since your code snippets aren't in the description, I'm constantly fighting the youtube playback bar to see the last thing you've typed. Keep up the awesome work.
i.imgur.com/wKlvbcp.png
You can navigate through CZcams videos with these keyboard shortcuts:
Use the "Left Arrow" key to rewind the video by 5 seconds.
Use the "Right Arrow" key to fast forward the video by 5 seconds.
For more precise control:
Hold down the "Shift" key.
Use the "Left Arrow" key to rewind by 1 second.
Use the "Right Arrow" key to fast forward by 1 second.
❤ Thanks you so much for sharing your knowledge. You will probably save me hours of experimentations… Great and simple video !
Thank you for explaining this! Can you do a detail comparison between AutoGen Studio and CrewAI? I'm torn between them.
Autogen has way more money to develop faster I guess..
Awesome video man - thank you for making this clean and easy.
Great tutorial! I would love to see you demonstrate some original complex multistep examples (that could not be executed by a single prompt in chatgpt).
Exactly my thought!
Great video. I followed all instructions but when I wanted to test the stock price example. It came back with "Error occurred while processing message: Connection error." Is there something that needs to be enabled on MacOS to run it?
Nice video. When would you recommend using autogen and when open interpretor?
Is there a way to have more than 2 agents in a workflow? I couldn't see how to add any more in the GUI?
Impressive! Thanks for sharing so quickly!
Thank you for this. I was just wondering last night the best way to run local LLM's with AutoGen Studio.
You're welcome, it's easy!
this is very valuable! personally I don't know how to work around the annoying memory problem with local models, it will stop generating after a few messages. Edit: I've had more luck getting it to generate more lengths with ollama, please do a video on using autogen skills! I feel like that's where the steerability is because this is similar to building a Rube Goldberg.
would be nice to see some real world agents application for personal use. maybe something like a tool that would help to research and compare different options when you wanna buy something, and then search for best deals. for instance, set a goal to find best wireless headphones in a certain price range that have certain set of features.
Q: Is it possible to view the output and messages generated by the agents during interactions?
A: Yes, you can view the generated messages in the debug console of the web UI, providing insights into the agent interactions. Alternatively, you can inspect the database.sqlite file for a comprehensive record of messages.
This looks great, can I chain multiple agents together (with outputs being the inputs for other agents)? I will deploy and test myself next week as well 👌
Tried to replicate the mistral workflow: Error occurred while processing message: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
I have no gpt4 model or similar left, only mistral everywhere.
It’s like a trim down open source version of the enterprise ready Watson X orchestrate on AWS
real nice Matthew! thank U so much for exposing me 2 U'r channel !!! U'r the super helpful quicky King!:) good luck!
How can we use lmstudio instead of ollama for windows?
Awesome video, thank you. How do I go about adding a model with data_source, such as AI Search?
Hey Matt - great video! I've been watching you quite a long time and remember when autogen first came out. If I create my own skills and agent profiles, how do I save them when I close down the app? Would we have to install from the source code? What do you recommend? thanks!
If you’re using auto gen studio, you can save your tools from the interface, and they will persist, even if you shut down the server
Mathew, one more time a great tutorial, thanks for sharing, one question can AutogenStudio ingest PDF documents for analysis? If so how to do so? Thanks
I haven't figured out how to set up my own group chat agent. AutoGen gives the Travel Agent Group Chat Workflow, but in there under Receiver>group_chat_manager>Group Chat Agents, I can't actually add my custom agents. Any ideas?
Anyone else getting a error message asking for an Openai api key when trying to run on a local model?
I did. Did you figure it out
@tobiakilo3413 yeah sort of. The version of Autogen in this video needs to have a placeholder key (i.e. "sk-not-needed"). Although I ran into the same issue with Autogen 2 and it took some tinkering to get past that error.
If you are savy with py notebooks, I recommend the non-studio version of AutoGen. More control, less UI bug complexity
Amazing tutorial!! Thanks!! It would be great to learn how we could connect Agents to an SQL database
This is awesome! Started playing with it and it’s really good. I can’t actually add my agents to the group chat manager though. The add button is there but doesn’t seem to do anything. Tried a couple different things but no luck. Anyone been able to add their own agents to the group chat manager?
what about windows if i want to run local LLM? we dont have ollama :/
Would be great to see a demo on how to use the API to autogen so that it can be used in other apps
I like the vids and have subbed.
Not trying to be a downer, just trying to understand the hassle:benefit ratio in this.
wait so can this be used as an alternative to custom GPT's with the same features?
If so can you please make a tutorial specifically dedicated to local custom GPT's, seen a lot of people also needing something like this.
If not then I'd love for someone to explain the differences between Agents and GPT's or at least the use cases. Ofcourse everyone uses them differently for example some people think GPT's are useless but for someone like me who needs a model to output based on specific premade knowlede bases, GPT's are insanely helpful, I never understood the use case for Agents tho
What about using Autogen Studio to build tools for your own applications? Do you have to run it using their interface indefinitely or can it be interacted with via API?
Awesome video! Thank u for sharing. One q: can i use a windows 10 Laptop to do the steps in this video?
Hi, big fan. I'm always on the lookout for your videos.
Could you add examples for your videos using APIs from together ai, Mistral, Hugging Face, or any other Open Source LLM provider?
Do you know if I can setup a especific type o agents? Like a GPTAssistantAgent or a RetrieveUserProxyAgent with AutogenStudio?
I'm trying to use LM Studio instead of ollama and lightllm. Seems like it should be even easier for single model agents?
Question, is there a config file that can be created for autogenstudio ?
I'm trying to figure out how to start autogenstudio up with a custom configuration.
Thank you, could you do an example of agents doing marketing research?
Can this software create Ebay listing agents that take the pictures from the user and then complete Ebay listings using API ?
I'd like to see something that breaks down how to have multiple local LLMs controlling agents at the same time.
the swarm I'm wanting to make is going to have GPT4, nous-hermes2-mixtral, mixtral, dolphin-mixtral, and mistral-openorca because a mixtral orca variant hasn't made its way to ollama yet.
how to get the workflow code of python so we can use it elsewhere? like in openai playground we get view code button, i made a workflow, tested it using playground, now where to get the code for it? or after testing it, we should write our own code seeing the settings we used in the workflow?
It's incredibly awesome 😃, Came across CrewAI. Can you please make a comparison video between Autogen Studio & CrewAI from the perspective of production ready solution with ease of use.
Can we import a document with this method ? Like using RAG to import a pdf or any text file ?
The future of AI really is what Karpathy said where a bunch of specialized AI models will work together to perform complex tasks like a computer.
Thanks. How to use workflow description correctly? Does it affect the output?
Oh boy, excited to see what’s in store, grabs 🍿 😁
Hope you enjoy!
@@matthew_berman🔥🔥🔥 as usual
Can you use lm studio instead of ollama? (running windows.)
Is it possible to use gemini api or lm studio?
Very interesting.
I see, you are running it on Mac. Does it run on PC as well?
and if yes, are the process to installing it the same?
Thanks for this video. Do you know/can you clarify how to connect Autogen Studio to Internet so it can do lookups?
@mathew can you talk about hardware requirements or point to an existing video?
How crewAI compare with this? And why you local speed seems faster than mine. My M1 Pro Mac seems answering very slow when using mistral 7b
well I have lm studio running Autogen studio, it curiously started working I was swapping in and out models and it said I had a setting wron would I like it changed I said Yes and now it runs, the output is rather odd as it says TERMINATE after every answer but its close any suggestions !
thank you for an excellent introduction
I wish I could somehow be brought up to speed on the level of skill needed to be able to do this confidently and comfortably and be able to explain any aspect you mentioned. I would pay someone to teach me.
Edit: 06:06:00-06:17:00 Beautiful 👍🏻👍🏻👍🏻
Learn Linux, that’ll allow you to run and understand the commands being run here. Then learn some basic Python so you can know how to configure these models. Once comfortable with that, I would learn web development so you can interface with ai tools you can build in Python. After learning that, learn the mathematics of machine learning i.e. Linear Algebra, Calculus, Statistics and so on. Finally learn a frame work like Tensorflow. Tensorflow is a machine learning framework.
With those first 2 steps though you can do most things that exist, like set up a RAG system or Agent
will lmstudio work with autogen?
I just had a zapier ad when you were talking about connecting to zapier 💀
Thanks you matt' u xplained well i think this IS a complete and easy way to manage llms. Can you do more of it. Case studies on working teams together. It Can be a researcher team and a products ou service créations team or orhers stuff thxxx again
That‘s promising. Just downloaded CrewAI, but this matches my needs more.
Really appreciate these videos!
Thanks so much!
We encounter issues when configuring the agents to use local models instead of an OpenAI key. Are there similar problems or proposed solutions? We implemented it with LM Studio instead of ollama because we faced error messages in LightLlm: 'ModuleNotFoundError: No module named 'pkg_resources'.' However, we consistently receive the following error message: 'openai.OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable.
I'm also getting this error using litellm and ollama
You need to set the environment variable for ChatGPT, then you can fix the API key error.@@thefutureisbright
Any luck? I'm also having this error
@@user-tz4sv5nc6b not really
Thanks for the video Matthew. It works on Mac and on Windos 11 with Conda and Python 3.10 (does not work with Conda and Python 3.11 for me).
Great job I would love to see a video using autogen and the free API from Gemini.
I was following this excited to get started, but then you got to the bit where you installed Ollama, and I switched off. As soon as Ollama runs on Windows I'll get excited again, until then, I'll have to go back to all of the other amazing AI things that are available.
Why don't you try using wsl on your machine and using that for your dev environment?
@@sebastienperceau-wells7932 Good idea. I'll have to track down a video on how to set wsl up, but it might be a good option in the meantime
What i have the most trouble is understanding how do you get to apply this into a real world. Like could you do a use case from start to finish? For example can you get an agent to create some sort of content and post it on tiktok? I would love to see how create that from scratch and build from there.
For anyone wondering, no, code execution does not seem to work with the Mistral/Mixtral models. The system prompts that Autogen creates are a bit too complicated for these local models to be useful. I’d say wait a few months for some better local models to be released and then try it again.
Could it be an issue with context length limitation? If not, should try to unpack what "too complicated" means.
@@john849ww By 'too complicated' I mean that the smaller parameter models have trouble following detailed prompts. Context length is not usually the limiting factor for system prompt performance.
@@GearForTheYear ok thanks
@GearForTheYear has anyone tried it with the code llama models
Can you do a video going over the practical application of using this?
Why do you have to use litellm? Can't you just use ollama serve to spin it up locally?
What is your value for the bash equivalent of "PS1", the prompt for the command line? My prompt didn't change when I launched the "ag" environment.
Thanks for the great intro!!!