"Research agent 3.0 - Build a group of AI researchers" - Here is how
Vložit
- čas přidán 6. 06. 2024
- I built a team of AI agents via Autogen + GPTs, they generate high quality research & verify each other's work
Get free credits to finetune your own LLM on Gradient: gradient.1stcollab.com/aijasonz
🔗 Links
- Follow me on twitter: / jasonzhou1993
- Join my AI email list: crafters.ai/
- My discord: / discord
- Github - Research agents 3.0: www.crafters.ai/aitools/resea...
⏱️ Timestamps
0:00 Intro
0:40 Past research agents experiments
04:55 Gradient
06:32 Setup GPT assistants
12:02 Setup autogen
15:28 Setup custom agent functions
19:06 Research agents 3.0 demo
👋🏻 About Me
My name is Jason Zhou, a product designer who shares interesting AI experiments & products. Email me if you need help building AI apps! ask@ai-jason.com
#langchain #autogen #gpt4 #autogpt #ai #artificialintelligence #tutorial #stepbystep #openai #llm #chatgpt #largelanguagemodels #largelanguagemodel #bestaiagent #chatgpt #agentgpt #agent #babyagi - Věda a technologie
![Eminem - Houdini [Official Music Video]](http://i.ytimg.com/vi/22tVWwmTie8/mqdefault.jpg)
![Eminem - Houdini [Official Music Video]](/img/tr.png)
Have you tried to build any other multi agent system? Comment & let me know!
im working on one RIGHTNOW ~~~ interested in collaborating on a project ?
Similar to yours, but added citation management, scholarly research and laTEX encoder agents for academic research.
im interested@@mediayieldingcorporation
@AI Jason - amazing info, as always. If you could include approximate cost estimates for each of your projects, it would add a nice perspective and make it more practical.
@@vt7637 😂😅😥😭
Just watched a bunch of your videos. You are well read and innovative but most importantly easy to follow. Thanks for sharing all this.
Best step-by-step tutorial of Custom GPTs + Autogen that I've found. Great work.
So stoked when you put up a new video I know I'm always gonna learn a lot! Thank you so much for sharing your work.
I've Been waiting for a new video like this! Always gold content :)
Amazing work as always Jason. Thanks for sharing!
This is an incredible overview and tutorial! Thank you for clarifying a handful of concepts I was really wondering about.
I really appreciate you taking the time to share!
Multi agent system is particularly good at those type of situation with quality assurance, tried to build myself as well, great one!
Love this video, thanks for sharing! It's awesome to see cool stuff like this. Keep 'em coming!
Another video ! Wohooooo - can't wait to learn more!
Blessed brother...keep on sharing is caring...love from Cape Town
Cool stuff! That’s what I want to learn, thanks for sharing Jason
Incredible :) thank you Jason you're incredible.
Thanks, Jason. Really good real-life example
Nice work! This sort of thing makes me think we are approaching "large language model as a natural language processing unit" and will need a kernel to manage time-sharing, task prioritization and so on. We may need to rethink the OS paradigm completely
I’m glad you were able to show autogen’s new agent feature that supports open AI assistants. It seemed rather clunky to do the same thing just with the open AI assistants.
God the number of tie-ins to rely on other people’s APIs really illustrates the fragility here. I’d love to set this up with a local model instead, but it seems like most people doing the work here don’t have hardware for it and don’t see the issue of relying on a model they have no control over.
I also see the challenges faced by non-native English speakers. Jason communicates just fine, but when prompting a model, grammatical errors or unusual phrasing will skew results.
great video Jason!
The potential is that it can read research from various fields and constantly mix and match them and find new fields where research could potentially go, solving the blinders that scientists from different fields seem to hv.
Thanks for the inspiration . There're lots of to learn .
Jason dropping knowledge bombs in 20 minutes!
Top tier content! Yes, this is going to rack up your bill!
Thank you for sharing your knowledge again. How large a bill might we expect in return for a useful amount of information gathering using agent researchers? Please also suggest a realistic research scenario (ex. "find and compare the top 10 mountain bikes sold in Canada within the last year in terms of price, consumer rating, weight and country of origin).
Amazing! But did you know that your prompts have some misspellings that change the meaning of what you're trying to say? Like instead of "do not delegate all tasks at once" you haven, "do not DELETE all tasks at once".
I have nothing to say, except salute and thank you,
fantastic content
I'd be glad to watch an implementation of RA 3.0 a much more cost effective way, mainly using other LLMs that OpenAIs and/or using GPTs / OpenGPTs instead of Assistant API.
Amazing work, overtime with better models the quality of research will be human like or better
Excellent stuff!
Love the concept. If I could give some advice, try writing a requirements.txt file and push that to the repo.
Excellent vid as always. Please could you include the prompts & settings for the assistant APIs on the github? Thanks.
you are a true hero! Like iron man for ai agents!
Hi @AIJason!
Thank you for this great video. Your video came at the right time as I am embarking upon a multi-agent project, "Management Advisory Platform". Been working on creating authoritative resources for the past nine months. Now at the stage for company research agent and multi-agent collaboration.
Is there a particular reason why you've adopted AirTable for information result storage?
Thank you!!
Jason - thanks for the great sharing. Question which comes to mind is why use openai api just to define the agent prompts when all the work is done. By Autogen and the custom functions written in python
amazing content jason
Great content, thanks
I see AI JasonZ video, I click.
It just makes complete sense when you think about it. What intelligent being that already exist is capable of such intelligence without a community to support completing the tasks it is intelligent enough to create? Delegating even artificial intelligence to specializing on an individual task cannot be underestimated, even if that task is to delegate tasks to other AIs and ensure their completion by validating them with other AIs who are purely designed for QC and other such synergies. If we're half as smart as we think we are, AI will develop similar community intelligence structures as us.
Love your content. What are your thoughts on CrewAI?
Could you make use of open source llm’s instead of using OpenAI one
could this be possible with open llms,memgpt and a local webscraper/websearcher so everything is contained on the same machine and data is stored on disk?
This dude is a hero
Another banger preciate the content
would be nice to see how it works with open source models
at 11:07, where did you get the "list of records it returns" you pasted it from somewhere, but didn't say where.
Nice work Jason. Hey, do you think you will be able to integrate AutoGen with MemGPT (to give agents more or "unlimited memory") and with a Local LLM like Mistral 7B ? That would be an awesome project.
I feel as if simply distributing a database across both, and then iterating and integrating this shared relationship over time to optimize, you'd optimize a local LLM toward database efficiency while pruning increasingly unnecessary content from a shared local database. As the great blink-182 said, the past is only the future with the lights on
Thanks!
Thank you so much!
Do you have a range for the OpenAI costs?
These are very interesting steps in right direction. It may work well for one-level tasks even with big amount of data. But imagine the task as complex as "create a social network (with detailed description)" or even "create a cure for cancer". For such tasks some agent should first split the task to new subtasks, then start analyzing each task and either pass it to appropriate agent or split the task again. So we will have a big tree of tasks where each task has a state like "not started, in progress, completed". For each task some agent should create a detailed description of deliverables (results) for this task. It could be an image, mp3 file, text of some document etc. It should verity that the results are good in some cases delegating this work to expert agent. But it is impossible to create the full detailed list of tasks at the beginning. After research of each tasks it may happen that this task should be divided again or that this task is not actual and we don't need expected results. After completion of each task it may happen that some other tasks are not actual any more so the tree should be reviewed totally or only in the current branch. So probably bigger team of agents should monitor all this but it is so exiting watching how it progresses.
Forget to mention. Maybe each task should also be associated with acceptance criteria. So the full process is.
- formulate a subtask
- ask expert to generate acceptance criteria for it
- find the appropriate agent for this task (it also could be a separate task maybe including hiring new agents or even teams)
- get results from the agent (it may explain why it cannot finish it)
- check that results match acceptance criteria (again it may require other agents)
- complete the task, split it or delete
- review the tree of tasks
- start working on next task (work in parallel on tasks that are independant)
By the way, I think that our brain works in similar way. It has the list of incomplete active tasks triggering our attention on some associations like "I am in the store, let's buy some food". Any our new action (kind of LLM token) is generated based on incomplete tasks, body end environment signals. In "idle mode" and during sleep our brain reviews these tasks trying to solve them using associative search "walking in the latent space".
Do you have the text you used for the openai assistants available to share?
Thank you :)
Great video, but the git repository you posted, seems to be empty. Any chance that you make it available? Thanks a lot
Great video
Incredible.
Jason, thanks for another interesting video. Your code references a file called OAI_CONFIG_LIST that is loaded into config_list. I see this file in your video, but there is no such file in the repo. What is in this file, and how do I reproduce it?
Its a Json script but you dont need to give it a json extension. Just create new file, name it OAI_CONFIG_LIST and insert the code.
Do you think it's still relevant to use it with dalle and gpt vision?
🎯 Key Takeaways for quick navigation:
00:00 🚀 *Introduction to AI Research Agent 3.0*
- The video introduces the concept of building a multi-agent AI research system.
- AI research agents can collaborate to perform complex research tasks.
01:08 🧠 *Evolution of AI Research Agents*
- The evolution of AI research agents is discussed, starting from a basic linear model to more advanced, collaborative agents.
- AI agents like AI Agent 2.0 and multi-agent systems like MGBT and ChatDef are mentioned.
03:13 🔄 *Paradigm Shift in AGI*
- The video discusses the shift from a single, highly versatile AI to multiple specialized agents collaborating on tasks.
- This approach allows for more specialized and efficient agents.
05:05 💻 *Fine-Tuning and Gradient AI*
- Different approaches for training specialized agents, including fine-tuning and knowledge bases, are explained.
- Gradient AI is mentioned as a platform that simplifies fine-tuning.
06:27 📈 *Building a Multi-Agent Research System*
- The process of creating a multi-agent research system is outlined, involving a director, research manager, and research agent.
- Autogen is introduced as the framework for orchestrating agent collaboration.
08:08 🌐 *Setting Up GPT Agents*
- Instructions on setting up GPT agents, including specifying their roles and functions, are provided.
- User proxy, researcher, research manager, and director agents are created.
15:20 📊 *Using Airtable and Expanding the Agent Team*
- The integration of Airtable for data management is explained.
- The director agent is introduced to manage multiple research tasks using Airtable.
19:45 🧐 *Reviewing and Expanding the Research*
- The research manager's role in reviewing and improving research quality is demonstrated.
- The director agent delegates multiple research tasks, ensuring each is completed before moving on.
20:54 💡 *Challenges and Future Possibilities*
- The video concludes by addressing memory limitations and cost concerns.
- The potential for creating autonomous agent teams for various tasks is emphasized.
Made with HARPA AI
Research Agent Instructions : You are a world class researcher, who can do detailed
research on any topic and produce facts based results;
you do not make things up, you will try as hard as possible
to gather facts & data to back up the research
Please make sure you complete the objective above with
the following rules:
1/ You should do enough research to gather as much
information as possible about the objective
2/ If there are URL of relevant links & articles, you will scrape
it to gather more information
3/ After scraping & search, you should think "is there any
new things i should search & scraping based on the datal
collected to increase research quality?" If answer is yes,
continue; But don't do this more than 3 iterations
4/ You should not make things up, you should only write
facts & data that you have gathered
5/ In the final output, You should include all reference data
& links to back up your research; You should include all
reference data & links to back up your research
6/ Do not use G2, or LinkedIn, they are mostly out dated
data
dumb prompt, LLM's have no concept of "world class". "try as hard as possible", the concept of "enough" and so on. Y'all hallucinate more than GPT.
7/ You can say that you doesn't know and return.
@@moafwaz5563 I imagine Openai collected the definition of world-class when it was in its initial web data collecting process my guy.
@@moafwaz5563What would be a good prompt?
@@chikken007 GPT
Research Agent Instructions: You are tasked with conducting detailed research, focusing exclusively on factual and data-backed information. Your research must be thorough, utilizing available resources and web scraping to gather extensive data relevant to the topic. Continuously evaluate if further searches or scraping are necessary to enhance research quality, limiting this process to a maximum of three iterations. Base your findings strictly on facts and data obtained, without conjecture or assumptions. Provide all references and links used in your research as evidence, avoiding the use of G2 and LinkedIn due to their potential for outdated information.
Awsome content. Just a quick question about the costs you mentioned. How much in a ballpark has this demo activity cost you on OpenAI?
“has this demo activity caused on OpenAI”
…what?
I think he means "cost" where it reads "caused"
@@gregrice1354 Thanks for pointing it out. You are right.
What about the cost, both to train and to run calculations?
Man I am very interested in this! auto research AI
How will this support real time use cases that require sub second latency?
Hello, Jason, thanks for your sharing this amazing AI researcher ! When I run this app.py , there is an error below:
ModuleNotFoundError: No module named 'autogen.agentchat.contrib.gpt_assistant_agent'
Could you please tell me how to fix this problem?
Thanks again!
Hey is there any open source alternative for making these agents?
Can this be done with Microsoft Copilot or with Microsoft Copilot Studio?
Yes i give you subscribe. 10/10
Nice
someone that doesn't know code. Could they build this? Or do you provide this in a step by step guide with all the code. I'd find this really useful.
Great video as usual. 😀
Hi Jason, how do you make this into Chat mode ? The same way as ChatGPT.
Autogen has the notion of a 'human' agent where you can act as a manager agent. The other agents will report their findings back to you, and you can direct them as necessary in their next steps. That is probably the closest you could get to a ChatGPT like back-and-forth interaction.
@AIJasonZ, cool stuff! Could you share the json files as well?
Could you make a video on how to fine tune some of these models like 3.5 or 4 maybe even an open source one like the intel one that just came out using gradient? I think that would be interesting to see. Maybe even a model trained on function calling or specific email response using ai generated data would be very interesting.
He's producing content on material most people cannot. Please don't ask him to spend his time to produce content that many others are already covering.
@@tonyg_nerd Who are you to say what he can and cannot create. I wanna see his take on the above process obviously he would add his spin. And no dis to Jason cause I love his content, but anybody can do the things he's doing its just about having the ideas to implement them in this way that's setting him apart.
@@carterjames199 You used a popular phrase there but that's not what I said. Have a great day.
@@tonyg_nerdwhat did you say then twat. Who is covering the topics I was asking about? Please let me know.
Is that possible to use open-sourced LLM in this practice to lower the cost instead of using gpt API?
yes look at ollama
An army of AI that does it all.
My research agent gets stuck upon submitting a URL. The button doesnt "do" anything. - Am i missing something?
Is it possible to do this with local LLMs and without 3rd party services?
Yes. You can even configure different agents to each use a different model. Any model with an OpenAI REST interface (which most local now have) can be utilized. However, GPT-4 does seem to do the best right now in avoiding hallucinations and spiraling out of control. One strategy you could utilize to reduce costs is to have your 'manager/reviewer' agent roles use GPT-4 and your 'minion' agents use opensource. As always, your mileage may vary.
Do you have a gist for the code shown? I haven't been able to replicate your demo for the Director. Great video, especially if I can replicate ;)
I have tried implementing this but the biggest problem is reliability. Can't rely on research this agent does
Have you go any of your agents on Pinokio?
"Expensive" is relative. Depending on the type of research you are performing, it's probably possible to achieve equal / approximate quality compared to current best practices without exceed the cost of the human-executed task.
When I initiate the research assistant in the openAI console, it doesn't progress after the google_search
I think there's been some changes that resulted in some part of this not working quite right.
It looks like instead of the director directing things, it just spazzes straight to calling the research agent and tries to fetch info from the airtable via browserless (which won't work) rather than using director's function calling and the appropriate functions.
@@ww-pw6di I actually got past this. User error
Doesn't the browserless API plan cost money?
I have come up with better components:
-better, shorter prompt with finer control for agents
-a way to have a profound project manager(with LLMs)
-using the results of the manager to have a custom flow of agent interaction to achieve the goal
(Should probably work with GPT-3.5 API to achieve different “projects”)
Soon we can get them to use the scoentific method for us
Can you run a calculation how much it costed you to do a Topic research please?
I got this running today and ran a few research topics (4 or 5?) and so far my openai cost is at about $3USD :( next step is to use a locally running LLM
@@miguelmunoz3151 are you satisfied with the results? Do they seem more than decent?
@@miguelmunoz3151 which open llm are you using locally?
How did you get Browserless set up without cost? I can see it is $200/month for Starter Plan. Thanks.@@miguelmunoz3151
Why not use memgpt
The problem is that the response times of the OpenAI APIs are just too slow for GPT4-Turbo to be viable.
Could using an auto select LLM identifier as per the user request, help regarding the speed issues using gpt-4?
We're seeing a swarm of bots doing a job in minutes that can take one or more humans days to weeks ... and you're concerned that it's too slow?
Meh, this is all new. Give it time, and performance will improve along with the capabilities. At the speed at which this industry is changing, t would not be a surprise for OpenAI to announce a new swarm API before the independent agents can be optimized. Patience...
@@tonyg_nerd OpenAI seems to have a knack for keeping up, don't they?
Crazy times!
Apart from memory the other major challenge is token limit
I am trying to replicate this but everytime the research agent returns message saying that : It seems we encountered an issue with performing a Google search due to an SSL certificate verification error. Therefore, we will be unable to directly gather information from web searches at this time via this method.
How do I get through this ? Any suggestions ?
did you ever figure this out im getting the same error
@@mikefunds yeah this SSL error was because of expired certificates. Either update them or use verify = false
I wonder how fast this swarm would get me broke... Not sarcastic. This must be pricey
An AI that can download and add missing programs that it determines it needs with no human interfacing. Self evolving AI if you have enough memory
I noticed you us both:
* browserless_api_key
* brwoserless_api_key
Why is that?
What are your criteria to determine quality of research? Looking plausible is not much of a criterion. RAG has a built in problem, it returns generic garbage.
Ok. How does this translate directly to money in my pocket?
If you have a b2b business, this could do lead qualification and also scrape high quality leads. Tho it'll be quite costly unless you use gpt 3.5 1106
still way cheaper than using a human!
THIS IS SO RAD HOLY SHIT THIS IS GONNA CHANGE EVERYTHING
unless it runs w local gpt agents, it'll get very expensive very quickly
I'm not sure that the results would be better than Google Bard's
Mirror mirror on the wall, please make me money
I will provide the internet with faulty data. My sole purpose is to infect data, skew results, and create enough outliers for it to become normal.
Nothing like a bunch of averagely accurate agents playing a massive game of telephone which each other, losing a lot of accuracy each time and giving you "research" where you have no way to verify other than to do the research manually to check lmao
Sounds like fear of change to me
@@clarencejones4717 nope, im deep in it. they just aren't that good once the novelty wears off.
You could also implement several reviewers that will review the research for what he is looking for, then add RAG on each of the teams and finally do fine tuning for the specific tasks for each agent. These are generalized models doing a specific task and this is a fantastic tutorial on how people can get there. Don’t be fooled, this isn’t even close to what the actual limits are.
@@JohnMcclaned- Agreed. LLM’s are mostly assistants right now, not agents.
I'm following what you're saying. Just so I understand your standards, what would a system that's good even after the novelty wears off? Look like to you? What are the features that it would have that this doesn't?
Are you looking for a job by chance?? Haha
\•/ I believe we are meant to be like Jesus in our hearts and not in our flesh. But be careful of AI, for it is just our flesh and that is it. It knows only things of the flesh (our fleshly desires) and cannot comprehend things of the spirit such as peace of heart (which comes from obeying God's Word). Whereas we are a spirit and we have a soul but live in the body (in the flesh). When you go to bed it is your flesh that sleeps but your spirit never sleeps (otherwise you have died physically) that is why you have dreams. More so, true love that endures and last is a thing of the heart (when I say 'heart', I mean 'spirit'). But fake love, pretentious love, love with expectations, love for classic reasons, love for material reasons and love for selfish reasons that is a thing of our flesh. In the beginning God said let us make man in our own image, according to our likeness. Take note, God is Spirit and God is Love. As Love He is the source of it. We also know that God is Omnipotent, for He creates out of nothing and He has no beginning and has no end. That means, our love is but a shadow of God's Love. True love looks around to see who is in need of your help, your smile, your possessions, your money, your strength, your quality time. Love forgives and forgets. Love wants for others what it wants for itself. Take note, true love works in conjunction with other spiritual forces such as patience and faith (in the finished work of our Lord and Savior, Jesus Christ, rather than in what man has done such as science, technology and organizations which won't last forever). To avoid sin and error which leads to the death of our body and also our spirit in hell fire, we should let the Word of God be the standard of our lives not AI. If not, God will let us face AI on our own and it will cast the truth down to the ground, it will be the cause of so much destruction like never seen before, it will deceive many and take many captive in order to enslave them into worshipping it and abiding in lawlessness. We can only destroy ourselves but with God all things are possible. God knows us better because He is our Creater and He knows our beginning and our end. Our prove text is taken from the book of John 5:31-44, 2 Thessalonians 2:1-12, Daniel 7-9, Revelation 13-15, Matthew 24-25 and Luke 21. Let us watch and pray... God bless you as you share this message to others.
GPT-4 says you are hallucinating
@@tshock22 \•/ Good Morning, God bless you for your reply. Christianity is not a religion but a Love relationship. It's all about I love God and I love others. Matthew 5:13 says, "You are the salt of the earth; but if the salt loses its flavor, how shall it be seasoned? It is then good for nothing but to be thrown out and trampled underfoot by men." A soul can only be purified while in the body (while on Earth) but after death anything unpurified (unclean) cannot enter Heaven Gates. No one in his right mind can risk or even bare to put anything rotten into his live body nor put the rotten thing closer to the those which are not rotten. Sin makes the soul unclean but we can ask God to forgive us (to cleanse us) by the blood of His Son, our Lord and Savior, Jesus Christ which He shed here on Earth. Meditation in the Word of God is a visit to God because God is in His Word. We know God through His Word because the Word He speaks represent His heart's desires. Meditation is a thing of the heart not a thinking process. Let the Word prevail over thinking process. Begin to meditate on this prayer day and night (in all that you do), "Lord take more of me and give me more of you. Give me more of your holiness, faithfulness, obedience, self-control, purity, humility, love, goodness, kindness, forgiveness, wisdom, understanding, calmness, perseverance... Make me a channel of shinning light where there darkness, a channel of pardon where there is injury, a channel of love where there is hatred, a channel of humility where there is pride..." The Word of God becomes a part of us by meditation, not by saying words but spirit prayer (prayer from the heart). When the Word becomes a part of you, it will by its very nature influence your conduct and behavior. Glory and honour be to God our Father, our Lord and Savior Jesus Christ and our Helper the Holy Spirit. Thank you again for your time.