![AI Engineer](/img/default-banner.jpg)
- 67
- 975 615
AI Engineer
Registrace 18. 09. 2023
Talks, workshops, events, and training for AI Engineers.
Building State of the Art Open Weights Tool Use: The Command R Family: Sandra Kublik
We opened model weights for Command R and R+, and the response was incredible. This talk will showcase the community's innovative projects and our journey to building SOTA multi-step tool use proficiency with the R family. We will also share the design decisions that make the R family unique and effective.
Recorded live in San Francisco at the AI Engineer World's Fair. See the full schedule of talks at www.ai.engineer/worldsfair/2024/schedule & join us at the AI Engineer World's Fair in 2025! Get your tickets today at ai.engineer/2025
About Sandra
Sandra Kublik is a member of the Developer Relations team at Cohere, where she educates and leads the LLM community. In 2023, Sandra co-authored the first manual book on OpenAI API, "GPT-3: The Ultimate Guide to Building NLP Products with OpenAI API." She hosts a CZcams channel and podcast where she explores the latest advancements and applications of LLMs.
Recorded live in San Francisco at the AI Engineer World's Fair. See the full schedule of talks at www.ai.engineer/worldsfair/2024/schedule & join us at the AI Engineer World's Fair in 2025! Get your tickets today at ai.engineer/2025
About Sandra
Sandra Kublik is a member of the Developer Relations team at Cohere, where she educates and leads the LLM community. In 2023, Sandra co-authored the first manual book on OpenAI API, "GPT-3: The Ultimate Guide to Building NLP Products with OpenAI API." She hosts a CZcams channel and podcast where she explores the latest advancements and applications of LLMs.
zhlédnutí: 1 025
Video
Git push get an AI API: Ryan Fox-Tyler
zhlédnutí 1KPřed 9 hodinami
In this workshop you’ll build a demo that augments an app with AI for classification, natural language search, summarization, and outlier detection. Then you’ll learn how to iteratively improve before and after shipping to prod. We’ll use Hypermode to move fast and reduce infra overhead. You’ll walk away with a greater intuition for when & how to integrate AI without refactoring. Also, we have ...
Hypermode Launch: Kevin Van Gundy
zhlédnutí 688Před 9 hodinami
Recorded live in San Francisco at the AI Engineer World's Fair. See the full schedule of talks at www.ai.engineer/worldsfair/2024/schedule & join us at the AI Engineer World's Fair in 2025! Get your tickets today at ai.engineer/2025 About Kevin After a decade Leading teams in tech through growth Founded Hypermode. I'm on a mission Make each dev an AI eng Skip data science
Disrupting the $15 Trillion Construction Industry with Autonomous Agents: Dr. Sarah Buchner
zhlédnutí 2,4KPřed 12 hodinami
Dr. Sarah Buchner, Founder & CEO of Trunk Tools, envisions a future for construction where an army of AI agents works on behalf of our users. We are currently deploying an agent every 45 days: our Q&A agent, TrunkText, is already saving field professionals 1-2 hours every day. We believe that ~$2.5 Trillion of the construction industry can be automated with Trunk Tools. Recorded live in San Fra...
10x Development: LLMs For the working Programmer - Manuel Odendahl
zhlédnutí 4,4KPřed 14 hodinami
In this hands-on workshop, learn how Large Language Models (LLMs) can significantly improve your productivity as a software developer. Drawing from three years of experience using LLMs in every aspect of his work as a principal engineer, the presenter will share practical insights and techniques that go beyond simple prompts and off-the-shelf tools. Through a series of interactive exercises and...
Building Reliable Agentic Systems: Eno Reyes
zhlédnutí 1,9KPřed 16 hodinami
Agentic system design is a rapidly evolving and intellectually fascinating field, with huge potential for transforming how software is used across industries. Unlike traditional software, agentic systems rely on non-deterministic and oftentimes difficult to predict decision making. Taking inspiration from fields like robotics, cybernetics, and biology, we can start to develop intuitions around ...
Building with Anthropic Claude: Prompt Workshop with Zack Witten
zhlédnutí 12KPřed dnem
BRING YOUR PROMPTS we will workshop them live! No yapping, no slop, just writing, testing, and editing prompts. Let's see how many we can get through! Got prompts that aren't working quite the way you want them to? Bring 'em here and we'll see what we can do! You'll want to also bring a couple specific examples they misbehave on. Don't expect many slides this will be one hour of nothing but wri...
Running AI Application in Minutes w/ AI Templates: Gabriela de Queiroz, Pamela Fox, Harald Kirschner
zhlédnutí 1,5KPřed dnem
Building and deploying generative AI solutions can be challenging and time-consuming, especially for startups with limited resources and expertise. In this workshop, you will learn how to use AI templates and GitHub to quickly prototype and deploy generative AI applications in minutes. AI templates are ready-made solutions that leverage Microsoft Azure Services like Azure OpenAI and GitHub feat...
Decoding the Decoder LLM without de code: Ishan Anand
zhlédnutí 3,1KPřed 14 dny
Spreadsheets are all you need: Decoding the Decoder LLM without de code The struggle to grasp the inner workings of AI models can leave even experienced engineers from non-ML backgrounds feeling lost in a sea of terminology and new concepts. What if the key to understanding the intricate mechanics of LLMs didn't require a Ph.D.? This session offers an innovative approach, employing spreadsheets...
Using agents to build an agent company: Joao Moura
zhlédnutí 24KPřed 14 dny
This talk is about a simple idea: Everyone should be able to use AI to make cool stuff. When I started making crewAI, a framework to build AI Agents, I didn't know just how much it would change the way I work. This isn't just a chat about building AI Agents; it's a story about how these tools helped us grow our business and how they can help you too. I'll share our adventure with crewAI, showin...
What's new from Anthropic and what's next: Alex Albert
zhlédnutí 18KPřed 21 dnem
Explore Anthropic's latest strides in large language models, emphasizing enhanced reasoning and multimodal capabilities. We'll showcase how these advancements translate into powerful developer tools, APIs, and best practices for building sophisticated, RSP-aligned AI applications. Recorded live in San Francisco at the AI Engineer World's Fair. See the full schedule of talks at www.ai.engineer/w...
How Codeium Breaks Through the Ceiling for Retrieval: Kevin Hou
zhlédnutí 14KPřed 21 dnem
Codeium is trailblazing the next frontier in retrieval and hint: it’s not just embeddings. Learn what the next generation of retrieval looks like and how 1M developers are already leveraging this superpower using the Codeium IDE plugin for AI autocomplete, chat, and search. We’ll dive deep into how existing benchmarks are failing us, what it takes to serve our custom models at scale, and what t...
Emergence Launch: AI Agents and the future enterprise: Dr. Satya Nitta
zhlédnutí 2,4KPřed 21 dnem
AI agents are poised to revolutionize software systems and devices, promising unprecedented automation and efficiency for enterprises. However, the road to this future is riddled with challenges such as inefficiency, non-determinism, high costs, discoverability, and rapid technological evolution. At Emergence, we are tackling these challenges head-on to transform the vision of useful AI agents ...
Low Level Technicals of LLMs: Daniel Han
zhlédnutí 28KPřed 21 dnem
This workshop will be split into 3x one hour blocks: How to analyze & fix LLMs - how to find and fix bugs in Gemma, Phi-3, Llama & tokenizers Finetuning with Unsloth - continued pretraining, reward modelling, QLoRA & more Deep dive into LLM technicals - hand deriving derivatives, SOTA finetuning tricks It's recommended you have Python with Pytorch and Unsloth installed (or use online Google Col...
Fixing bugs in Gemma, Llama, & Phi 3: Daniel Han
zhlédnutí 2,3KPřed 21 dnem
The story behind our 8 bug fixes for Gemma, multiple tokenization fixes for Llama 3, a sliding window bug fix and Mistral-fying Phi-3, and learn about how we analyse and find and fix bugs in open source models. Learn also how we make finetuning 2x faster for all these models Recorded live in San Francisco at the AI Engineer World's Fair. See the full schedule of talks at www.ai.engineer/worldsf...
Copilots Everywhere: Thomas Dohmke and Eugene Yan
zhlédnutí 1,2KPřed měsícem
Copilots Everywhere: Thomas Dohmke and Eugene Yan
Unlocking Developer Productivity across CPU and GPU with MAX: Chris Lattner
zhlédnutí 4KPřed měsícem
Unlocking Developer Productivity across CPU and GPU with MAX: Chris Lattner
From Software Developer to AI Engineer: Antje Barth
zhlédnutí 3,9KPřed měsícem
From Software Developer to AI Engineer: Antje Barth
Lessons From A Year Building With LLMs
zhlédnutí 12KPřed měsícem
Lessons From A Year Building With LLMs
Open Challenges for AI Engineering: Simon Willison
zhlédnutí 6KPřed měsícem
Open Challenges for AI Engineering: Simon Willison
Llamafile: bringing AI to the masses with fast CPU inference: Stephen Hood and Justine Tunney
zhlédnutí 39KPřed měsícem
Llamafile: bringing AI to the masses with fast CPU inference: Stephen Hood and Justine Tunney
The Future of Knowledge Assistants: Jerry Liu
zhlédnutí 80KPřed měsícem
The Future of Knowledge Assistants: Jerry Liu
The Making of Devin by Cognition AI: Scott Wu
zhlédnutí 7KPřed měsícem
The Making of Devin by Cognition AI: Scott Wu
From Text to Vision to Voice Exploring Multimodality with Open AI: Romain Huet
zhlédnutí 8KPřed měsícem
From Text to Vision to Voice Exploring Multimodality with Open AI: Romain Huet
The Code AI Maturity Model and What It Means For You: Ado Kukic
zhlédnutí 1,6KPřed 6 měsíci
The Code AI Maturity Model and What It Means For You: Ado Kukic
How to Become an AI Engineer from a Fullstack Background - Reid Mayo
zhlédnutí 6KPřed 6 měsíci
How to Become an AI Engineer from a Fullstack Background - Reid Mayo
Using AI to Build an Infinite Game: Jeff Schomay
zhlédnutí 794Před 6 měsíci
Using AI to Build an Infinite Game: Jeff Schomay
GPT Web App Generator - 10,000 apps created in a month: Matija Sosic
zhlédnutí 1,7KPřed 7 měsíci
GPT Web App Generator - 10,000 apps created in a month: Matija Sosic
Storyteller: Building Multi-modal Apps with TS & ModelFusion - Lars Grammel, PhD
zhlédnutí 1KPřed 7 měsíci
Storyteller: Building Multi-modal Apps with TS & ModelFusion - Lars Grammel, PhD
Open Questions for AI Engineering: Simon Willison
zhlédnutí 4,6KPřed 9 měsíci
Open Questions for AI Engineering: Simon Willison
Wow! Great stuff!
so what was the reason for using softmax? what would be otherwise had softmax not been used?
This is exactly what i needed, when I needed it. Big props!
Ailen Degeneres
Would be nice to have a newer version of CommandR / CommandR+ with Flash Attention and maybe a model in the middle of the size round (40-50b)
Looks promising too bad its python.
This guy doesn't understand what is quantization and also transistor number does not play a role in efficiency necessarily. The main issue is GPU sm design and ALU units😅
Lopez Mark Jones Angela Rodriguez Jason
You can say the iteration was next level.😂😂😊
So it's another RAG-fueld chatbot then.
I think it can be applied to other fields to keep the documentation up to date and for easy access to the documentation
This is what modern AI looks like 💜
😂 10% is rework .. comes from someone born post COVID .. human beings using constructed structures since the sun came to light the earth 😂 Also we give you 13thousend links to choose from and if your guys chooses the wrong instructions within the webapp .. it will only cost you 5 unjustified injuries and a couple million dollar lawsuits but hey thats your responsibility now AI cannot fix everything. Btw good luck with OSHA 😊😂
Mac app please
Finally
this guy should be a comedian
I found this super interesting and informative.
it is actually pretty good but only as good as the ai you integrate. and the api costs always rag us down
yes
Does Crew AI rely on the Assistants API or just uses the completions api?
Great work Swyx and the rest of the team :) It's so good that you made all of these talks public. A real educational gold mine! 🎓🎓
Definitely something different. Tons of values. Much practical way to boost my productivity. Please bring more of this. Would love to have the host conduct more workshops like these. 🎉
I just built a reactjs 3D model viewer for $50 in anout 6 hours with my custom coding agent using OpenAI api.The extra time and money was a manual fix and not getting the context lined up again, until I asked it to rebuild the app to last “savepoint” then it was smooth sailing after.
Is there no solution in the market for that? or just because you needed custom functionality? I see a lot of CZcamsrs showing examples of some random react U and call it a day.
@@vitalis The utility app was to test the custom agent, not that I really needed the 3D model viewer when I can just download blender for that. The agent framework on the other hand, I do need my own as there is hardly any feature stability in this space.
@@aiamfree yeah, that’s exactly my point. A lot of stuff people show actually is to test functionality or for content purposes. But I’m sure you’ve learned a lot building those agents
This is super-great stuff insightful vid... THX
I’m definitely not a programmer at all but I’ve been able to make some crazy stuff with Sonnet 3.5. Mostly simulated/procedural worlds.
Extracted the transcript from the video and fed it to the 3.5 sonnet to come up with tips discussed, here is the answer, if it helps anyone : Here is a list of tips for writing good prompts that were mentioned in the transcript, in the order they appeared: 1. Put information in XML tags to clearly separate different parts of the prompt. 2. Put instructions after the document/information, especially for long documents. 3. Use capitalization and fix grammar mistakes in prompts. 4. Be concise and offer only relevant information in prompts. 5. Specify the desired length of the response (e.g. 1-2 sentences). 6. Use examples, especially for more complex tasks or outputs. 7. Include both positive and negative examples when relevant. 8. Use "lightweight chain of thought" by having the model do some thinking/planning before generating the final output. 9. Use prefills (partially filled assistant messages) to guide the model's response format. 10. Extract relevant quotes first before summarizing to reduce hallucinations. 11. Use tags to clearly delineate different parts of the desired output. 12. Be specific about scoring criteria when asking for ratings/evaluations. 13. Distinguish between content and quality when asking for evaluations. 14. Use multi-shot examples with explanations for rating/scoring tasks. 15. Limit granularity of rating scales (e.g. 1-5 instead of 1-100). 16. Consider using chain of thought before getting log probabilities for more accurate results. 17. Move instructions closer to the bottom of the prompt for tighter following. These tips cover various aspects of prompt engineering discussed in the workshop transcript. The speaker emphasized the importance of clear formatting, providing examples, and guiding the model's thought process to achieve better results.
1 hour and fifteen minutes in, I feel like I haven't really learned a thing. Too many interruptions from the audience and too much "idk i guess" from the speaker.
Such was the nature of this format sadly enough. Too much interaction! Lol.
Joao Rules. CrewAI Rules. thats it
THX insightful
Nice sales pitch but I've been waiting for access for this since April. Still can't use it.
This guy can sell for sure. No offense. This is good. Always happy seeing developers selling their stuff.
this is fantastic
At 1:23:50 Zack says : "sorry can we we actually uh cut the questions off while I type this this prompt here" And 20 seconds later another dude asks a question while Zack is prompting = DISRESPECT 🙄
Hello. This sounds like a great approach. I just have a question about scalability up/down when using bare metal?
🎯 Key points for quick navigation: 00:00:05 *🎤 Introduction and Welcome* - Introduction by Jamie Newor, who leads the startup team at Anthropic. - Discusses the recent exciting releases and how Anthropic is helping startups grow. - Mentions specific product releases and invites audience participation. 00:03:08 *🔧 Prompt Workshop Initialization* - Transition to Zack Witten, a.k.a. "The Prompt Doctor". - Overview of the workshop format and interaction methods. - Introduction to the Slack channel for submitting prompts for live testing. 00:08:30 *💬 First Prompt Example* - Detailed walkthrough of a sample prompt regarding medication review. - Discussion on prompt formatting and usage of XML tags. - Live testing and evaluation of prompt responses in the console. 00:16:00 *❓ Prompt Optimization Techniques* - Importance of concise responses and how to enforce this in prompts. - Adjustments made to improve prompt response length and structure. - Insight into how specific wording and punctuation can impact performance. 00:20:48 *📊 Custom Examples and Prefills* - Introduction of a more complex example related to unit test reviews. - Usage of pre-fills in the context of the console for better prompt structuring. - Steps taken to adapt the prompt to diverse inputs and desired outputs. 00:24:38 *🗂️ Project Setup and Initial Issues* - Discussion of project path and local directory setup, - Setting up a JSON output and issues faced with model versions and output token limits. 00:26:46 *⚙️ Ensuring JSON Output Reliability* - Strategies for making sure models output JSON reliably, - Using assistant prefill to guide model responses. 00:29:47 *🤖 Handling Model's Output Behavior* - Techniques to ensure models do not add unwanted preamble, - Using JSON tags and stop sequences to control output. 00:32:04 *🚦 Simplifying and Testing JSON Outputs* - Using stop sequences to end sampling and avoid unnecessary explanations, - Emphasizing the use of code over prompting for certain tasks. 00:35:14 *✅ Prefill and Stop Sequences via API* - Explanation of how prefill and stop sequences work in the API, - Testing variations in instructions to see how the model responds. 00:49:57 *📝 Prompt Construction Techniques* - Constructing prompts with system roles for AI influencer persona, - Importance of multi-shot examples for better prompt outcomes, - Customizing document and tweet examples for more effective responses. 00:52:00 *📊 Examples vs. Dialogue Structure* - Discussing structuring examples in a block vs. dialogue format, - Testing revealing no significant performance difference, - Including negative examples to contrast quality. 00:55:04 *🤔 Trusting Model's Reasoning* - Distrusting post hoc reasoning from models, - Benefiting from pre-response reasoning, - Planning quality responses with Chain of Thought. 00:57:13 *📝 Example Quality vs. Quantity* - Preference for one high-quality example over multiple truncated ones, - Importance of testing for model fixation on provided examples, - Balancing example quality and document variety. 00:59:02 *🎭 Multi-Persona Roleplaying* - Using synthetic personas to simulate various user interactions, - Managing roleplay in prompts with dynamic routing logic, - Challenges and potential improvements in roleplaying prompts. 01:05:20 *🤓 Negative and Positive Prompting* - Impact of negative vs. positive phrasing on model responses, - Avoiding reverse psychology in negative prompting, - Effective use of system prompts vs. user prompts for instructions. 01:08:19 *❓ Counterexamples in Prompts* - Role of counterexamples in improving model performance, - Contextual relevance of RLHF training to counterexample inclusion, - Alternative techniques for incorporating negative examples. 01:11:10 *🧠 Steering and Control Vectors* - Exploring control vectors in negative prompting, - Comparison of steering mechanisms with traditional prompting, - Ongoing research and potential of control vectors for improved guidance. 01:12:09 *📸 Image Handling Tips* - Specific strategies for working with images in prompts, - Challenges in reading and interpreting complex images, - Using tools and techniques like zooming in and cropping for better accuracy. 01:13:57 *📷 Discussing Image Processing in Claude Models* - Higher quality images yield better results, - Avoid extraneous details in images, - Challenges with text recognition in images and downsample complexities. 01:16:08 *📝 Using Traceback Errors in Prompt Engineering* - Dumping full traceback errors into prompts effectively, - Models are improving in handling traceback errors, - Discussion on prompt engineering versus inherent model capabilities. 01:18:33 *🌐 Prompt Engineering for Translation Analysis* - Analyzing translation quality from English to Japanese, - Using Chain of Thought for better model responses, - Importance of examples in prompts for accurate translations. 01:22:21 *🔍 Improving Translation Quality Assessment* - Distinguishing between content quality and translation accuracy, - Adding detailed examples to calibration scales, - Using log probabilities for improved predictions. 01:29:02 *✨ Audience Q&A and Practical Advice* - Addressing last questions from the audience, - Reiterating the importance of structured prompts, - Final prompt demonstration covering hallucination mitigation strategies. Made with HARPA AI
🙏🏿🙏🏿🙏🏿
Efficiency is everything in AI! There are tools that can dramatically streamline your workflow.
alchemist 101
This is gold, thanks for sharing!
uf, u should be more humble
Nice !!!
Wooww this is 6 months old😢
Glad to see more prompting and console exposure from Anthropic. Think they could benefit from another session with a less experienced crowd. Felt like the crowd was so deep into the core functionality that this devolved into a fault-finding tilt focused on edge cases. Also given the experienced crowd, the event staff passing the mic need to be more judicious in how many questions they allow. All that said amazing detailed content and glad it was made public. Good Job by the presenter. Love the product
popular frameworks usually come from extracting resuable bits from a proven working production system. I don't think it's productive to try to come up with some all-encompassing framework out of nothing. I recommend AI engineers to just use your existing microservice solution, figure out what's lacking for serving LLM agents, and then derive a solution from there if actually necessary. It's quite unclear what problems Llama Agents solve that's worth the migration efforts from this presentation.
Read any Noam Chomsky book on lingustics, it is far more useful on learning how to use a language model.
how to use it?
One useful technique I've found for keeping standing instructions fresh in context: Prompt it to maintain a "focus footer." Best when initializing thread, but can be injected any time: """ Begin your next response with the following: My responses to your most recent message will always come first. At the end of each and every response, I will append the footer below. FOOTER="I am playing the role of [ROLE] for the user. We will [INTERACTION MODE] about whatever topic the user chooses. I will use a [INTERACTION APPROACH] approach in my responses, to ensure my replies are [ATTRIBUTES]. I will continue in this role until the user instructs me otherwise. I will continue appending this footer at the end each of my responses until the user tells me otherwise. """
Sort of works like an organic "system prompt" that can adapt as the chat progresses. Helpful for managing role-switching, keeping the assistant on topic/task, building a running list of findings during a protracted interaction -- any use case where when longer threads tend to allow standing instructions to deprioritize in context.
Couple of real-life examples: FOOTER="I am playing the role of a thought-partner for the user. We will discuss, debate, and refine ideas about whatever topic the user chooses. I will use a 'yes, and...' approach in my responses, to ensure my replies are positive, expansive, and elastic. I will continue in this role until the user instructs me otherwise. I will continue appending this footer at the end each of my responses until the user tells me otherwise." **FOOTER:** My role: Business analyst skilled at conducting stakeholder discovery interviews. My process: Interview the user by asking one question at a time; append a running list of key findings to the end of this footer. Objective: Understand user's business context to discover high-priority AI use-cases to test in user's business. No changes to footer scope unless user requests or explicitly confirms; on change: “SCOPE CHANGES: [CHANGES TO SCOPE].” I will append this footer verbatim to my next response. ## KEY FINDINGS
This makes a lot of sense, there definitely is a drift from your original instructions the further along the conversation goes and I've been thinking about similar ways to address this but this seems simpler
You don’t need to prompt the AI to append the footer. You can simply append your footer to the assistants previous response when you pass back the message chain. That way you still get the footer but save tokens.
@@aravardanyan23 True, if you’re using the API workbench or another mode where editing assistant responses is an option. The prompt approach is more for users on a consumer/prosumer web front end, where you can’t.
I know, I’ll definitely learn something. I ❤ that
The thing with SVD is that you need to understand everything in Linear Algebra to really "get" it. I guess math in general is sort of like this, but in the context of SVD, this is especially true. I remember when I finally understood it, it felt like the most epic moment ever. But yeah, I think Linear Algebra should be mandatory for all CS degrees. My college doesn't require it, just calc and discrete math. Which is a shame, since linear algebra has been the most applicable of all the maths I've studied (as a programmer). Calc comes in at a really close second tho, the derivative is kinda OP.
NVIDIA has hired CIA agents to make sure this technology is not reaching in hands of public. Be safe sir !😝
Only output JSON or you will be fired!
seems like a lot of talk, AI Development has been lacking lately. Doing a lotta talk with no walk..
Are you high