Video

Wow! Great stuff!

so what was the reason for using softmax? what would be otherwise had softmax not been used?

This is exactly what i needed, when I needed it. Big props!

Ailen Degeneres

Would be nice to have a newer version of CommandR / CommandR+ with Flash Attention and maybe a model in the middle of the size round (40-50b)

Looks promising too bad its python.

This guy doesn't understand what is quantization and also transistor number does not play a role in efficiency necessarily. The main issue is GPU sm design and ALU units😅

Lopez Mark Jones Angela Rodriguez Jason

You can say the iteration was next level.😂😂😊

So it's another RAG-fueld chatbot then.

I think it can be applied to other fields to keep the documentation up to date and for easy access to the documentation

This is what modern AI looks like 💜

😂 10% is rework .. comes from someone born post COVID .. human beings using constructed structures since the sun came to light the earth 😂 Also we give you 13thousend links to choose from and if your guys chooses the wrong instructions within the webapp .. it will only cost you 5 unjustified injuries and a couple million dollar lawsuits but hey thats your responsibility now AI cannot fix everything. Btw good luck with OSHA 😊😂

Mac app please

Finally

this guy should be a comedian

I found this super interesting and informative.

it is actually pretty good but only as good as the ai you integrate. and the api costs always rag us down

yes

Does Crew AI rely on the Assistants API or just uses the completions api?

Great work Swyx and the rest of the team :) It's so good that you made all of these talks public. A real educational gold mine! 🎓🎓

Definitely something different. Tons of values. Much practical way to boost my productivity. Please bring more of this. Would love to have the host conduct more workshops like these. 🎉

I just built a reactjs 3D model viewer for $50 in anout 6 hours with my custom coding agent using OpenAI api.The extra time and money was a manual fix and not getting the context lined up again, until I asked it to rebuild the app to last “savepoint” then it was smooth sailing after.

This is super-great stuff insightful vid... THX

I’m definitely not a programmer at all but I’ve been able to make some crazy stuff with Sonnet 3.5. Mostly simulated/procedural worlds.

Extracted the transcript from the video and fed it to the 3.5 sonnet to come up with tips discussed, here is the answer, if it helps anyone : Here is a list of tips for writing good prompts that were mentioned in the transcript, in the order they appeared: 1. Put information in XML tags to clearly separate different parts of the prompt. 2. Put instructions after the document/information, especially for long documents. 3. Use capitalization and fix grammar mistakes in prompts. 4. Be concise and offer only relevant information in prompts. 5. Specify the desired length of the response (e.g. 1-2 sentences). 6. Use examples, especially for more complex tasks or outputs. 7. Include both positive and negative examples when relevant. 8. Use "lightweight chain of thought" by having the model do some thinking/planning before generating the final output. 9. Use prefills (partially filled assistant messages) to guide the model's response format. 10. Extract relevant quotes first before summarizing to reduce hallucinations. 11. Use tags to clearly delineate different parts of the desired output. 12. Be specific about scoring criteria when asking for ratings/evaluations. 13. Distinguish between content and quality when asking for evaluations. 14. Use multi-shot examples with explanations for rating/scoring tasks. 15. Limit granularity of rating scales (e.g. 1-5 instead of 1-100). 16. Consider using chain of thought before getting log probabilities for more accurate results. 17. Move instructions closer to the bottom of the prompt for tighter following. These tips cover various aspects of prompt engineering discussed in the workshop transcript. The speaker emphasized the importance of clear formatting, providing examples, and guiding the model's thought process to achieve better results.

1 hour and fifteen minutes in, I feel like I haven't really learned a thing. Too many interruptions from the audience and too much "idk i guess" from the speaker.

Joao Rules. CrewAI Rules. thats it

THX insightful

Nice sales pitch but I've been waiting for access for this since April. Still can't use it.

This guy can sell for sure. No offense. This is good. Always happy seeing developers selling their stuff.

this is fantastic

At 1:23:50 Zack says : "sorry can we we actually uh cut the questions off while I type this this prompt here" And 20 seconds later another dude asks a question while Zack is prompting = DISRESPECT 🙄

Hello. This sounds like a great approach. I just have a question about scalability up/down when using bare metal?

🎯 Key points for quick navigation: 00:00:05 *🎤 Introduction and Welcome* - Introduction by Jamie Newor, who leads the startup team at Anthropic. - Discusses the recent exciting releases and how Anthropic is helping startups grow. - Mentions specific product releases and invites audience participation. 00:03:08 *🔧 Prompt Workshop Initialization* - Transition to Zack Witten, a.k.a. "The Prompt Doctor". - Overview of the workshop format and interaction methods. - Introduction to the Slack channel for submitting prompts for live testing. 00:08:30 *💬 First Prompt Example* - Detailed walkthrough of a sample prompt regarding medication review. - Discussion on prompt formatting and usage of XML tags. - Live testing and evaluation of prompt responses in the console. 00:16:00 *❓ Prompt Optimization Techniques* - Importance of concise responses and how to enforce this in prompts. - Adjustments made to improve prompt response length and structure. - Insight into how specific wording and punctuation can impact performance. 00:20:48 *📊 Custom Examples and Prefills* - Introduction of a more complex example related to unit test reviews. - Usage of pre-fills in the context of the console for better prompt structuring. - Steps taken to adapt the prompt to diverse inputs and desired outputs. 00:24:38 *🗂️ Project Setup and Initial Issues* - Discussion of project path and local directory setup, - Setting up a JSON output and issues faced with model versions and output token limits. 00:26:46 *⚙️ Ensuring JSON Output Reliability* - Strategies for making sure models output JSON reliably, - Using assistant prefill to guide model responses. 00:29:47 *🤖 Handling Model's Output Behavior* - Techniques to ensure models do not add unwanted preamble, - Using JSON tags and stop sequences to control output. 00:32:04 *🚦 Simplifying and Testing JSON Outputs* - Using stop sequences to end sampling and avoid unnecessary explanations, - Emphasizing the use of code over prompting for certain tasks. 00:35:14 *✅ Prefill and Stop Sequences via API* - Explanation of how prefill and stop sequences work in the API, - Testing variations in instructions to see how the model responds. 00:49:57 *📝 Prompt Construction Techniques* - Constructing prompts with system roles for AI influencer persona, - Importance of multi-shot examples for better prompt outcomes, - Customizing document and tweet examples for more effective responses. 00:52:00 *📊 Examples vs. Dialogue Structure* - Discussing structuring examples in a block vs. dialogue format, - Testing revealing no significant performance difference, - Including negative examples to contrast quality. 00:55:04 *🤔 Trusting Model's Reasoning* - Distrusting post hoc reasoning from models, - Benefiting from pre-response reasoning, - Planning quality responses with Chain of Thought. 00:57:13 *📝 Example Quality vs. Quantity* - Preference for one high-quality example over multiple truncated ones, - Importance of testing for model fixation on provided examples, - Balancing example quality and document variety. 00:59:02 *🎭 Multi-Persona Roleplaying* - Using synthetic personas to simulate various user interactions, - Managing roleplay in prompts with dynamic routing logic, - Challenges and potential improvements in roleplaying prompts. 01:05:20 *🤓 Negative and Positive Prompting* - Impact of negative vs. positive phrasing on model responses, - Avoiding reverse psychology in negative prompting, - Effective use of system prompts vs. user prompts for instructions. 01:08:19 *❓ Counterexamples in Prompts* - Role of counterexamples in improving model performance, - Contextual relevance of RLHF training to counterexample inclusion, - Alternative techniques for incorporating negative examples. 01:11:10 *🧠 Steering and Control Vectors* - Exploring control vectors in negative prompting, - Comparison of steering mechanisms with traditional prompting, - Ongoing research and potential of control vectors for improved guidance. 01:12:09 *📸 Image Handling Tips* - Specific strategies for working with images in prompts, - Challenges in reading and interpreting complex images, - Using tools and techniques like zooming in and cropping for better accuracy. 01:13:57 *📷 Discussing Image Processing in Claude Models* - Higher quality images yield better results, - Avoid extraneous details in images, - Challenges with text recognition in images and downsample complexities. 01:16:08 *📝 Using Traceback Errors in Prompt Engineering* - Dumping full traceback errors into prompts effectively, - Models are improving in handling traceback errors, - Discussion on prompt engineering versus inherent model capabilities. 01:18:33 *🌐 Prompt Engineering for Translation Analysis* - Analyzing translation quality from English to Japanese, - Using Chain of Thought for better model responses, - Importance of examples in prompts for accurate translations. 01:22:21 *🔍 Improving Translation Quality Assessment* - Distinguishing between content quality and translation accuracy, - Adding detailed examples to calibration scales, - Using log probabilities for improved predictions. 01:29:02 *✨ Audience Q&A and Practical Advice* - Addressing last questions from the audience, - Reiterating the importance of structured prompts, - Final prompt demonstration covering hallucination mitigation strategies. Made with HARPA AI

Efficiency is everything in AI! There are tools that can dramatically streamline your workflow.

alchemist 101

This is gold, thanks for sharing!

uf, u should be more humble

Nice !!!

Wooww this is 6 months old😢

Glad to see more prompting and console exposure from Anthropic. Think they could benefit from another session with a less experienced crowd. Felt like the crowd was so deep into the core functionality that this devolved into a fault-finding tilt focused on edge cases. Also given the experienced crowd, the event staff passing the mic need to be more judicious in how many questions they allow. All that said amazing detailed content and glad it was made public. Good Job by the presenter. Love the product

popular frameworks usually come from extracting resuable bits from a proven working production system. I don't think it's productive to try to come up with some all-encompassing framework out of nothing. I recommend AI engineers to just use your existing microservice solution, figure out what's lacking for serving LLM agents, and then derive a solution from there if actually necessary. It's quite unclear what problems Llama Agents solve that's worth the migration efforts from this presentation.

Read any Noam Chomsky book on lingustics, it is far more useful on learning how to use a language model.

One useful technique I've found for keeping standing instructions fresh in context: Prompt it to maintain a "focus footer." Best when initializing thread, but can be injected any time: """ Begin your next response with the following: My responses to your most recent message will always come first. At the end of each and every response, I will append the footer below. FOOTER="I am playing the role of [ROLE] for the user. We will [INTERACTION MODE] about whatever topic the user chooses. I will use a [INTERACTION APPROACH] approach in my responses, to ensure my replies are [ATTRIBUTES]. I will continue in this role until the user instructs me otherwise. I will continue appending this footer at the end each of my responses until the user tells me otherwise. """

I know, I’ll definitely learn something. I ❤ that

The thing with SVD is that you need to understand everything in Linear Algebra to really "get" it. I guess math in general is sort of like this, but in the context of SVD, this is especially true. I remember when I finally understood it, it felt like the most epic moment ever. But yeah, I think Linear Algebra should be mandatory for all CS degrees. My college doesn't require it, just calc and discrete math. Which is a shame, since linear algebra has been the most applicable of all the maths I've studied (as a programmer). Calc comes in at a really close second tho, the derivative is kinda OP.

NVIDIA has hired CIA agents to make sure this technology is not reaching in hands of public. Be safe sir !😝

Only output JSON or you will be fired!

seems like a lot of talk, AI Development has been lacking lately. Doing a lotta talk with no walk..