Video

Absolutely! That's one of the immediate applications of such a stack. Eventually, you can achieve this with a true multimodal LLM that understands audio, image, and text natively. Right now, it makes sense to use Whisper to transcribe audio to text and then use an LLM to generate a response from the transcribed text. But imagine a multimodal model where you pass audio and text together and directly get text in return. Currently, models like GPT-4 or Gemini Pro support image and text, but soon they'll release models that support audio as well.

I will work on that. Trying to have a version without model dependency as well

You can send multiple links, so first crawl the main page, then get links and send them again. However soon I will release the ability to se the depth and get a cool result for that

Agree! I actually made another version using an open-source model for Fast Whisper, running a server on a Colab L4, and it is surprisingly fast. I will make a video about it soon.

😄😄 Perhaps, I should make some content for mix of flute and AI! You gave me an idea, I try to come up with something

You can load the GGFU version over there. Anyway, I'm working on a research project, and we're close to releasing what might be the smallest ever language model solely for function calls. Stay tuned!

Hahahaha!! Ok, ok, message received

Thanks! Will do!

Result is an object like this: class CrawlResult(BaseModel): url: str html: str success: bool cleaned_html: str = None markdown: str = None extracted_content: str = None metadata: dict = None error_message: str = None So you can access using this property (cleaned_html, markdown, extracted_content), or dump the model into a python dictionary using "result.model_dump()`

Yes, L4 would be good enough

de nada.

Haha you right, I will consider that for the next video, stay tuned

It should be by now. Please check again

Yes, it should support that. I'm compiling everything into a library to provide a seamless function call experience across any LLM. I'll be releasing it soon.

@@unclecode788 what’s your discord? Working on a few major project that would be perfect for this

@@unclecode788 I am working on some large projects that will be perfect for this. Please let me know when you release this.

Thanks for your interest, working on it, meanwhile you can follow me on X where I shared more of these contents @unclecode

You are very right, I have to do that. Btw I'm developing a new library focused solely on tool (function) calls. I've expanded it to include any LLM, like Ollama, and it's working like a charm. I'm also incorporating "tools binding." I'll be releasing it soon and will record a video to showcase it. I definitely make it available in JS as well.

@@unclecode788 I have one question, Is openAI, groq and other providers using the same approach for function calling by using phidata/webLLM or they are doing something different like their own in-house library under the hood.

@@unclecode788 Curiously waiting for that? Do OpenAI and others use WebLLM and PhiData for function calling, or are they using their own secret library behind the wall?

There are two ways to use Copilot: the code completion feature, which I love, and Copilot Chat, which I rarely use because ChatGPT works better for me, though Copilot Chat does have extra tools for interacting with your code. Honestly, for the last 40 days, I've switched to Claude Opus, and it's been the best experience so far. Regarding Gemini model series, I'm not sure why, but I couldn't get the feelings I am looking for. And In real work, I use many models at the same time, including Mistral chat and a few open sources with my Ollama client. Sometimes, one answers a specific question much better, and I've gotten used to their personalities and know which model to use for what task.

Exactly! Expecting a model trained to generate tokens to deliver the perfect answer on the first try is a misunderstanding of how transformers work. Allowing the model to "think" by generating more content can refine the attention matrix, preparing it to statistically converge on the desired conclusion. This approach prevents forcing an output prematurely when the model isn't ready, leading to more accurate and reliable responses. Generating tokens is essentially a sampling problem from the word distribution. If you need to adjust the distribution, you have two main options: fine-tuning the model or letting the model navigate towards your desired output while maintaining the integrity of the original sampling distribution. This approach ensures that modifications are more natural and aligned with the model's learned patterns.

Absolutely, while speed doesn't inherently affect the quality of Mixtral, it does open up numerous possibilities with prompt engineering. For instance, self-ranking involves generating multiple responses to a single query and then using an LLM to select the top K answers to combine. Here, speed allows you to leverage a model like Mixtral-or any model-to its fullest potential. You can experiment with techniques like the tree of thought among others. In my new library, I'm compiling these "techniques of thought" to maximize the advantages offered by Groq's speed. This is where speed truly makes a difference.

Yes, I've primarily used Groq, the Mistral model, but I'm developing a new library focused solely on tool (function) calls. I've expanded it to include any LLM, like Ollama, and it's working like a charm. I'm also incorporating "tools binding." I'll be releasing it soon and will record a video to showcase it.

I'm developing a new library tailored specifically for tool (function) calls. This focus stems from my belief that function calls are a crucial component of AI as agents, providing a dynamic framework where LLMs can not only execute but also generate their own functions dynamically. This capability is essential for adaptive, intelligent systems. I've expanded this functionality to work seamlessly with any LLM, including testing with Ollama, and the results are promising. I'm introducing "tools binding" to enhance this integration. I'll be releasing this library soon and will create a detailed video to explain its features and the concept behind focusing on tool calls.

Glad you enjoyed it!