Efficiently Build Custom LLMs on Your Data with Open-source Ludwig

Large language models (LLMs) have grabbed the attention of AI-minded organizations thanks to their ease of use and broad applications. While commercial offerings like OpenAI’s GPT4 enable teams to rapidly prototype with LLMs, most organizations want to build custom models tuned on proprietary data. Unfortunately, this complex endeavor requires deep expertise in model training and infrastructure management.
Open-sourced by Uber in 2019, Ludwig has empowered 1000s of engineers and data scientists to build state-of-the-art ML-powered applications through its low-code, declarative interface. That same develop-friendly approach is now being extended to the world of LLMs.
With Ludwig v0.8, we’ve released the first open-source, low-code framework optimized for efficiently building custom LLMs on private data. In this hands-on webinar we were joined by the original author of Ludwig, Piero Molino, and lead maintainer, Arnav Garg.
Topics covered:
• About the motivation and design of Ludwig, the open-source declarative framework fro building custom ML and LLM models
• How integrations with Deepspeed and parameter-efficient fine-tuning techniques make training LLMs with Ludwig fast and easy
• How to declaratively fine-tune open-source LLMs in a few simple commands
•How to use Ludwig's capabilities for prompt templating and in-context learning for your tasks
Here is the free fine-tuning notebook from the session: pbase.ai/3YDMrcz.
Here are the webinar slides: pbase.ai/44cDNTp.
If you're interested in trying the new capabilities of Ludwig v0.8 visit: ludwig.ai/.
If you'd like to instantly deploy and customize LLMs on state-of-the-art infrastructure, try Predibase for free: predibase.com/....

[@gabrielecastaldi1618]

Excellent explanation of Ludwig features and potentials with hands-on assessment of alternative approaches to optimize the output. I look forward to new compelling applications in various industrial fields.

[@prashantjain2023]

I tried to follow the colab and I was able to fine-tune LLAMA2-7b on my own dataset. After fine-tuning, I'm trying to load the fine-tuned model on my VM (30GB RAM and GPU T4) but my system keep crashing due to OOM. Is there any other tested way to load the fine-tuned model binaries with ludwig? Would you be able to share code / video for that?

[@user-ze3hi6rf5w]

how to get access to the notebook?