A local AI Voice Assistant using Granite-7B, Ollama, InstructLab and AlwaysReddy

I had an idea today to be able to chat with my local LLM that is powered by the granite foundation model from the InstructLab Project. Luckily, I was able to find an open source project called AlwaysReddy that fit the bill. In this video, I go from zero to chatting with the granite open source model in under 10 minutes. ALL IN REAL TIME with no cuts or edits.
github.com/instructlab
github.com/ILikeAI/AlwaysReddy
ollama.com/download
huggingface.co/instructlab/gr...

[@HarishPillay]

Do please do another demo on Fedora.

[@billychen2103]

awesome! thanks for sharing!

[@jerryyuan3958]

How many GPU on your window? Whisper also need GPU if you want to do a quick convert text to audio.

[@gshipley21]

Oh sorry - I didn't know that. I do have a RTX 4090 in my windows machine.