Is Open Webui The Ultimate Ollama Frontend Choice?
Vložit
- čas přidán 15. 06. 2024
- On 04/25/2024 I did a livestream where I made this video...and here is the final product. It’s a look at one of the most used frontends for Ollama. It's not perfect, but there is a lot to like.
Someone clarified something that I missed... It seems that you can specify the model to use in the prompt using the @ sign. This is great. They should highlight that in the docs and make it a bit more discoverable.
Be sure to sign up to my monthly newsletter at technovangelist.substack.com/...
Or if you feel like it there is a patreon at / technovangelist
00:00 Introduction
02:47 Getting Started with Open WebUI
04:01 Let's setup Open WebUI
04:51 How Often is Open WebUI updated
05:16 The Actual Install Process
06:52 The Parts of the UI
07:17 Setting the Settings
09:59 Connect to Multiple Models
11:19 Working with Prompts
13:04 Talking to a Website
13:36 Talking to Documents
14:54 What do you think? - Věda a technologie
Wow... this is the kind of detailed, helpful and to the point app review we should see more of from people. Thanks!
Good review, I have been using open-webui for a while and learned a bunch of new stuff, thanks. It appears to get better all the time which should continue especially after you've uncovered areas for improvement. BTW, I like the new chat archive feature.
Thanks for this Matt, very easy to work with this tool!
Have my subscription Matt. I like your highly clear and structured way of speaking.
Awesome! Thank you!
hi Matt, amazing content. Thank you for sharing your thoughts with us and chatting with me during your stream.
I really appreciated this video. I've only been using this tool for about a week and was really excited to get answers to all of the confounding and non-working features I kept running into...only to find out that they're actually confounding or non-working. 😂
Good stuff and one of the more useful things i have watched in a while debating uis
Loving your videos and surely will give Open WebUI a try. Keep it going on such amazing work with these content.
The new required login doesn't go to any remote site, it stays on the local computer. This way multiple users can store chat history and settings. I agree that it should be optional, but at least it's local.
Correct. It’s for access to openwebui. But it’s intended as a feature for hosting it on another system online.
Great video enjoy your content helpful. I have a question about agent's how can i contact you private
I am on the ollama discord. Or you can find me on twitter. Same name as this channel
Thanks for the info. Love the videos
Love your review, sir. I came up with exactly same notes and frustrations as you did 😂
Great ideas (as usual 😂), cannot wait for the whole series…
Thanks for teaching me how to get started. The only downside of Ollama is that it's unable to integrate with HuggingFace, but it is able to import the raw GUF files or whatever they are called, by manually filling out a Modelfile. It's amazing.
I basically fill out FROM, TEMPLATE, PARAMETER context size and PARAMETER stop words. Then import it. The result is perfect.
I even imported inside a Docker environment. Just place the image folder inside the mounted colume path. Then use "bash" inside the container and then you can do the import.
When you set additional hosts in the 'Connections" settings, they will act as redundancy assuming you have the same models installed on each host. So if I serve to multiple users, all using the same model at the same time, it will queue up requests to the current unoccupied host, in sequence. I've tested it locally with 3 separate hosts, and it works quite well. BTW thank you for the great video!
I was wondering whether a model like Llama3:70b or Llava could run on one PC with a lot of hardware resources but on a seperate PC you could run a light model like Phi 3. Then... I could turn off the powerful pc at night /weekends to save power and the chats model could default to Phi3? Maybe this is what it could be used for.
After all when the powerful PC is off Open web UI wouldn't know things like Llava even exist... Maybe open web UI would need a restart to notice things had changed? What are your thoughts?
always your videos so informative thank you 🥰
my main language is Arabic and i am weak in English but i understand you without using any subtitles
Love your videos, mate. Even if we are on opposite sides of the fence re. Dark mode! Cheers.
thank u...cool presentation style my man..#thumbs👍🏾
Thanks for the detailed description. Opens up a lot of possibilities, what I am missing on command line is a history which this provides... I also discovered Enchanted desktop client for Mac which does this as well, so this is easier to install.
Hey, this is great, Matt! :D Having you try and review all the Ollama frontends will be super useful! I'm really looking forward to the rest of the series! :D
I currently use Open WebUI as well as the Ollama CLI, and I completely agree with the pros and cons you outlined.
By the way, could you tell me where to find the comparative chart you mentioned in the video? I couldn’t find it on your website, but I'm really interested in having a look at it :)
There’s a tiny button after the response that gives you data on tokens per second etc, I love that about this particular UI, easy to compare speeds
Yes that is nice. It’s pretty interesting to see how much they have been able to replicate from the cli
BTW awesome tutorial - I was using the olv version and I didnt know the prompts, web scraper, document and the voice was available. Thank you for sharing.
I how they will fix the whisper TTS soon as the generic windows TTS is so annoying .. sounds like we are year 2000 :)
I use open webui some but also use the command line. I'm not familiar enough with advanced usage of either, though. I appreciated this video and am looking forward to learning more. At this point, I'm just a sponge. Thanks!
i usually use anythingllm but after you explain open web Ui i will try it
The user login default worked well for me - at a company that can’t use cloud based LLMs for security reasons, the default workflow allows you to immediately install this tool and share it with regular users (who don’t know what a command line is). But I agree maybe there ought to be a “dev” switch that turns it off.
Really great video, looking forward to more.
I’ve been looking at this and other tools and the one thing I find elusive is the ability to fine tune a model with desired prompt/inference examples to help fast track the usefulness of a newly downloaded model. Including this in your reviews would be amazing if possible.
Great content. Thank you
Glad you liked it!
What would make a great addition to this , would be a RAG backed to load bulk documents . The way to do this would simply mount an external volume to the docker image. Then have a file watcher to load up any new document which are added to the external directory . All documents are avaiable to all users of Web-UI for RAG use.
Or having a RAG from a webcrawl where the user just puts in the starting URL and domain
User management is actually a good thing if you want to share your LLM among other ppl without giving them an ability to mess with your stuff.
Hi Matt, thanks for your detail video.
Do you recommend another WebUI tool?
It would be great if you can make a video about deploying the model into the could and use its endpoints see how api friendly it is
It would be nice if this were an assistant with a wake word. If there were a page to add actions, that would be terrific.
Once again a great video with super easy explanation, thank you soo much!
I use Open WebUI every day and I love it! I love how it formats results nicely and stores the conversations for easy reference. The login page works with my password manager so it's not that inconvenient and I feel better that my conversations are kept private this way because privacy is such a huge motivation for running a private AI after all.
But the login provides no privacy on your local machine. Maybe if it were hosted on an external server.
The login for me has use. I host this on a server. I have the admin account with all the trial models and so on and the user account which only has access to the one or two models that just work. As a result, when my wife wants to use it or when I want to just get stuff done, the user accounts are great. When I am fiddling and don't care there are loads of available models and duplicates with slightly different names, then admin it is.
I do host it online but I do not treat the login page as any degree of security, just as a way of segregating functionality.
Very captivating explanation, there is others like I would like you to review is LLM studios and anythingLLM as a suggestion. Thanks 🙏
Thanks Matt, what about using the spew hint capabilities. Could you go through that?
The CLI is less handy because it doesn't record the chat and editing a prompt in CLI id tough, but in general I agree with you. Especially the custom model would benefit of additional functionalities like using the unlsoth framework for fine-tuning, then saving, benchmarking and loading custom models, it is very compatible with ollama's local philosophy,
I wouldn't go as far as to say that this is even close to a 1-1 command line tool.
I use both regularly, it's nice to be able to easily setup a DB for file storage and to have a smooth and extremely easy way to integrate your files into your chats. It's also pretty nice to have an easy interface for model building rather than needing to build out your models in text files and then create them from that.
It's also pretty nice to be able to provide feedback to your models in a concise way using the thumbs up and down feature, it's especially noticeable if you test with local files and repeatedly give the model poor praises when it answers correctly and visa versa, the model reflects the mixed judgment and starts to act foolish.
Additionally, it makes it very easy to serve models to friends, family, and in a work environment.
Wow, that's a lot of models. Which ones are your favorites and what do you use them for?
Fully agree. Its an all-new technology for many of us and some terms used aren't that obvious. So, good help text and tool-tips are as important as the feature itself. Having a good UI is great. But, as far as understood this video, it is not *just* a UI. Built-in RAG, vector database etc. means there's more "stuff" than just the UI itself. It is needed of course, but it is rather a full blown frontend application than just a UI. Things grow over time. ;-)
i use it all the time. i use it as apposed to faster services for privacy reasons. i serve it from home and have it sitting behind a reverse proxy server so im able to reach it from a FQDN. it suits me well :D
Great video and information. Thank you!
I envision a chat app with a tree-like structure, including a main trunk and collapsible branches for topics created by users. You could invite and select bots, similar to users, that are defined with Flowise with various capabilities defined by their flows. A respond from a chatbot would be triggered by selecting a bot or activating a checkbox when sending a message. Additionally, I’d like to have a Telegram client interface in a specific topic / branch and include SST/TTS functionalities. 🙂 (All I need is a seed investor and a engineer to do all the work. 😋)
Thanks for the detailed video. I am trying to create a chat with voice. Something like amazon alexa. Can you please create a video around it ?
I am wondering if there is a way to apply a white label style updating to the UI. Can you recommend customization for those who want to demo LLM centered ideas using Ollama UI? Perhaps an alternative from end with similar features?
I deployed open webui on my kubernetes cluster and I am pretty happy with it. It makes it easy to test some LLMs and compare their output. I wish one could add langchain code and select that as a model in the dropdown. Then it would be easy to integrate your own RAG/agent pipeline.
Thank you for your videos! Your content is awesome!
Awesome to hear that you've successfully deployed Open WebUI on your Kubernetes cluster, are enjoying using it to test and compare LLM outputs! We appreciate your feedback and enthusiasm for the project.
We love to hear from our users about their ideas and suggestions. In fact, we've had similar requests to yours for a while and we absolutely plan to address them. While we haven't had the bandwidth to implement these features yet, we're excited to know that there's continued interest in this direction.
If you or anyone else in the community is interested in contributing to Open WebUI, we'd be happy to see pull requests for these features! Even if it's not directly related to Langchain integration, any PRs or answers to questions in our community can help free up time for our developers to focus on bigger features.
Thanks again for your kind words and for being part of the Open WebUI community!
Hey! Kubernetes for this is a great idea specially because the time it takes for Ollama to switch models before give back the response. Would you share your kubectl command?
thanks @W1ldTangent for that reply. I look forward to seeing open webui progress over each of the releases. Its amazing to see how far it has come.
I was using Lm Studio with Anything LLM... After seeing this video I think is time to change...
I like that color scheme though
-d is actually deatacched
‘docker run -help’
-d, --detach Run container in background and print container ID
I like this tool. For us people that maybe arent as comfortable with code it makes things easier. If the whole point of these open source models is to open LLMs to as many people as possible then these tools are needed. If the developers see this I'll throw one idea out there. Start people off by having them pick a model if if its a small model and have a help system that can run off of that. I actually dont get why more people arent doing that already. You just need structured documentation to make it that even the really small models can work with them. If someone doesnt understand something it would be really simple to just have a question mark button that they can click which the person can chat with. They could even go as far as having features requests or bug reports that use a similar system. On the developers side they can take in that data and use a larger model to do more processing on it to find common themes which would make it easy to prioritize everything.
I've tried setting up several different RAGs. In most cases, the rosey docs don't capture the snagging issues I have ran into. I can't help but feel we're in an early days state, and that in a few months RAGs will evolve. Right now, I'm kinda backing away from investing further time as they only work partially, and in document, image, and sound handling - there is ... work to do :/
You leave my precious dark mode alone, you.. you meanie! 🙃 I use openwebui and I agree, why do we have to sign in and what is that modelfiles area for? I have not tried other addons yet, tho, but I am about to, which is part of why I watched this video. So. keep going through that addons list! Excellent video!
I’ll move on to the other user interfaces. My goal is to see if there is one that improves on the built in cli
What are the other best alternatives with agent and tools options ?
Highly appreciated!
User login is a pain if you’re on your own but if you have a family using it then it’s good to have your conversations stored per user. Many users will run ollama and a UI like this on a powerful machine in a cupboard for example then access it via phones or laptops from the couch. Then if you have 3 or 4 of you then user management becomes almost a necessity. I agree though it should be optional.
Optional is the key word
Plus if with a password your kids can see all your conversations by looking at the unsecured information that’s stored as plaintext.
Thanks for the Video. I use this tool regularly but I did not cover all the features you mentioned- time to explore them 😃
The document chat for me feels a little bit less configurable than I would like to use it, e.g specific Text Splitters, working with external vector stores but maybe some of the features will be added sometime. I am very impressed by the release frequency of these guys!
Just had a thought - does the Ollama server already include a login management system? It’d be great if it could handle user credentials similar to Git, allowing specific access based on rules. For example, certain users could access specific models, or there might be usage restrictions. This would make it so much easier to deploy Ollama as an offline LLM service for small businesses. Not sure if this feature exists already, but if not, it could be a cool addition. By the way, awesome project! Really helpful for deploying LLMs locally. 🚀👊
No it doesn’t. It’s designed to be the best way to run models locally on your own hardware. Some folks are hosting solutions using ollama but they need to come up with that authentication and authorization system on their own. There are lots of tools for that depending on the specific needs of the project. In fact there are many large companies only focused on that part and none of them can provide all the options some folks want.
Open WebUI is fantastic, but I agree some features need refinement
Yes ibused it using docker, but why when i tried to stop the containers the port 8080 are still used for the ui?
Yes it is
FWIW, I think the ModelFiles section is the most powerful part of Open WebUI.
Uh oh, not a fan then?
So it’s not just me who struggles with some of these options. The OpenAI API key is not properly saved between restarts, something that drove me nuts.
For local hosting ollama UI, AnythingLLM is better for RAG use case, but Open WebUI offers a closer UX to ChatGPT interface.
Does AnythingLLM have an API endpoint for prompting with RAG functionality?
how can you customize the outpu length so that it can write longer responses
PrivateGPT as UI?
It's thanks to PrivateGPT that I learned about Ollama, it works pretty well on my Jetson Xavier AGX 32GB, not a simple task due to ARM64+CUDA.
The only thing that is missing for me when it comes to the web ui is
A) do a sequential websearch (i.e. google stuff, if condition unsatisfied, google more, integrate into chromadb)
B) digest my pdf data folder (e.g. a list of pdf publications) and store in DB. this could also be done in cli
Those things can’t be done in the cli as is but this webui doesn’t really do those things all that well either.
How to get webui connect directly with lan (Ollama o pc, webui on nas, access from iPad? I tried tunneling but text refresh is slow.
Is there a way to pull models from hugging face to open webs ui?
I don’t want generic models, they are often too censored to answer questions.
My big hop in open webui was the capability to call google search api, but I didn't find it here.
How good is the rag capability with open webui?
My use case involves querying email archives, so it is crucial that the documents are not sent to external servers. I used the sentence-transformers/all-MiniLM-L6-v2 as the embedding model, and I believe that the documents I added are not sent to outside servers. I found GPT-4o much better than the models in Chat. My question is: will my email archives be exposed to external servers, or is only the question in my chat sent to OpenAI?
You embed to add them to a vector database so that you can find the most appropriate email to ask a question against. Then the email gets sent in plaintext with the question to whichever model you are using. If that’s going to be OpenAI, you are am sending the emails there. There is no way around this
@@technovangelist Thanks, Matt, for the clarification.
How does this compare with LM Studio using AnythingLLM?
You can actually use @ to interact with a different model.
And I also find modelfile a bit interesting way to override model's default configuration.
If it’s not documented it doesn’t exist
I see the docs have been updated to include this. that’s great. Its not everything I was mentioning, but it’s a good part of it. The modelfile is the key part of ollama that makes it amazing, but I didn't see any improvement on the basics in open webui
Just hope we get to see function calling in it soon!
Great overview, thanks! I hope you will consider reviewing several of these. I use textgenwebui right now, but the options and processes can be overwhelming. They do have the most options for hardware tuned models, GPTQ, etc. A review and explainer for this tool would be appreciated.
Oobabooga is an alternative to ollama rather than open web ui.
I would have like more info about system prompt at 7:35
Funnily enough i specifically downloaded this UI because it supported Automatic111.
AFAIK the sign in is local to your server, so it's mostly "pretend sign in".
I love open webui. It is for sure the best. It even has some support for images if you don’t want to go to say the native automatic1111 webui
how does this compare to AnythingLLM? It seems to have the connectivity to Ollama done in east interface
So far there are a lot of starts of UI's but none of them are very good. It's still very early. Overall so far, Open WebUI seems far more polished and complete than AnythingLLM but maybe that will change.
I think it would be better to have the 1st ollama web ui account be admin and have better user management and generation of local api keys also would be awesome so security is there from step 1 in case its ever in production in future!
Yes the user management is a bit lackluster not really providing much security and really only offering a little speed bump. So make it optional and then for folks that want the security offer it in a real way.
There are no models listed by default after web ui docker installation. Models installed locally are not shown unless one install all those models again on docker instance. Are there any easy way to install Ollama and pull the models instead of repeating what was already done locally on the system. Am I missing something?
From the comment it sounds like you installed ollama on docker in addition to the host. You need only one
@@technovangelistfor some reason it works on my windows pc with GPU and not on Intel Mac. The UI / dashboard menu is also different in Mac. Are there any additional settings to be done for Mac.
Nothing needed to get it to work on an Apple Silicon Mac, but it won't use the GPU on an Intel Mac. There is no way to enable that.
SIr can you make guide to make ollama on homalab server and access it from windows client ot linux or mac in other with sam network
Last time I tried RAG from a big PDF in Open WebUI it did a terrible job, while Cheshire Cat did a good job with the same PDF. I tried also BionicGPT, bigAGI and chatd and Cheshire Cat was the clear winner in RAG. Besides that its plugin system offers many functionalities and its ability to delete specific memories is something I haven't seen anywhere else. I think it deserves a try, it seems a joke but it's not
Open WebUI is the way it is because of how Ollama is designed and the intention the make copy of the ChatGPT UI, which was at least so successful that I confused it once with one of ChatGPT. If you worked with Ollama more on the command line, it I think it's pretty obvious what the modelfile section is about. Ollama annoyingly mixes-up modeles and weights. So it took me a while that an "Ollama" model is just a parameter file for a the files that contain the weights, that thing that we usually call a model. Except that the very first model file is kinda tied to the weights but you can create as many more model files as you want. I don't know if there is a set of settings already included with a GGUF file or if Ollama just assumes certain defaults as many modelfiles you can download from the Ollama site are mostly empty. In some cases the chat template is not even known, let alone additional parameters. Some models (that is modefile + weights) you can download from Ollama are even broken, so you can't get it working. Sometimes it is because the model was incorrectly trained like Westlake. Sometimes parameters are missing like for example the context size for the llama3 models. I don't know if they already fixed this but a llama3 model has a context size of 8k, if you don't set this in the model file, you will get trash responses after reaching this limit. The Ollama CLI also seems to store whole chat sessions either within a linked to specific model files. So can end up quickly in having tons of modelfiles which are listed as model but actually reference the same weights. If you need a different set of parameters then in the CLI you'd export the current modelfile, change it as needed and the "create" a new model from this. If you "create" it undere the same name, then the current one will be overriden. That's most likely the process that is included in Open WebUI, I say "most likely" because I actually do all of this on the command line and only had a quick look at this section of Open WebUI. ;-) Now since you can also change all parameters on the fly for the current session, just like you'd do with "/set parameter temperature 1" in the CLI, you can set these in the global settings of Open WebUI so the defaults from the modelfile or whatever defined them before will be overriden. Of course that's complete nonsense and very annoying, since you obviously want to have different settings for different models. So the option they give you is to create new modelfiles instead. So all this is quite clunky and unintuitive but that's more or less because Ollama is like it is. Of course you could work arround with that by for example not creating additional modelfiles by keeping profiles in the UI, we'll see of they do this.
I had a longer reply first but removed it. It seems most of this comment stems from a misunderstanding of how ollama works. Perhaps join the discord and ask there.
What you think about AnythingLLM as an alternative choice?
You’ll see what I think this week
Thing is, any gui would be better than commandline. Because you can use arrows to go back or forth, edit, select, etc. You cannot do that using Ollama in the terminal.
Those are things you can do in the ollama cli.
@@technovangelist well i can't. my own cli app has readline which allows me to do that but the official ollama cli does not allow me. when i hit left arrow for example i get ^[[D
it would be nice if you have the ability to replace the basic authentication with SSA or TFA.
You don't have to review any other webui - this is the best one :) Only the login is annoying.
I’m hoping there is one that does a good job with everything. This is nice but far from perfect
And the goal isn’t to look at web uis but rather all clients in general. It doesn’t offer that much over the cli but I am sure there is one that blows everything away.
I got addicted to ollama last year and got to play around with openwebui when it was still called ollama webui
The name change messed up my docker installs, not gonna lie
But then, we decided to try it as a corporate AI companion, but as it was a testing phase, we didn't scale our cloud very high, so it was pretty slow
On my machine though, I wanted to try and use every bit of feature, which led me to install and learn ComfyUI, and while the image generations options from openwebui is limited whatever the backend you use, it's still useable
Interesting. I haven't really played with ComfyUI.
I appreciate that you introduced me to ollama and are sharing your experiences and frustrations with the deployment of it's various features. Your videos are a combination of joy and frustration that are part of every software development cycle, and it feels great that I am not alone feeling this. Thank you 😊
The most shocking new information I got from this video is the idea that dark mode hurts someone's eyes! I thought everyone was saying, "I wish light mode would die because it hurts my eyes". By the way, a 100% white screen flashing at me in the middle of the night... literally causes my eyes physical pain.
The only ones shocked by that are the ones who don’t hear all the folks complaining about dark mode.
How do I get that theme? Thanks for the video!
Which one? I dont remember seeing one
Running on Windows 11, under Settings - General, my theme dropdown only contains "System, Dark, OLED Dark, and Light" but in this video Matt has a different, colorful type theme (purple text, red icons).
I didn't do anything special. But I am on Mac, so maybe something is different
What hardware are you using?
M1 Max MacBook Pro with 64GB ram
I'm developing a new UI tool that deals with some of the problems you're talking about, but I can't paste a link here. If you're interested, I'd love you to take a look.
My twitter is the same name as the channel
I tried to change the model, to no avail... despite having three models.
Hi
How do I setup this interface
That’s what the video shows
I'm in agreement with Matt about dark mode. Hard to read. That's why I use discord as little as possible.
Yep, all the young kids like to say that dark mode is easier to read. Wait until they get old.
matt if this not too much to ask... can i know, how many people ollama founder originally?
7
@@technovangelist wow
is this just for linux?...... I've been using anythingllm, I cant even see the install files on github....pretty confusing site, whey dont they have an arrow or something obvious that says download here!!... I see source files that appear to be for linux
It installs with docker on any platform and it’s a website.
today i just know... i can run phi3 while run llama3 and use both!
keep getting this after signup : Account Activation Pending
Contact Admin for WebUI Access
Your account status is currently pending activation. To access the WebUI, please reach out to the administrator. Admins can manage user statuses from the Admin Panel...
You need to go in as your admin account and approve it. The auth piece of this isn’t well thought out.