llamafile test with OmniFusion - вопросы llama на русском
Vložit
- čas přidán 1. 05. 2024
- original - github.com/Mozilla-Ocho/llama...
models from - habr.com/ru/companies/airi/ar...
to start server:
llamafile.exe -ngl 9999 -m OmniFusion-1.1-Q5_K_M.gguf --mmproj mmproj-model-f16.gguf
or only cli:
llamafile.exe -ngl 9999 --temp 0 --image test.png -m OmniFusion-1.1-Q5_K_M.gguf --mmproj mmproj-model-f16.gguf -e -p "### User: What do you see?
### Assistant: "
llamafile.exe -ngl 9999 -m OmniFusion-1.1-Q5_K_M.gguf -p "[INST]Write a story about llamas[/INST]"
Can I start server from any llamafile? If I will start like ./llava-v1.5-7b-q4.llamafile -ngl 9999 -m OmniFusion-1.1-Q5_K_M.gguf --mmproj mmproj-model-f16.gguf is it suppose work correct?