Salta al contenuto principale

friendica (DFRN) - Collegamento all'originale

LLM runs native on any OS

Wonderful #LLM work by Justine Tunney!


Here's an incredible new way to run LLMs on your own machine: llamafile - [url=https://hacks.mozilla.org/2023/11/introducing-llamafile/]https://hacks.mozilla.org/2023/11/introducing-llamafile/

It bundles an LLM with the code needed to run it in a single binary using DEEP magic (Cosmopolitan Libc) such that the same binary works on 6 different operating systems

Best part: it works with LLaVA multi-modal... so you can a 4GB file from and:

chmod 755 llamafile-server-0.1-llava-v1.5-7b-q4
./llamafile-server-0.1-llava-v1.5-7b-q

Visit

And now:

Screenshot. llama.cpp - then a photo I took of a plant User: Describe this plant Llama: The image features a large, green plant with numerous thin branches and leaves. Among the many stems of this plant, there is an orange flower visible near its center. This beautifully decorated plant stands out in the scene due to its vibrant colors and intricate structure. 18ms per token, 54.24 tokens per second Powered by llama.cpp, ggml.ai, and llamafile

Introducing llamafile - Mozilla Hacks - the Web developer blog

We're thrilled to announce the first release of llamafile, inviting the open source community to join this groundbreaking project.
Stephen Hood (Mozilla Hacks - the Web developer blog)

#llm

Giorgio Sidari reshared this.

⇧