KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models. It’s a single self contained distributable from Concedo, that builds off llama.cpp, and adds a versatile Kobold API endpoint, additional format support, backward compatibility, as well as a fancy UI with persistent stories, editing tools, save formats, memory, world info, author’s note, characters, scenarios and everything Kobold and Kobold Lite have to offer.
https://github.com/LostRuins/koboldcpp
Installing KoboldCpp
This guide shows you how to install KoboldCpp on your computer. Once set up, you can load large language models for text-based interaction. The KoboldCpp also offers API functionality, allowing integration with Voxta for speech-driven experiences. For step-by-step instructions, see the attached video tutorial.
For more detailed information, FAQs, and support, please visit the KoboldCpp Wiki.