superduper_llamacpp
Superduper allows users to work with self-hosted LLM models via Llama.cpp.
Installation​
pip install superduper_llamacpp
API​
| Class | Description |
|---|---|
superduper_llamacpp.model.LlamaCpp | Llama.cpp connector. |
superduper_llamacpp.model.LlamaCppEmbedding | Llama.cpp connector for embeddings. |
Examples​
LlamaCpp​
from superduper_llama_cpp.model import LlamaCpp
model = LlamaCpp(
identifier="llm",
model_name_or_path="mistral-7b-instruct-v0.2.Q4_K_M.gguf",
)
model.predict("Hello world")