superduper_llamacpp

Superduper allows users to work with self-hosted LLM models via Llama.cpp.

Installation

pip install superduper_llamacpp

API

• Code
• API-docs

Class	Description
superduper_llamacpp.model.LlamaCpp	Llama.cpp connector.
superduper_llamacpp.model.LlamaCppEmbedding	Llama.cpp connector for embeddings.

Examples

LlamaCpp

from superduper_llama_cpp.model import LlamaCpp

model = LlamaCpp(
    identifier="llm",
    model_name_or_path="mistral-7b-instruct-v0.2.Q4_K_M.gguf",
)
model.predict("Hello world")