vLLM

Installation

pip install superduper_vllm

API

superduper allows users to work with self-hosted LLM models via "vLLM".

Class	Description	GitHub	API-docs
superduper.ext.vllm.VllmModel	Completes a prompt with natural language (LLM) based on a self hosted LLM	Code	Docs
superduper.ext.vllm.VllmAPI	Completes a prompt with natural language (LLM) based on a self hosted LLM behind the vLLM API server	Code	Docs