DSPy

The sie-dspy package provides DSPy-compatible modules for SIE: dense and sparse embedders, a reranker, and an entity extractor.

Installation

pip install sie-dspy

This installs sie-sdk and numpy as dependencies.

Start the Server

# Docker (recommended)
docker run -p 8080:8080 ghcr.io/superlinked/sie:default

# Or with GPU
docker run --gpus all -p 8080:8080 ghcr.io/superlinked/sie:default

Embedder

SIEEmbedder is callable and works directly with DSPy’s retrieval components.

from sie_dspy import SIEEmbedder

embedder = SIEEmbedder(
    base_url="http://localhost:8080",
    model="BAAI/bge-m3",
)

# Embed a single text
vector = embedder("What is machine learning?")
print(vector.shape)  # (1024,)

# Embed multiple texts
vectors = embedder(["First document", "Second document"])
print(vectors.shape)  # (2, 1024)

Configuration Options

Parameter	Type	Default	Description
base_url	str	http://localhost:8080	SIE server URL
model	str	BAAI/bge-m3	Model to use
gpu	str	None	Target GPU type for routing
options	dict	None	Model-specific options
timeout_s	float	180.0	Request timeout in seconds

Full Example

Build a complete DSPy RAG program using SIE embeddings:

import dspy
from sie_dspy import SIEEmbedder

# 1. Configure DSPy with your LLM
lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

# 2. Create SIE embedder
embedder = SIEEmbedder(
    base_url="http://localhost:8080",
    model="BAAI/bge-m3",
)

# 3. Build retriever with your corpus
corpus = [
    "Machine learning is a branch of artificial intelligence.",
    "Neural networks are inspired by biological neurons.",
    "Deep learning uses multiple layers of neural networks.",
    "Python is popular for machine learning development.",
]

retriever = dspy.retrievers.Embeddings(
    corpus=corpus,
    embedder=embedder,
    k=2,
)

# 4. Define a RAG signature
class RAG(dspy.Signature):
    """Answer questions using retrieved context."""
    context: list[str] = dspy.InputField(desc="retrieved passages")
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

# 5. Build the RAG program
class RAGProgram(dspy.Module):
    def __init__(self):
        self.retriever = retriever
        self.generate = dspy.ChainOfThought(RAG)

    def forward(self, question: str):
        passages = self.retriever(question)
        return self.generate(context=passages, question=question)

# 6. Run the program
rag = RAGProgram()
result = rag("What is deep learning?")
print(result.answer)

Sparse Embeddings

SIESparseEmbedder provides sparse vectors for hybrid search workflows.

from sie_dspy import SIEEmbedder, SIESparseEmbedder

dense_embedder = SIEEmbedder(model="BAAI/bge-m3")
sparse_embedder = SIESparseEmbedder(model="BAAI/bge-m3")

# Index corpus — store both in your vector DB (Qdrant, Weaviate, etc.)
dense_vecs = dense_embedder(corpus)
sparse_vecs = sparse_embedder.embed_documents(corpus)

# Query
dense_query = dense_embedder(query)
sparse_query = sparse_embedder.embed_query(query)

Configuration Options

Parameter	Type	Default	Description
base_url	str	http://localhost:8080	SIE server URL
model	str	BAAI/bge-m3	Model to use
gpu	str	None	Target GPU type for routing
options	dict	None	Model-specific options
timeout_s	float	180.0	Request timeout in seconds

Reranking

SIEReranker is a dspy.Module that reranks passages by relevance to a query using a cross-encoder model.

from sie_dspy import SIEReranker

reranker = SIEReranker(
    base_url="http://localhost:8080",
    model="jinaai/jina-reranker-v2-base-multilingual",
)

result = reranker(
    query="What is machine learning?",
    passages=["ML learns from data.", "Weather is sunny.", "Deep learning uses neural nets."],
    k=2,
)

print(result.passages)  # Top 2 most relevant passages
print(result.scores)    # Corresponding relevance scores

Configuration Options

Parameter	Type	Default	Description
base_url	str	http://localhost:8080	SIE server URL
model	str	jinaai/jina-reranker-v2-base-multilingual	Reranker model
gpu	str	None	Target GPU type for routing
options	dict	None	Model-specific options
timeout_s	float	180.0	Request timeout in seconds

Entity Extraction

SIEExtractor is a dspy.Module that extracts named entities from text using GLiNER models.

from sie_dspy import SIEExtractor

extractor = SIEExtractor(
    base_url="http://localhost:8080",
    model="urchade/gliner_multi-v2.1",
    labels=["person", "organization", "location"],
)

result = extractor(text="Tim Cook announced new products at Apple Park in Cupertino.")

for entity in result.entities:
    print(f"{entity.label}: {entity.text} ({entity.score:.2f})")
# person: Tim Cook (0.96)
# organization: Apple (0.91)
# location: Cupertino (0.88)

# Also available as dicts for JSON serialization
print(result.entities_dict)

Configuration Options

Parameter	Type	Default	Description
base_url	str	http://localhost:8080	SIE server URL
model	str	urchade/gliner_multi-v2.1	Extraction model
labels	list[str]	["person", "organization", "location"]	Default entity labels
gpu	str	None	Target GPU type for routing
options	dict	None	Model-specific options
timeout_s	float	180.0	Request timeout in seconds

What’s Next

• Encode Text - embedding API details
• Score / Rerank - reranking details
• Extract Entities - extraction details
• Model Catalog - all supported models
• Troubleshooting - common errors and solutions