Open source observability and analytics for LLM applications

🪢 Langfuse

Open source observability & analytics for LLM-based applications

Observability: Explore and debug complex logs & traces in a visual UI

Analytics: Measure & improve costs, latency and response quality

Join the Langfuse Discord »
langfuse.com · Docs · Report Bug · Feature Request

What is Langfuse?

Langfuse is an open source observability & analytics solution for LLM-based applications. It is mostly geared towards production usage but some users also use it for local development of their LLM applications.

Langfuse is focused on applications built on top of LLMs. Many new abstractions and common best practices evolved recently, e.g. agents, chained prompts, embedding-based retrieval, LLM access to REPLs & APIs. These make applications more powerful but also unpredictable for developers as they cannot fully anticipate how changes impact the quality, cost and overall latency of their application. Thus Langfuse helps to monitor and debug these applications.

Demo (2 min)

Muted by default, enable sound for voice-over

Explore demo project in Langfuse here (free account required): https://langfuse.com/demo

Observability

Langfuse offers an admin UI to explore the ingested data.

• Nested view of LLM app executions; detailed information along the traces on: latency, cost, scores
• Segment execution traces by user feedback, to e.g. identify production issues

Analytics

Reporting on

• Token usage by model
• Volume of traces
• Scores/evals

Broken down by

• Users
• Releases
• Prompt/chain versions
• Prompt/chain types
• Time

→ Expect releases with more ways to analyze the data over the next weeks.

Get started

Step 1: Run Server

Langfuse Cloud

Managed deployment by the Langfuse team, generous free-tier (hobby plan) available, no credit card required.

Links: Create account, learn more

Localhost

Requirements: Docker, Node.js >=18, npm

# Clone repository
git clone git@github.com:langfuse/langfuse.git
cd langfuse

# Run server and database
docker compose up -d

# Apply db migrations
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/postgres DIRECT_URL=postgresql://postgres:postgres@localhost:5432/postgres npx prisma migrate deploy

Self-host (Docker)

→ Instructions

Step 2: Data ingestion

Langchain applications

The Langfuse callback handler automatically instruments Langchain applications. Currently available for Python, JS/TS support is in progress (add +1 to issue here).

pip install langfuse

# Initialize Langfuse handler
from langfuse.callback import CallbackHandler
handler = CallbackHandler(PUBLIC_KEY, SECRET_KEY)

# Setup Langchain
from langchain.chains import LLMChain
...
chain = LLMChain(llm=llm, prompt=prompt)

# Add Langfuse handler as callback
chain.run(input="<user_input", callbacks=[handler])

→ More details: Langchain integration docs

SDKs to manually instrument application

Fully async, typed SDKs to instrument any LLM application. Currently available for Python & JS/TS.

→ Guide with an example of how the SDK can be used

Package	Description	Links
	Python	docs, repo
	JS/TS: Node >= 18, Edge runtimes	docs, repo
	JS/TS: Node <18	docs, repo

Add scores/evaluations to traces (optional)

Quality/evaluation of traces is tracked via scores (docs). Scores are related to traces and optionally to observations. Scores can be added via:

• Backend SDKs (see docs above): {trace, event, span, generation}.score()

• API (see docs below): POST /api/public/scores

• Client-side using Web SDK, e.g. to capture user feedback or other user-based quality metrics:

  npm install langfuse

  // Client-side (browser)
  
  import { LangfuseWeb } from "langfuse";
  
  const langfuseWeb = new LangfuseWeb({
    publicKey: process.env.LANGFUSE_PUBLIC_KEY,
  });
  
  // frontend handler (example: React)
  export function UserFeedbackComponent(props: { traceId: string }) {
    const handleUserFeedback = async (value: number) => {
      await langfuseWeb.score({
        traceId: props.traceId,
        name: "user_feedback",
        value,
      });
    };
    return (
      <div>
        <button onClick={() => handleUserFeedback(1)}>👍</button>
        <button onClick={() => handleUserFeedback(-1)}>👎</button>
      </div>
    );
  }

API

Api reference

• POST/PATCH routes to ingest data
• GET routes to use data in downstream applications (e.g. embedded analytics)

Questions / Feedback

The maintainers are very active in the Langfuse Discord and are happy to answer questions or discuss feedback/ideas regarding the future of the project.

Contributing to Langfuse

Join the community on Discord.

To contribute, send us a PR, raise a github issue, or email at contributing@langfuse.com

Development setup

Requirements: Node.js >=18, npm, Docker

# Install dependencies
npm install

# Run the db
docker-compose -f docker-compose.dev.yml up -d

# create an env file
cp .env.dev.example .env

# Migration
npm run db:migrate

# Optional: seed the database
# npm run db:seed
# npm run db:seed:examples

# Start the server
npm run dev

License

Langfuse is MIT licensed, except for ee/ folder. See LICENSE and docs for more details.

Misc

Upgrade Langfuse (localhost)

# Stop server and db
docker compose down

# Pull latest changes
git pull
docker-compose pull

# Run server and db
docker compose up -d

# Apply db migrations
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/postgres DIRECT_URL=postgresql://postgres:postgres@localhost:5432/postgres npx prisma migrate deploy

Run Langfuse in CI for integration tests

Checkout GitHub Actions workflows of Python SDK and JS/TS SDK.

Star History