Agent skill

integrate

Add Olakai monitoring to existing AI code — wrap your LLM client, configure custom KPIs, and validate the integration end-to-end

View SKILL.md on GitHub Repository

Stars 12,831

Forks 1,123

Install this agent skill to your Project

npx add-skill https://github.com/andrewyng/context-hub/tree/main/content/olakai/skills/integrate

Metadata

Additional technical details for this skill

tags: olakai,integration,monitoring,sdk,kpi,governance
source: maintainer
revision: 1
updated on: 2026-03-10

SKILL.md

Integrate Olakai into Existing AI Code

This skill guides you through adding Olakai monitoring to an existing AI agent or LLM-powered application with minimal code changes.

For full SDK documentation, see: https://app.olakai.ai/llms.txt

Prerequisites

Existing working AI agent/application using OpenAI, Anthropic, or other LLM
Olakai CLI installed and authenticated (npm install -g olakai-cli && olakai login)
Olakai API key for your agent (get via CLI: olakai agents get AGENT_ID --json | jq '.apiKey')
Node.js 18+ (for TypeScript) or Python 3.7+ (for Python)

Note: Each agent can have its own API key. Create one with olakai agents create --name "Name" --with-api-key

Why Custom KPIs Are Essential

Adding monitoring is only the first step. The real value of Olakai comes from tracking custom KPIs specific to your agent's business purpose.

Without KPIs configured:

Only basic token counts and request data
No aggregated business KPIs on dashboard
No alerting capabilities
No ROI tracking

With KPIs configured:

Custom KPIs (items processed, success rates, quality scores)
Trend analysis and performance dashboards
Threshold-based alerting
Business value calculations

Plan to configure at least 2-4 KPIs that answer: "How do I know this agent is performing well?"

KPIs are unique per agent. If adding monitoring to an agent that needs the same KPIs as another already-configured agent, you must still create new KPI definitions for this agent. KPIs cannot be shared or reused across agents.

Understanding the customData to KPI Pipeline

Before adding monitoring, understand how custom data flows through Olakai:

SDK customData → CustomDataConfig (Schema) → Context Variable → KPI Formula → kpiData

Critical Rules

Rule	Consequence
Only CustomDataConfig fields become variables	Unregistered customData fields are NOT usable in KPIs
Formula evaluation is case-insensitive	`stepCount`, `STEPCOUNT`, `StepCount` all work in formulas
NUMBER configs need numeric values	Don't send `"5"` (string), send `5` (number)

IMPORTANT: The SDK accepts any JSON in customData, but only fields registered as CustomDataConfigs are processed. Unregistered fields are stored but cannot be used in KPIs.

Quick Start (5-Minute Integration)

For TypeScript/JavaScript

1. Install the SDK:

bash

npm install @olakai/sdk

2. Add tracking after your LLM call:

Before:

typescript

import OpenAI from "openai";
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const response = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: userMessage }],
});

After:

typescript

import OpenAI from "openai";
import { olakaiConfig, olakai } from "@olakai/sdk";

olakaiConfig({ apiKey: process.env.OLAKAI_API_KEY });

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const response = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: userMessage }],
});

// Track the interaction (fire-and-forget)
olakai("event", "ai_activity", {
  prompt: userMessage,
  response: response.choices[0].message.content,
  tokens: response.usage?.total_tokens,
  userEmail: user.email,
  task: "Customer Experience",
});

For Python

1. Install the SDK:

bash

pip install olakai-sdk

2. Add tracking after your LLM call:

Before:

python

from openai import OpenAI
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": user_message}],
)

After:

python

from openai import OpenAI
from olakaisdk import olakai_config, olakai, OlakaiEventParams

olakai_config(os.getenv("OLAKAI_API_KEY"))
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": user_message}],
)

# Track the interaction
olakai("event", "ai_activity", OlakaiEventParams(
    prompt=user_message,
    response=response.choices[0].message.content,
    tokens=response.usage.total_tokens,
    userEmail=user.email,
    task="Customer Experience",
))

Detailed Integration Guide

Step 1: Identify Your Integration Pattern

Pattern A: Single LLM Client You have one OpenAI/Anthropic client used throughout your app. Use the fire-and-forget olakai() call after each completion.

Pattern B: Multiple LLM Calls per Request Your agent makes several LLM calls to complete one task. Use manual event tracking to aggregate calls into a single event.

Pattern C: Streaming Responses You stream LLM responses to users. Track after the stream completes with the full accumulated response.

Pattern D: Third-Party LLM (not OpenAI/Anthropic) You use Perplexity, Groq, local models, etc. Use manual event tracking via olakai() or olakai_event().

Step 2: Install and Configure

TypeScript Setup

typescript

// lib/olakai.ts - Initialize once at app startup
import { olakaiConfig } from "@olakai/sdk";

olakaiConfig({
  apiKey: process.env.OLAKAI_API_KEY!,
  debug: process.env.NODE_ENV === "development",
});

Python Setup

python

# lib/olakai.py - Initialize once at app startup
import os
from olakaisdk import olakai_config

olakai_config(
    api_key=os.getenv("OLAKAI_API_KEY"),
    debug=os.getenv("DEBUG") == "true"
)

Step 3: Add Context to Calls

Adding User Information

TypeScript:

typescript

olakai("event", "ai_activity", {
  prompt: userMessage,
  response: aiResponse,
  userEmail: user.email,
  task: "Customer Experience",
});

Python:

python

olakai("event", "ai_activity", OlakaiEventParams(
    prompt=user_message,
    response=ai_response,
    userEmail=user.email,
    task="Customer Experience",
))

Grouping Events by Conversation (chatId)

For assistive AI (chatbots/copilots), use chatId to group multiple turns of a conversation together. This is required for CHAT-scoped KPIs that analyze the full conversation.

typescript

olakai("event", "ai_activity", {
  prompt: userMessage,
  response: aiResponse,
  chatId: conversationId,  // groups turns in the same conversation
  userEmail: user.email,
});

When to use chatId: If your agent handles multi-turn conversations and you want KPIs that evaluate the entire conversation (e.g., sentiment scoring, satisfaction), pass a consistent chatId across all turns.

Adding Custom Data

IMPORTANT: Only send fields you've registered as CustomDataConfigs (Step 5.3). Unregistered fields are stored but cannot be used in KPIs.

Only send data you'll use in KPIs or for filtering. Don't duplicate fields already tracked by the platform (session ID, agent ID, user email, timestamps, token count, model, provider — all tracked automatically).

TypeScript:

typescript

olakai("event", "ai_activity", {
  prompt: userMessage,
  response: aiResponse,
  userEmail: user.email,
  customData: {
    // Only include fields registered as CustomDataConfigs
    Department: user.department,
    ProjectId: currentProject.id,
    Priority: ticket.priority,
  },
});

Step 4: Handle Agentic Workflows

If your agent makes multiple LLM calls per task, aggregate them into a single event.

taskExecutionId — Critical for multi-agent workflows. If multiple agents collaborate on the same task, the orchestrator must generate ONE taskExecutionId and pass it to all agents. This is how Olakai correlates cross-agent work as a single logical task.

typescript

async function processDocument(doc: Document): Promise<string> {
  const startTime = Date.now();
  let totalTokens = 0;

  // Step 1: Extract
  const extraction = await openai.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: `Extract from: ${doc.content}` }],
  });
  totalTokens += extraction.usage?.total_tokens ?? 0;

  // Step 2: Analyze
  const analysis = await openai.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: `Analyze: ${extraction.choices[0].message.content}` }],
  });
  totalTokens += analysis.usage?.total_tokens ?? 0;

  const result = analysis.choices[0].message.content ?? "";

  // Track the complete workflow as ONE event
  olakai("event", "ai_activity", {
    prompt: `Process document: ${doc.title}`,
    response: result,
    tokens: totalTokens,
    requestTime: Date.now() - startTime,
    taskExecutionId: crypto.randomUUID(),
    task: "Data Processing & Analysis",
    customData: {
      DocumentType: doc.type,
      StepCount: 2,
      Success: 1,
    },
  });

  return result;
}

Step 5: Configure Custom KPIs (Essential for Value)

This step is required to get real value from Olakai. Without KPIs, you're only tracking events — not gaining actionable insights.

5.1 Install CLI (if not already)

bash

npm install -g olakai-cli
olakai login

5.2 Register Your Agent

bash

olakai agents create \
  --name "Document Processor" \
  --description "Processes and summarizes documents" \
  --workflow WORKFLOW_ID \
  --with-api-key

5.2.1 Ensure Agent Has a Workflow

Every agent MUST belong to a workflow, even if it's the only agent.

bash

# Check if agent has a workflow
olakai agents get YOUR_AGENT_ID --json | jq '.workflowId'

# If null, create a workflow and associate:
olakai workflows create --name "Your Workflow Name" --json
olakai agents update YOUR_AGENT_ID --workflow WORKFLOW_ID

5.3 Create Custom Data Configs FIRST

IMPORTANT: Create configs for ALL fields you send in customData. Only registered fields can be used in KPIs. CustomDataConfigs are agent-scoped.

bash

olakai custom-data create --agent-id YOUR_AGENT_ID --name "DocumentType" --type STRING
olakai custom-data create --agent-id YOUR_AGENT_ID --name "StepCount" --type NUMBER
olakai custom-data create --agent-id YOUR_AGENT_ID --name "Success" --type NUMBER

# Verify all configs exist for this agent
olakai custom-data list --agent-id YOUR_AGENT_ID

5.4 Create KPIs

bash

olakai kpis create \
  --name "Documents Processed" \
  --agent-id YOUR_AGENT_ID \
  --calculator-id formula \
  --formula "IF(Success = 1, 1, 0)" \
  --aggregation SUM

olakai kpis create \
  --name "Avg Steps per Document" \
  --agent-id YOUR_AGENT_ID \
  --calculator-id formula \
  --formula "StepCount" \
  --aggregation AVERAGE

5.5 Update SDK Code to Match

After creating configs, ensure your SDK code sends exactly those field names:

typescript

customData: {
  DocumentType: doc.type,     // Matches CustomDataConfig "DocumentType"
  StepCount: 2,               // Matches CustomDataConfig "StepCount"
  Success: true ? 1 : 0,      // Matches CustomDataConfig "Success"
}

Framework-Specific Integrations

Next.js API Routes

typescript

// app/api/chat/route.ts
import { NextRequest, NextResponse } from "next/server";
import { olakai } from "@olakai/sdk";
import { auth } from "@/auth";

export async function POST(req: NextRequest) {
  const session = await auth();
  if (!session?.user) {
    return NextResponse.json({ error: "Unauthorized" }, { status: 401 });
  }

  const { message } = await req.json();

  const response = await openai.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: message }],
  });

  olakai("event", "ai_activity", {
    prompt: message,
    response: response.choices[0].message.content,
    userEmail: session.user.email!,
    task: "Customer Experience",
  });

  return NextResponse.json({ reply: response.choices[0].message.content });
}

FastAPI (Python)

python

from fastapi import FastAPI, Depends
from olakaisdk import olakai_config, olakai, OlakaiEventParams

app = FastAPI()

@app.on_event("startup")
async def startup():
    olakai_config(os.getenv("OLAKAI_API_KEY"))

client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

@app.post("/chat")
async def chat(message: str, user: User = Depends(get_current_user)):
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": message}]
    )

    olakai("event", "ai_activity", OlakaiEventParams(
        prompt=message,
        response=response.choices[0].message.content,
        userEmail=user.email,
        task="Customer Experience",
    ))

    return {"reply": response.choices[0].message.content}

Handling Edge Cases

Streaming Responses

Track after the stream completes with the full response:

typescript

const stream = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: userMessage }],
  stream: true,
});

let fullResponse = "";
for await (const chunk of stream) {
  fullResponse += chunk.choices[0]?.delta?.content ?? "";
  res.write(chunk.choices[0]?.delta?.content ?? "");
}

// Track after stream completes
olakai("event", "ai_activity", {
  prompt: userMessage,
  response: fullResponse,
  userEmail: user.email,
});

Error Handling

typescript

try {
  const response = await openai.chat.completions.create({
    model: "gpt-4o",
    messages,
  });
  return response.choices[0].message.content;
} catch (error) {
  // Track the failed attempt
  olakai("event", "ai_activity", {
    prompt: messages[messages.length - 1].content,
    response: `Error: ${error instanceof Error ? error.message : "Unknown"}`,
    task: "Software Development",
    customData: { Success: 0 },
  });
  throw error;
}

Non-OpenAI Providers

For Anthropic, Perplexity, or other providers, use manual tracking:

typescript

import Anthropic from "@anthropic-ai/sdk";

const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

async function callClaude(prompt: string): Promise<string> {
  const startTime = Date.now();

  const response = await anthropic.messages.create({
    model: "claude-sonnet-4-20250514",
    max_tokens: 1024,
    messages: [{ role: "user", content: prompt }],
  });

  const content = response.content[0].type === "text" ? response.content[0].text : "";

  olakai("event", "ai_activity", {
    prompt,
    response: content,
    tokens: response.usage.input_tokens + response.usage.output_tokens,
    requestTime: Date.now() - startTime,
    task: "Content Development",
  });

  return content;
}

Test-Validate-Iterate Cycle

Never assume your integration is working. Always validate by generating a test event and inspecting the actual data.

Step 1: Generate a Test Event

Run your application to trigger at least one LLM call.

Step 2: Fetch and Inspect the Event

bash

olakai activity list --limit 1 --json
olakai activity get EVENT_ID --json

Step 3: Validate Each Component

Check customData is present:

bash

olakai activity get EVENT_ID --json | jq '.customData'

Check KPIs are numeric (not strings or null):

bash

olakai activity get EVENT_ID --json | jq '.kpiData'

CORRECT:

json

{ "My KPI": 42 }

WRONG (formula stored as string):

json

{ "My KPI": "MyVariable" }

Fix: olakai kpis update KPI_ID --formula "MyVariable"

WRONG (null value):

json

{ "My KPI": null }

Fix by ensuring:

CustomDataConfig exists: olakai custom-data create --agent-id ID --name "MyVariable" --type NUMBER
Field name case matches exactly (case-sensitive)
SDK actually sends the field in customData

Validation Flow

1. Trigger LLM call (generate event)
           ↓
2. Fetch: olakai activity get ID --json
           ↓
3. Event exists? NO → Check API key, SDK init, debug mode
           ↓
4. customData correct? NO → Fix SDK customData parameter
           ↓
5. kpiData numeric? NO → olakai kpis update ID --formula "X"
           ↓
6. kpiData not null? NO → Create CustomDataConfig, check field name case
           ↓
✅ Integration validated

KPI Formula Reference

Supported Operators

Category	Operators
Arithmetic	`+`, `-`, `*`, `/`
Comparison	`<`, `<=`, `=`, `<>`, `>=`, `>`
Logical	`AND`, `OR`, `NOT`
Conditional	`IF(condition, true_val, false_val)`
Null handling	`ISNA(value)`, `ISDEFINED(value)`

Common Formula Patterns

bash

--formula "StepCount"                          # passthrough
--formula "SuccessRate * 100"                  # percentage conversion
--formula "IF(Success = 1, 1, 0)"              # conditional counting
--formula "IF(PII detected, 1, 0)"             # built-in variable
--formula "IF(ISDEFINED(MyField), MyField, 0)" # null-safe

Aggregation Types

Aggregation	Use For
`SUM`	Totals, counts
`AVERAGE`	Rates, percentages

Quick Reference

typescript

// TypeScript — initialize once
import { olakaiConfig, olakai } from "@olakai/sdk";
olakaiConfig({ apiKey: process.env.OLAKAI_API_KEY });

// Track any interaction
olakai("event", "ai_activity", {
  prompt: "input",
  response: "output",
  tokens: 1500,
  requestTime: 5000,
  userEmail: "[email protected]",
  chatId: "conversation-id",
  taskExecutionId: "uuid-shared-across-agents",
  task: "Data Processing & Analysis",
  customData: { StepCount: 3, Success: 1 },
});

python

# Python — initialize once
from olakaisdk import olakai_config, olakai, OlakaiEventParams
olakai_config(os.getenv("OLAKAI_API_KEY"))

# Track any interaction
olakai("event", "ai_activity", OlakaiEventParams(
    prompt="input",
    response="output",
    tokens=1500,
    requestTime=5000,
    userEmail="[email protected]",
    chatId="conversation-id",
    taskExecutionId="uuid-shared-across-agents",
    task="Data Processing & Analysis",
    customData={"StepCount": 3, "Success": 1},
))

Maintainer

andrewyng Core maintainer

Source details

Full Name: andrewyng/context-hub
Branch: main
Path in repo: content/olakai/skills/integrate
License: MIT License

Featured Tools

Join Our Newsletter

Stay updated with the latest AI tools, news, and offers by subscribing to our weekly newsletter.

Recommended Agent Skills

Expand your agent's capabilities with these related and highly-rated skills.

andrewyng/context-hub

login-flows

Common login automation patterns for web apps using Playwright

12,831 1,123

Explore

andrewyng/context-hub

document-extraction

Use this skill for intelligent document processing and content extraction using LandingAI's Agentic Document Extraction (ADE). Trigger when users need to (1) Parse documents (PDFs, images, spreadsheets, presentations) into structured Markdown with layout understanding, (2) Extract specific structured data from documents using schemas (invoice fields, form data, table data, etc.), (3) Classify and separate multi-document batches by type (invoices vs receipts, statements vs forms, etc.), (4) Process large documents asynchronously (up to 1GB/1000 pages), (5) Get visual grounding (bounding boxes, page numbers) for extracted content — use when users mention bounding boxes, word locations, grounding, highlighting extracted content, or showing where data appears in a document. Use this skill when the task involves understanding document content for a set of documents. In particular this skill can help you write code that run on sets of documents. This will increase speed, and reduce the cost of loading the documents on the Agent context window because you can use a single script to extract the information needed.

12,831 1,123

Explore

andrewyng/context-hub

document-workflows

Use this skill for building end-to-end document processing workflows and pipelines using LandingAI ADE. Trigger when users need to: (1) Process batches of documents in parallel or async, (2) Build classify-then-extract pipelines for mixed document types, (3) Prepare parsed documents for RAG systems with chunking and vector DB ingestion, (4) Load extraction results into databases like Snowflake or export to CSV/DataFrames, (5) Visualize extraction results: draw bounding box overlays on pages, crop chunk images, or highlight/annotate specific words or phrases found in documents, (6) Build Streamlit or web UIs for document processing, (7) Find and highlight specific terms within document sections using word-level grounding (e.g. highlight "L2S" in the Introduction, redact PII, annotate extracted values on the original page). This skill complements the document-extraction skill which covers ADE SDK basics. Use document-extraction to write code that executes parse/extract/split operations with more precision and less cost than adding the document image to the prompt and asking the LLM to find the relevant info. Use document-workflows when composing those operations into pipelines, or when you need visualization, annotation, or word-level grounding on parsed documents.

12,831 1,123

Explore

andrewyng/context-hub

new-project

Build a new AI agent with Olakai monitoring from scratch — project setup, SDK integration, KPI configuration, and end-to-end validation

12,831 1,123

Explore

andrewyng/context-hub

tavily-best-practices

Build production-ready Tavily integrations with best practices for web search, content extraction, crawling, and research in agentic workflows, RAG systems, and autonomous agents

12,831 1,123

Explore

andrewyng/context-hub

electronics-sourcing

Guide for AI agents to source electronic components using parts-mcp — tool sequencing, decision patterns, and multi-step workflows

12,831 1,123

Explore

Didn't find tool you were looking for?

Search AI Tools

Install this agent skill to your Project

Metadata

SKILL.md

Integrate Olakai into Existing AI Code

Prerequisites

Why Custom KPIs Are Essential

Understanding the customData to KPI Pipeline

Critical Rules

Quick Start (5-Minute Integration)

For TypeScript/JavaScript

For Python

Detailed Integration Guide

Step 1: Identify Your Integration Pattern

Step 2: Install and Configure

TypeScript Setup

Python Setup

Step 3: Add Context to Calls

Adding User Information

Grouping Events by Conversation (chatId)

Adding Custom Data

Step 4: Handle Agentic Workflows

Step 5: Configure Custom KPIs (Essential for Value)

5.1 Install CLI (if not already)

5.2 Register Your Agent

5.2.1 Ensure Agent Has a Workflow

5.3 Create Custom Data Configs FIRST

5.4 Create KPIs

5.5 Update SDK Code to Match

Framework-Specific Integrations

Next.js API Routes

FastAPI (Python)

Handling Edge Cases

Streaming Responses

Error Handling

Non-OpenAI Providers

Test-Validate-Iterate Cycle

Step 1: Generate a Test Event

Step 2: Fetch and Inspect the Event

Step 3: Validate Each Component

Validation Flow

KPI Formula Reference

Supported Operators

Common Formula Patterns

Aggregation Types

Quick Reference

Recommended Agent Skills

login-flows

document-extraction

document-workflows

new-project

tavily-best-practices

electronics-sourcing