Usage Panda - Security and Observability for OpenAI

What is Usage Panda?

Usage Panda is a security and observability platform designed for OpenAI. It acts as a "Prompt Application Firewall," allowing organizations to implement granular control and gain enhanced visibility over their OpenAI API usage. The platform sits between clients and OpenAI's APIs, inspecting each request for adherence to predefined policies before it is sent to OpenAI.

Usage Panda offers robust features for managing and securing OpenAI interactions. These include policy enforcement to ensure compliance, cost management tools to prevent unexpected expenses, and detailed logging for debugging and monitoring. The platform also provides prompt monitoring to protect against malicious attempts, along with dashboards and metrics for comprehensive usage analysis.

Features

Policy Enforcement: Evaluates security policies for requests before they're sent to OpenAI.
Cost Management: Avoid surprise bills by allowing only requests that fall below a cost threshold.
Debug Logging: Opt-in to log the complete request, parameters, and response for every request.
Granular Controls: Create unlimited connections, each with custom policies and limits.
Prompt Monitoring: Monitor, redact, and block malicious attempts to alter or reveal system prompts.
Dashboards & Metrics: Explore usage in granular detail using visualization tools and custom charts.
Billing Alerts: Get notified via email or Slack before reaching a usage limit.
User Tracking: Associate costs and policy violations back to end users.
Real-Time Data: Logs requests in real-time.
Secure Connection: OpenAI API key is never saved or logged; prompt logs are opt-in.

Use Cases

Enforcing security policies for OpenAI API requests.
Managing and controlling costs associated with OpenAI usage.
Debugging OpenAI API interactions with detailed logging.
Monitoring prompt injection and other malicious attempts.
Gaining real-time insights into OpenAI API usage patterns.
Setting up billing alerts and user-specific rate limits.
Provisioning OpenAI API access without distributing OpenAI API keys

FAQs

How does Usage Panda work?

Usage Panda is a security and observability platform for OpenAI that sits between clients and OpenAI's APIs, either as a hosted cloud service or a self-hosted proxy. Usage Panda inspects each request for adherence to a defined policy, before it is sent to OpenAI. Usage Panda also collects observability metrics, such as latency and error rate, for application monitoring purposes.
Is Usage Panda secure?

Yes! Usage Panda never stores or logs your OpenAI API key. The contents of your prompts and completion responses are only logged if you explicitly opt-in. You can read more about security here.
What kind of reports can Usage Panda generate?

Reports include timestamps, token usage, costs, and completion model types with up to one year of history. Trend reports help visualize your most popular models, changes in daily activity, and more. Currency conversions are also supported.
How much does Usage Panda cost?

Usage Panda is completely free, for up to 500 OpenAI API requests/month. Usage Panda's paid plans are coming soon and include 5,000 monthly API requests and 6 months of retention. We are planning an enterprise plan if you need more requests, or longer retention, so please get in touch.
What kinds of policies can Usage Panda enforce?

With Usage Panda enabled, you can enforce a max token limit and max character limit, block certain categories of PII from being sent to OpenAI, rate limit requests, block requests that will exceed a cost threshold, monitor for certain moderation flags, and more.