LLM app development tools - AI tools

LM Studio is a desktop application that allows users to run Large Language Models (LLMs) locally and offline, supporting various architectures including Llama, Mistral, Phi, Gemma, DeepSeek, and Qwen 2.5.

• Free

BenchLLM is a tool for evaluating LLM-powered applications. It allows users to build test suites, generate quality reports, and choose between automated, interactive, or custom evaluation strategies.

• Other

Flowise is an open-source low-code platform that enables developers to build customized LLM orchestration flows and AI agents through a drag-and-drop interface.

• Freemium
• From 35$

Laminar is an open-source platform that enables developers to trace, evaluate, label, and analyze Large Language Model (LLM) applications with minimal code integration.

• Freemium
• From 25$

Inductor enables developers to rapidly prototype, evaluate, and improve LLM applications, ensuring high-quality app delivery.

• Freemium

LM Studio is a user-friendly desktop application that allows users to run various large language models (LLMs) locally and offline, including Llama 2, PN3, Falcon, Mistral, StarCoder, and GEMMA models from Hugging Face.

• Free

PromptMage is a Python framework that streamlines the development of complex, multi-step applications powered by Large Language Models (LLMs), offering version control, testing capabilities, and automated API generation.

• Other

Weights & Biases (W&B) Weave is a comprehensive framework designed for tracking, experimenting with, evaluating, deploying, and enhancing LLM-based applications.

• Other

GPT-LLM Playground is a macOS application designed for advanced experimentation and testing with Language Learning Models (LLMs). It offers features like multi-model support, versioning, and custom endpoints.

• Free

LLMStack is an open-source development platform that enables users to build AI agents, workflows, and applications by integrating various model providers and custom data sources.

• Other

LM-Kit provides .NET developers with tools for AI agent customization, creation, and orchestration. It enables multimodal generative AI systems integration in C# and VB.NET applications.

• Paid
• From 1000$

Missing Studio is an open-source AI platform designed for developers to build and deploy generative AI applications. It offers tools for managing LLMs, optimizing performance, and ensuring reliability.

• Free

Helicone is an all-in-one platform for monitoring, debugging, and improving production-ready LLM applications. It provides tools for logging, evaluating, experimenting, and deploying AI applications.

• Freemium
• From 20$

Keywords AI is a comprehensive developer platform for LLM applications, offering monitoring, debugging, and deployment tools. It serves as a Datadog-like solution specifically designed for LLM applications.

• Freemium
• From 7$

LMQL is a programming language designed for large language models, offering robust and modular prompting with types, templates, and constraints.

• Free

Langtail is a comprehensive testing platform that enables teams to test and debug LLM-powered applications with a spreadsheet-like interface, offering security features and integration with major LLM providers.

• Freemium
• From 99$

Langfuse provides an open-source platform for tracing, evaluating, and managing prompts to debug and improve LLM applications.

• Freemium
• From 59$

Kalavai offers a platform for deploying Large Language Models (LLMs) across various devices, scaling from personal laptops to full production environments. It simplifies LLM deployment and experimentation.

• Paid
• From 29$