What is Baseten?
Baseten offers a robust infrastructure platform designed for deploying, managing, and scaling machine learning models in production environments. It caters to teams requiring high performance, security, and reliability, coupled with an intuitive developer experience. The platform supports both custom-built and open-source models, providing flexibility through various deployment options, including Baseten's managed cloud, the user's own virtual private cloud (self-hosted), or a hybrid approach.
The platform emphasizes state-of-the-art performance across different AI modalities, featuring optimized inference engines, rapid cold starts, and low latency crucial for real-time applications. Baseten streamlines the development-to-deployment pipeline using Truss, an open-source standard for model packaging. Key features include effortless GPU autoscaling, comprehensive observability tools for monitoring metrics like inference counts and response times, efficient resource and cost management, and robust security measures, including SOC 2 Type II certification and HIPAA compliance.
Features
- High-Performance Inference: Utilizes optimized serving engines and hardware for speed and low latency.
- Flexible Deployment Options: Supports Baseten Cloud, self-hosted (user's VPC), and hybrid deployment models.
- Truss Model Packaging: Employs an open-source standard for packaging models from any framework.
- Effortless GPU Autoscaling: Automatically adjusts model replicas based on traffic demands.
- Fast Cold Starts: Optimized pipeline ensures quick model readiness from zero replicas.
- Enterprise Readiness: Includes security (SOC 2 Type II, HIPAA), resource/cost management, and observability.
- Streamlined Developer Workflow: Simplifies model deployment using tools like Truss.
- Instant API Endpoint: Automatically generates an API endpoint for deployed models.
Use Cases
- Deploying custom machine learning models.
- Scaling large language model (LLM) inference.
- Serving image generation models.
- Running high-throughput transcription services.
- Powering text-to-speech applications.
- Building real-time AI applications like chatbots.
- Managing embedding and reranker model inference.
- Developing and deploying compound AI systems.
FAQs
-
Which models can I run on Baseten?
You can deploy open source and custom models on Baseten. Start with an off-the-shelf model from our model library. Or deploy any model using Truss, our open source standard for packaging and serving models built in any framework. -
Which GPUs are available on Baseten?
You have control over what GPUs your models use. See our instance type reference for a full list of the GPUs currently available on Baseten. Reach out to us to request additional GPU types. -
Do you offer free credits to get started?
Yes, new Baseten accounts come with $30 of free credit so that you can start running models for free. -
Is Baseten secure?
Yes, Baseten is SOC 2 Type II certified and HIPAA compliant. You can read more about our SOC 2 Type II certification here. And you can read more about our HIPAA compliance here. -
Do I pay for idle time on Baseten?
No, you do not pay for idle time – you only pay for the time your model is using compute on Baseten. This includes the time your model is actively deploying, scaling up or down, or making predictions. And you have full control over how your model scales up or down.
