Rubra favicon

Rubra
Open-weight, tool-calling LLMs

What is Rubra?

Rubra offers a suite of open-weight large language models specifically enhanced for tool-calling functionality. This capability allows the models to interact deterministically with user-defined external tools, making them highly suitable for developing AI agents and complex automated workflows.

The models are derived from leading open-source LLMs (such as Llama 3, Mistral, Gemma, Phi-3, and Qwen2) and undergo further post-training to integrate tool-calling skills while minimizing the loss of their original capabilities. Rubra facilitates easy deployment by extending popular inferencing projects like llama.cpp and vLLM, enabling users to run these enhanced models locally using an OpenAI-compatible tool-calling format.

Features

  • Tool-Calling Capability: Enhances LLMs to interact with external user-defined tools.
  • Open-Weight Models: Based on popular open-source LLMs like Llama 3, Gemma, Mistral, Phi-3, and Qwen2.
  • Agentic Use Cases: Ideal for building AI agents that can perform actions.
  • Mitigated Catastrophic Forgetting: Post-training methods preserve original model capabilities.
  • Local Deployment Support: Extensions for llama.cpp and vLLM for local inferencing.
  • OpenAI-Compatible Format: Tool-calling follows a familiar format.
  • GGUF Quants Available: Provides quantized versions for efficient deployment.

Use Cases

  • Developing AI agents
  • Automating workflows requiring external tool interaction
  • Integrating LLMs with existing APIs and services
  • Creating chatbots with dynamic external data access
  • Building custom reasoning engines that leverage external tools

FAQs

  • What models does Rubra enhance?
    Rubra enhances models like Meta-Llama-3 (8B & 70B), gemma-1.1-2b-it, Mistral-7B-Instruct (v0.2 & v0.3), Phi-3-mini-128k-instruct, and Qwen2-7B-Instruct.
  • How can I run Rubra models locally?
    You can run Rubra models locally using extended versions of inferencing tools like llama.cpp and vLLM, following an OpenAI-compatible tool-calling format.
  • What license are Rubra models released under?
    Rubra enhanced models are published under the same license as their parent model. The Rubra code itself is licensed under Apache 2.0.
  • Is there a way to try Rubra without installing anything?
    Yes, you can try the models for free without login via the Huggingface Spaces demo.
  • Are there any known issues with specific models?
    Llama3 models (8B and 70B) may experience degraded function-calling performance with quantization; using vLLM or fp16 quantization is recommended for these models.

Related Queries

Helpful for people in the following professions

Related Tools:

Blogs:

  • AI tools for video voice overs

    AI tools for video voice overs

    Discover the next level of video production with AI-powered voiceover tools. Enhance your content effortlessly, ensuring professional-quality narration for your videos.

  • Chat with PDF AI Tools

    Chat with PDF AI Tools

    Easily interact with your PDF documents using our advanced AI-powered tool. Whether you're reading lengthy reports, research papers, contracts, or eBooks, our platform lets you chat directly with your PDF files, ask questions, extract insights, and get summaries in real-time.

Didn't find tool you were looking for?

Be as detailed as possible for better results