This website uses cookies to anonymously analyze website traffic using Google Analytics.
AcceptDecline

TogetherInference

Run inference on 200+ open-source models powered by the Together Inference Stack and scale effortlessly in production.

Start building nowContact us

200+ generative AI models

Build with open-source and specialized multimodal models for chat, images, code, and more. Migrate from closed models with OpenAI-compatible APIs.

All
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Try now
together.ai

Chat

Llama 4 Maverick

SOTA 128-expert MoE powerhouse for multilingual image/text understanding, creative writing, and enterprise-scale applications.

TRY
THIS MODEL

New

Chat

Llama 4 Scout

SOTA 109B model with 17B active params & large context, excelling at multi-document analysis, codebase reasoning, and personalized tasks.

TRY
THIS MODEL

New

Chat

DeepSeek R1

Open-source reasoning model rivaling OpenAI-o1, excelling in math, code, reasoning, and cost efficiency.

TRY
THIS MODEL

Chat

DeepSeek R1 Distilled Llama 70B Free

Free endpoint to experiment the power of reasoning models. This distilled model beats GPT-4o on math & matches o1-mini on coding.

TRY
THIS MODEL

Free

Chat

Gemma 3 27B

Lightweight model with vision-language input, multilingual support, visual reasoning, and top-tier performance per size.

TRY
THIS MODEL

New

Chat

Llama 3.3 70B Instruct Turbo Free

Free endpoint to try this 70B multilingual LLM optimized for dialogue, excelling in benchmarks and surpassing many chat models.

TRY
THIS MODEL

Free

Vision

Qwen2.5-VL 72B Instruct

Vision-language model with advanced visual reasoning, video understanding, structured outputs, and agentic capabilities.

TRY
THIS MODEL

New

Chat

Cogito V1 Preview Llama 70B

Best-in-class open-source LLM trained with IDA for alignment, reasoning, and self-reflective, agentic applications.

TRY
THIS MODEL

New

Image

FLUX.1 [schnell] Free

Free endpoint for the SOTA open-source image generation model by Black Forest Labs.

TRY
THIS MODEL

Free

Vision

Llama 3.2 11B Free

Free endpoint to try Llama 3.2 11B.

TRY
THIS MODEL

Free

Chat

DeepSeek-V3-0324

DeepSeek's latest open Mixture-of-Experts model challenging top AI models at much lower cost.

TRY
THIS MODEL

Chat

Qwen QwQ-32B

Qwen series reasoning model excelling in complex tasks, outperforming conventional instruction-tuned models on hard problems.

TRY
THIS MODEL

New

Chat

Cogito V1 Preview Qwen 32B

Best-in-class open-source LLM trained with IDA for alignment, reasoning, and self-reflective, agentic applications.

TRY
THIS MODEL

New

Chat

Qwen2.5 72B

Powerful decoder-only models available in 7B and 72B variants, developed by Alibaba Cloud's Qwen team for advanced language processing.

TRY
THIS MODEL

Chat

Cogito V1 Preview Qwen 14B

Best-in-class open-source LLM trained with IDA for alignment, reasoning, and self-reflective, agentic applications.

TRY
THIS MODEL

New

Audio

Cartesia Sonic-2

Low-latency, ultra-realistic voice model, served in partnership with Cartesia.

TRY
THIS MODEL

New

Chat

Mistral Small 3

24B model rivaling GPT-4o mini, and larger models like Llama 3.3 70B. Ideal for chat use cases like customer support, translation and summarization.

TRY
THIS MODEL

Chat

Cogito V1 Preview Llama 8B

Best-in-class open-source LLM trained with IDA for alignment, reasoning, and self-reflective, agentic applications.

TRY
THIS MODEL

New

Chat

Llama 3.1 Nemotron 70B Instruct

This LLM is customized by NVIDIA to improve the helpfulness of LLM generated responses to user queries.

TRY
THIS MODEL

Chat

Cogito V1 Preview Llama 3B

Best-in-class open-source LLM trained with IDA for alignment, reasoning, and self-reflective, agentic applications.

TRY
THIS MODEL

New

Chat

Llama 3.3 70B

The Meta Llama 3.3 multilingual large language model (LLM) is a pretrained and instruction tuned generative model in 70B (text in/text out). The Llama 3.3 instruction tuned text only model is optimized for multilingual dialogue use cases and outperform many of the available open source and closed chat models on common industry benchmarks.

TRY
THIS MODEL

Image

FLUX1.1 [pro]

Premium image generation model by Black Forest Labs.

TRY
THIS MODEL

Chat

Llama 3.1 405B

The Meta Llama 3.1 collection of multilingual large language models (LLMs) is a collection of pre-trained and instruction tuned generative models in 8B, 70B and 405B sizes, that outperform many available open source and closed chat models on common industry benchmarks.

TRY
THIS MODEL

Code

Qwen 2.5 Coder 32B Instruct

SOTA code LLM with advanced code generation, reasoning, fixing, and support for up to 128K tokens.

TRY
THIS MODEL

Chat

Gemma-2 Instruct (27B)

Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models.

TRY
THIS MODEL

Chat

Llama 3.1 8B

The Meta Llama 3.1 collection of multilingual large language models (LLMs) is a collection of pre-trained and instruction tuned generative models in 8B, 70B and 405B sizes, that outperform many available open source and closed chat models on common industry benchmarks.

TRY
THIS MODEL

Vision

Qwen2-VL-72B-Instruct

A powerful OSS vision model by Alibaba that combines advanced vision capabilities with instruction-tuned language understanding for sophisticated visual reasoning tasks.

TRY
THIS MODEL

Chat

DeepSeek R1 Distilled Llama 70B

Llama 70B distilled with reasoning capabilities from Deepseek R1. Surpasses GPT-4o with 94.5% on MATH-500 & matches o1-mini on coding.

TRY
THIS MODEL

Chat

DeepSeek R1 Distilled Qwen 14B

Qwen 14B distilled with reasoning capabilities from Deepseek R1. Outperforms GPT-4o in math & matches o1-mini on coding.

TRY
THIS MODEL

Chat

DeepSeek R1 Distilled Qwen 1.5B

Small Qwen 1.5B distilled with reasoning capabilities from Deepseek R1. Beats GPT-4o on MATH-500 whilst being a fraction of the size.

TRY
THIS MODEL

Chat

Gemma 3 1B

Most lightweight Gemma 3 model (1B) with 128K context, vision-language input, and multilingual support for on-device AI.

TRY
THIS MODEL

New

Chat

Gemma 3 4B

Lightweight model with vision-language input, multilingual support, visual reasoning, and top-tier performance per size.

TRY
THIS MODEL

New

Chat

Gemma 3 12B

Lightweight model with vision-language input, multilingual support, visual reasoning, and top-tier performance per size.

TRY
THIS MODEL

New

Chat

DBRX-Instruct

DBRX Instruct is a mixture-of-experts (MoE) large language model trained from scratch by Databricks. DBRX Instruct specializes in few-turn.

TRY
THIS MODEL

Image

FLUX.1 [dev]

12 billion parameter rectified flow transformer capable of generating images from text descriptions.

TRY
THIS MODEL

Image

FLUX.1 [pro]

First generation premium image generation model by Black Forest Labs.

TRY
THIS MODEL

Image

FLUX.1 Canny [dev]

12 billion parameter rectified flow transformer capable of generating an image based on a text description while following the structure of a given input image.

TRY
THIS MODEL

Image

FLUX.1 Depth [dev]

12 billion parameter rectified flow transformer capable of generating an image based on a text description while following the structure of a given input image.

TRY
THIS MODEL

Image

FLUX.1 Redux [dev]

Adapter for FLUX.1 models enabling image variation, refining input images, and integrating into advanced restyling workflows.

TRY
THIS MODEL

Image

FLUX.1 [schnell]

Fastest available endpoint for the SOTA open-source image generation model by Black Forest Labs.

TRY
THIS MODEL

Chat

Deepseek-67B

Trained from scratch on a vast dataset of 2 trillion tokens in both English and Chinese.

TRY
THIS MODEL

Vision

Llama 3.2 90B

The Llama 3.2-Vision collection features multimodal LLMs (11B and 90B) optimized for visual recognition, image reasoning, captioning, and answering image-related questions.

TRY
THIS MODEL

Vision

Llama 3.2 11B

The Llama 3.2-Vision collection features multimodal LLMs (11B and 90B) optimized for visual recognition, image reasoning, captioning, and answering image-related questions.

TRY
THIS MODEL

Chat

Llama 3.2 3B Instruct Turbo

The Llama 3.2-Vision collection features multimodal LLMs (11B and 90B) optimized for visual recognition, image reasoning, captioning, and answering image-related questions.

TRY
THIS MODEL

Chat

Gemma Instruct (2B)

2B instruct Gemma model by Google: lightweight, open, text-to-text LLM for QA, summarization, reasoning, and resource-efficient deployment.

TRY
THIS MODEL

Chat

Gemma-2 Instruct (9B)

Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models.

TRY
THIS MODEL

Chat

Qwen2.5 7B Instruct Turbo

Qwen2.5 is the latest series of Qwen large language models. For Qwen2.5, we release a number of base language models and instruction-tuned language models ranging from 0.5 to 72 billion parameters.

TRY
THIS MODEL

Language

Mistral

7.3B parameter model that outperforms Llama 2 13B on all benchmarks, approaches CodeLlama 7B performance on code, Uses Grouped-query attention (GQA) for faster inference and Sliding Window Attention (SWA) to handle longer sequences at smaller cost

TRY
THIS MODEL

Image

Stable Diffusion XL 1.0

A text-to-image generative AI model that excels at creating 1024x1024 images.

TRY
THIS MODEL

Chat

Mistral Instruct

Instruct fine-tuned version of Mistral-7B-v0.1

TRY
THIS MODEL

Chat

Mixtral 8x7B Instruct v0.1

The Mixtral-8x7B Large Language Model (LLM) is a pretrained generative Sparse Mixture of Experts.

TRY
THIS MODEL

Language

Mixtral 8x7B v0.1

The Mixtral-8x7B Large Language Model (LLM) is a pretrained generative Sparse Mixture of Experts.

TRY
THIS MODEL

Embeddings

BGE-Large-EN v1.5

BAAI general embedding - large, english - model v1.5. FlagEmbedding can map any text to a low-dimensional dense vector which can be used for tasks like retrieval, classification, clustering, or semantic search. And it also can be used in vector databases for LLMs.

TRY
THIS MODEL

Rerank

Salesforce LlamaRank

Salesforce Research's proprietary fine-tuned rerank model with 8K context, outperforming Cohere Rerank for superior document retrieval.

TRY
THIS MODEL

Chat

Mistral (7B) Instruct v0.2

The Mistral-7B-Instruct-v0.2 Large Language Model (LLM) is an improved instruct fine-tuned version of Mistral-7B-Instruct-v0.1.

TRY
THIS MODEL

Chat

Mistral (7B) Instruct v0.3

The Mistral-7B-Instruct-v0.3 Large Language Model (LLM) is an instruct fine-tuned version of the Mistral-7B-v0.3.

TRY
THIS MODEL

Chat

Upstage SOLAR Instruct v1 (11B)

Built on the Llama2 architecture, SOLAR-10.7B incorporates the innovative Upstage Depth Up-Scaling.

TRY
THIS MODEL

Chat

WizardLM-2 (8x22B)

WizardLM-2 8x22B is Wizard's most advanced model, demonstrates highly competitive performance compared to those leading proprietary works and consistently outperforms all the existing state-of-the-art opensource models.

TRY
THIS MODEL

Chat

Llama 3.1 70B

The Meta Llama 3.1 collection of multilingual large language models (LLMs) is a collection of pre-trained and instruction tuned generative models in 8B, 70B and 405B sizes, that outperform many available open source and closed chat models on common industry benchmarks.

TRY
THIS MODEL

Chat

Mixtral-8x22B Instruct v0.1

The Mixtral-8x22B-Instruct-v0.1 Large Language Model (LLM) is an instruct fine-tuned version of the Mixtral-8x22B-v0.1.

TRY
THIS MODEL

Chat

Nous Hermes 2 - Mixtral 8x7B-DPO

Nous Hermes 2 Mixtral 7bx8 DPO is the new flagship Nous Research model trained over the Mixtral 7bx8 MoE LLM. The model was trained on over 1,000,000 entries of primarily GPT-4 generated data, as well as other high quality data from open datasets across the AI landscape, achieving state of the art performance on a variety of tasks.

TRY
THIS MODEL

Chat

Qwen 2

A transformer-based decoder-only language model pre-trained on a large amount of data. In comparison with the previously released Qwen

TRY
THIS MODEL

Chat

Typhoon 2 70B Instruct

Instruct Thai large language model with 70 billion parameters, based on Llama3.1-70B.

TRY
THIS MODEL

Chat

Typhoon 2 8B Instruct

Instruct Thai large language model with 8 billion parameters based on Llama3.1-8B.

TRY
THIS MODEL

Chat

Typhoon 1.5 8B Instruct

Instruct Thai large language model with 8 billion parameters based on Llama3-8B.

TRY
THIS MODEL

Language

Llama Guard (7B)

Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations

TRY
THIS MODEL

Language

Llama Guard 2 8B

8B Llama 3-based safeguard model for classifying LLM inputs and outputs, detecting unsafe content and policy violations.

TRY
THIS MODEL

Language

Llama Guard 3 11B Vision Turbo

Llama 3.2 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.

TRY
THIS MODEL

Language

Llama Guard 3 8B

8B Llama 3.1 model fine-tuned for content safety, moderating prompts and responses in 8 languages with MLCommons alignment.

TRY
THIS MODEL

Chat

MythoMax-L2

MythoLogic-L2 and Huginn merge using a highly experimental tensor type merge technique. The main difference with MythoMix is that I allowed more of Huginn to intermingle with the single tensors located at the front and end of a model

TRY
THIS MODEL

Embeddings

UAE-Large v1

An universal English sentence embedding model by WhereIsAI. Its embedding dimension is 1024, it takes up to 512 context length.

TRY
THIS MODEL

Chat

Typhoon 1.5X 70B-awq

Thai language 70B instruct model rivaling GPT-4-0612; optimized for RAG, constrained generation, and reasoning tasks.

TRY
THIS MODEL

Embeddings

M2-BERT 80M 2K Retrieval

An 80M checkpoint of M2-BERT, pretrained with sequence length 2048, and it has been fine-tuned for long-context retrieval.

TRY
THIS MODEL

Embeddings

M2-BERT 80M 8K Retrieval

An 80M checkpoint of M2-BERT, pretrained with sequence length 8192, and it has been fine-tuned for long-context retrieval.

TRY
THIS MODEL

Embeddings

BGE-Base-EN v1.5

BAAI general embedding - base, english - model v1.5. FlagEmbedding can map any text to a low-dimensional dense vector which can be used for tasks like retrieval, classification, clustering, or semantic search. And it also can be used in vector databases for LLMs.

TRY
THIS MODEL

Chat

Gryphe MythoMax L2 Lite (13B)

MythoLogic-L2 and Huginn merge using a highly experimental tensor type merge technique. The main difference with MythoMix is that I allowed more of Huginn to intermingle with the single tensors located at the front and end of a model.

TRY
THIS MODEL

Chat

Llama 3 70B Instruct Lite

Llama 3 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.

TRY
THIS MODEL

Language

LLaMA-2

Language model trained on 2 trillion tokens with double the context length of Llama 1. Available in three sizes: 7B, 13B and 70B parameters

TRY
THIS MODEL

Chat

Llama 3 70B Instruct Reference

Llama 3 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.

TRY
THIS MODEL

Chat

Llama 3 70B Instruct Turbo

Llama 3 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.

TRY
THIS MODEL

Chat

Llama 3 8B Instruct Lite

Llama 3 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.

TRY
THIS MODEL

Chat

Llama 3 8B Instruct Reference

Llama 3 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.

TRY
THIS MODEL

Chat

Llama 3 8B Instruct Turbo

Llama 3 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.

TRY
THIS MODEL

Chat

LLaMA-2 Chat (13B)

Llama 2-chat leverages publicly available instruction datasets and over 1 million human annotations. Available in three sizes: 7B, 13B and 70B parameters

TRY
THIS MODEL

Chat

LLaMA-2 Chat (7B)

Llama 2-chat leverages publicly available instruction datasets and over 1 million human annotations. Available in three sizes: 7B, 13B and 70B parameters.

TRY
THIS MODEL

Image

FLUX.1 Schnell [fixedres]

FLUX.1 [schnell] is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions.

TRY
THIS MODEL

Embeddings

M2-BERT 80M 32K Retrieval

An 80M checkpoint of M2-BERT, pretrained with sequence length 32768, and it has been fine-tuned for long-context retrieval.

TRY
THIS MODEL

Inference that is fast, simple, and scales as you grow.

  • Fast

    Run leading open-source models like DeepSeek and Llama on the fastest inference engine available, up to 4x faster than vLLM.

    ‍

    Outperforms Amazon Bedrock, and Azure AI by over 2x.

    ‍

  • COST-EFFICIENT

    Together Inference is 11x lower cost than GPT-4o when using Llama 3.3 70B and 9x lower cost than OpenAI o1 when using DeepSeek-R1. Our optimizations bring you the best performance at the lowest cost.

  • scalable

    We obsess over system optimization and scaling so you don’t have to. As your application grows, capacity is automatically added to meet your API request volume.

Run fast Serverless Endpoints for leading open-source models

import os, requests

url = "https://api.together.xyz/v1/chat/completions"

payload = {
    "model": "deepseek-ai/DeepSeek-R1",
    "max_tokens": 512,
    "temperature": 0.7,
    "top_p": 0.7,
    "top_k": 50,
    "repetition_penalty": 1
}
headers = {
    "accept": "application/json",
    "content-type": "application/json",
    "Authorization": "Bearer TOGETHER_API_KEY"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

#  Sign up to get your API key here: https://api.together.ai/
#  Documentation for API usage: https://docs.together.ai/
Try now
Watch demo
together endpoints create \
--model mistralai/Mixtral-8x7B-Instruct-v0.1 \
--gpu h100 \
--min-replicas 1 \
--max-replicas 8 \
--gpu-count 2 \
--display-name "My Endpoint" \
--wait

Deploy on Dedicated Endpoints for unmatched price-performance at scale

Powered by the Together Inference Stack

Built by AI researchers for AI innovators, the Together Inference Stack gives you  the fastest NVIDIA GPUs running our proprietary Inference Engine, optimized by Together Kernel Collection and customized to your traffic.

  • Fast Inference Engine

    Running on the latest NVIDIA GPUs with custom optimizations, our Inference Engine offers inference that's 4x faster than vLLM.

  • Together Kernel Collection

    The Together Kernel Collection, from our Chief Scientist and FlashAttention creator Tri Dao, provides up to 10% faster training and 75% faster inference.

  • Customized to Your Traffic Profile

    Together AI’s Research team will fine-tune and optimize your deployment using our proprietary optimizations such as custom speculators.

""Together AI offers optimized performance at scale, and at a lower cost than closed-source providers – all while maintaining strict privacy standards."

‍

- Vineet Khosla, CTO, The Washington Post

Performance

You get more tokens per second, higher throughput and lower time to first token. And, all these efficiencies mean we can provide you compute at a lower cost.

SPEED RELATIVE TO VLLM

4x FASTER

LLAMA 3.3 70B
DEPLOYED ON 2x h100

6100 TOKENS/SEC

COST RELATIVE TO GPT-4o

11x lower cost

Enterprise-grade security and data privacy

We take security and compliance seriously, with strict data privacy controls to keep your information protected. Your data and models remain fully under your ownership, safeguarded by robust security measures.

Learn more

Flexible deployment options

  • Together Cloud

    • Get started quickly with fully managed serverless endpoints with pay-per-token pricing
    • Dedicated GPU endpoints with autoscaling for consistent performance
    TRY NOW

  • Your Cloud

    • Dedicated serverless deployments in your cloud provider
    • VPC deployment available for additional security
    • Use your existing cloud spend
    CONTACT SALES

  • Together
    GPU Clusters

    • For large-scale inference workloads or foundation model training
    • NVIDIA H100 and H200s clusters with Infiniband and NVLink
    • Available with Together Training and Inference Engines for up to 25% faster training and 75% faster inference than PyTorch
    CONTACT SALES

Run inference with the best price-performance at scale

Explore our model library

Subscribe to newsletter

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.