together.solutions

Problem

Latitude.io faced challenges hosting LLMs at scale due to high infrastructure costs, complex GPU deployment management, and the need for scalable AI solutions. As 95% of gameplay is driven by AI models, these hurdles affected their ability to drive gamer experiences to their high ambitions.

Solution

Together Inference allowed Latitude.io to keep latency low while increasing total daily tokens by 8x. Receiving immediate access to the latest models such as Meta Llama 3 and Mixtral at low costs, Latitude.io was able to improve model quality and achieve longer context lengths. They also spent 80% less time managing GPU deployments leading to significant cost savings while being assured of their cluster’s health.

Result

Latitude.io tripled their average input tokens per request, resulting in improved player value. In addition, their average requests per user per day have doubled, further establishing the player’s acknowledgement of these improvements. While using Together AI, Latitude has increased their user value and engagement as well as reinforced their core mission: giving back to the player.

Problem

Cartesia wanted a partner with a deep understanding of the inference stack for new model architectures. While latency was important to create seamless user experience, they also wanted the flexibility to optimize between latency, throughput and cost for their inference deployment

‍

Solution

The ability for Cartesia to use Together AI with their own custom model enabled them to serve their state-of-the-art custom state space model, Sonic, in less than 2 weeks. With Together AI, Cartesia was able to optimize for real-time inference with industry leading latency of <200 ms and 2x faster performance while maintaining the highest accuracy; all at half the cost of other providers.

‍

Result

Cartesia was able to achieve the industry’s fastest text-to-voice performance with <200 ms end-to-end latency and 135 ms model latency to provide real-time inference to their users. With their cutting-edge technology and the fast Together AI service, they achieved lower cost and real-time text-to-voice generations, passing on these benefits to their end users.

Problem

Vals.ai needed an AI platform to run their eval suite on a variety of industries and across multiple benchmarks. They didn’t want to provision compute to host each new model themselves for testing. Instead, they wanted to use a provider that had high throughput, was reliable, and hosted several models as soon as they were available to ensure their benchmarks are kept up to date.

‍

Solution

Since Vals.ai has made Together AI their default provider for all their open source model evaluations, they have been able to efficiently and affordably run many evals across multiple industries. Additionally, since Together AI is so agile in incorporating new OSS models, Vals.ai have been able to test models like Llama-3 on the same day they are released.

Result

Vals.ai has been able to run ~ 320k API calls, 200M tokens in a single day on Together AI while keeping their costs low and steady. Due to unprecedented low latency they have also been able to run evaluations very efficiently which has become one their company’s biggest value adds.

Problem

Needed efficient compute capacity that scaled from prototype to production. Having fast and efficient performance for training was a must. They needed to move quickly – they didn’t have time to worry about setting up their own training infrastructure and they needed a partner who could scale with their difficult-to-forecast traffic.

Solution

Pika used Together Inference API to rapidly prototype using the easy-to-use open-source model library. Once the team decided to build their own models from the ground up, they opted for the unparalleled compute power of Together GPU Clusters. And once they launched the product and saw user traction grow exponentially, Pika scaled inference seamlessly.

Results

Pika grew to millions of videos generated per month with the top users spending ~10 hours per day on the platform — all within 6 months of being founded.

‍

Problem‍

Upstage needed to host Solar, their most popular LLM, so that it could be used by the widest possible audience. When the model charted on the Hugging Face Open LLM Leaderboard, they also needed a place that could scale to handle high traffic while maintaining fast performance and cost efficiency.

‍

Solution

Upstage chose Together Inference serverless endpoints to host their model because of the user-friendly interface of the API, its competitive pricing, and Together AI’s expert support that made bringup super easy.

Result

The Solar model was deployed on Together Inference, and published on Poe.com. Together Inference easily scaled to serve over 2.8 million peak tokens per hour with exceptional performance — over 45 tokens per second. The Upstage team expanded their partnership and integrating Together AI into their own service.

‍

‍

Wordware's mission is to enhance the machine learning workflow by removing the dependency on extensive 'ground truth' datasets. Their platform empowers domain experts to quickly refine prompts, improving collaboration and speeding up iterations. Wordware wanted to focus on building the best collaborative web-based IDE for language model programming with seamless model selection and not on the hassle of managing expensive infrastructure. 

Wordware adopted Together's infrastructure for its versatility and user-friendly interface. The ability to rapidly prototype and scale using Together's Inference API and the powerful compute capabilities of the service was integral to their progress. The platform's low latency, minimal cold start times, and cost-effectiveness allowed Wordware to experiment with various models, enabling their customers to transition from GPT-4 to Mistral, leading to significant cost reductions, enhanced reliability and reduced latency.

Wordware's innovative approach has led to groundbreaking applications. One notable customer example is the development of AI-powered NPC interactions, in which the cost of operation was reduced by 16x after transitioning to Wordware. This efficiency is attributed to Wordware's token-based pricing and the ability to integrate multiple models seamlessly, like Mistral and OpenChat, offering a unique balance of speed, flexibility, and cost-effectiveness, which Wordware attributes to Together’s API.

Problem

To enhance the capabilities of existing base models with public data, Nexusflow required a cost-effective, reliable, and scalable compute partner. Traditional cloud providers were not able to simultaneously offer the cost-efficiency and the level of guaranteed availability that Nexusflow needed to scale their specialized workloads.

Solution

The team at Nexusflow opted for Together GPU Clusters, seeing it as the perfect "trifecta" in terms of contract length, pricing, and compute availability. They utilized GPUs suitable for their specific workload requirements, and benefited from the unparalleled support that Together’s expert team offers.

Results

Nexusflow completed the onboarding process in <90 minutes and was able to run workloads. Initial hiccups were resolved by Together's support team, ensuring a smooth experience. Nexusflow managed to cut their R&D cloud compute costs by 40%, while experiencing faster response times and lower latency in technical support than other cloud providers.

Problem

Arcee was looking for more reliable, factual systems that are also cost effective to build domain adaptive language models for Arcee’s customers.

Solution

Arcee made a strategic decision to build a fine-tuned model with Together AI for several compelling reasons: the accessibility of Together API, the quality of the Together AI team, and their commitment to build a good model not just as a technical provider but as a collaborative partner.

Results

Arcee built their model using Together Custom Models including domain-specific data. To optimize the quality of the model, it was trained with a data mixture optimized using DoReMi, an algorithm for finding the optimal mixture of language datasets using Distributionally Robust Optimization.