Top-Rated LLMs for Chat in 2025

The rapid rise of large language models (LLMs) has transformed how we build modern AI applications. They now power everything from customer support chatbots to complex LLM agents that can reason, plan, and take actions across tools.

For many AI teams, closed-source options like GPT-5 and Claude Sonnet 4 are convenient. With just a simple API call, you can prototype an AI product in minutes — no GPUs to manage and no infrastructure to maintain. However, this convenience comes with trade-offs: vendor lock-in, limited customization, unpredictable pricing and performance, and ongoing concerns about data privacy.

That’s why open-source LLMs have become so important. They let developers self-host models privately, fine-tune them with domain-specific data, and optimize inference performance for their unique workloads.

In this post, we’ll explore the best open-source LLMs for chat in 2025. After that, we’ll answer some of the FAQs teams have when evaluating LLMs for production use.

DeepSeek-V3.1#

DeepSeek-V3.1 is one of the best open-source LLMs available today. Built with a Mixture-of-Experts (MoE) architecture, it combines the strengths of predecessors V3 and R1 into a single hybrid model. It features a total of 671B parameters (37B activated) and supports context lengths up to 128K.

The model is built on DeepSeek-V3.1-Base, trained with an expanded long-context process: 630B tokens for the 32K phase and 209B tokens for the 128K phase.

DeepSeek came to the spotlight during the “DeepSeek moment” in early 2025, when its R1 model demonstrated ChatGPT-level reasoning at significantly lower training costs. Now that DeepSeek-V3.1 integrates reasoning capabilities, it's unclear whether they still plan to release a standalone DeepSeek-R2 model. Learn about the latest DeepSeek model series.

Why should you use DeepSeek-V3.1:

• Hybrid thinking: DeepSeek-V3.1 supports both thinking and non-thinking modes. You can switch between fast responses and deeper reasoning simply by changing the chat template. This makes it highly adaptable for different types of chat and agentic tasks.
• Efficient reasoning performance: The V3.1-Think variant achieves reasoning quality on par with DeepSeek-R1-0528 but responds faster. DeepSeek’s internal evaluations show that with chain-of-thought compression training, it reduces output tokens by 20–50% while maintaining similar performance.
• Advanced tool use: Post-training optimization has improved tool usage. V3.1 outperforms both DeepSeek-V3-0324 and DeepSeek-R1-0528 in benchmarks for code agents and search agents. This makes it an ideal choice for building LLM agents.
• Fully open-source: Released under the permissive MIT License, DeepSeek-V3.1 is free to use for commercial, academic, and personal projects. It's an attractive option for teams building self-hosted LLM deployments, especially those looking to avoid vendor lock-in.

Also note that DeepSeek-V3.1 requires substantial compute resources. Running it efficiently requires multi-GPU setups, like 8 NVIDIA H200 (141GB of memory) GPUs. This level of infrastructure is cost-prohibitive for smaller teams or individual developers.

Deploy DeepSeek-V3.1Deploy DeepSeek-V3.1

gpt-oss-120b#

gpt‑oss‑120b is OpenAI’s most capable open-source LLM to date. With 117B total parameters and a Mixture-of-Experts (MoE) architecture, it rivals proprietary models like o4‑mini. More importantly, it’s fully open-weight and available for commercial use.

OpenAI trained the model with a mix of reinforcement learning and lessons learned from its frontier models, including o3. The focus was on making it strong at reasoning, efficient to run, and practical for real-world use. The training data was mostly English text, with a heavy emphasis on STEM, coding, and general knowledge. For tokenization, OpenAI used an expanded version of the tokenizer that also powers o4-mini and GPT-4o.

The release of gpt‑oss marks OpenAI’s first fully open-weight LLMs since GPT‑2. It has already seen adoption from early partners like Snowflake, Orange, and AI Sweden for fine-tuning and secure on-premises deployment.

Why should you use gpt‑oss‑120b:

• Excellent performance: gpt‑oss‑120b matches or surpasses o4-mini on core benchmarks like AIME, MMLU, TauBench, and HealthBench (even outperforms proprietary models like OpenAI o1 and GPT‑4o).

• Efficient and flexible deployment: Despite its size, gpt‑oss‑120b can run on a single 80GB GPU (e.g., NVIDIA H100 or AMD MI300X). It's optimized for local, on-device, or cloud inference via partners like vLLM, llama.cpp and Ollama.

• Adjustable reasoning levels: It supports low, medium, and high reasoning modes to balance speed and depth.

  • Low: Quick responses for general use.
  • Medium: Balanced performance and latency
  • High: Deep and detailed analysis.

• Permissive license: gpt‑oss‑120b is released under the Apache 2.0 license, which means you can freely use it for commercial applications. This makes it a good choice for teams building custom LLM inference pipelines.

Deploy gpt-oss-120bDeploy gpt-oss-120b

Kimi-K2#

Kimi-K2 is a MoE model optimized for agentic tasks, with 32 billion activated parameters and a total of 1 trillion parameters. It delivers state-of-the-art performance in frontier knowledge, math, and coding among non-thinking models. Two variants are open-sourced: Kimi-K2-Base (for full-control fine-tuning) and Kimi-K2-Instruct (drop-in chat/agent use).

The latest release is Kimi-K2-Instruct-0905, which improves agentic and front-end coding abilities and extends context length to 256K tokens.

Why should you use Kimi-K2:

• Agent-first design: The agentic strength comes from two pillars: large-scale agentic data synthesis and general RL. Inspired by ACEBench, K2’s pipeline simulates realistic multi-turn tool-use across hundreds of domains and thousands of tools, including real MCP tools + synthetic ones.

  Some teams already saw great real-world results. For example, Guillermo Rauch, CEO of Vercel, mentioned that Kimi K2 ran up to 5× faster and was about 50% more accurate than some top proprietary models like GPT-5 and Claude-Sonnet-4.5 in their internal agent tests.

  

• Competitive coding & tool use: In head-to-head evaluations, Kimi-K2-Instruct matches or outperforms open-source and proprietary models (e.g., DeepSeek-V3-0324, Qwen3-235B-A22B, Claude Sonnet 4, Gemini 2.5 Flash) on knowledge-intensive reasoning, code generation, and agentic tool-use tasks.

• Long-context: Updated weights support 256K tokens, useful for agent traces, docs, and multi-step planning.

Note that Kimi-K2 is released under a modified MIT license. The sole modification: If you use it in a commercial product or service with 100M+ monthly active users or USD 20M+ monthly revenue, you must prominently display “Kimi K2” in the product’s user interface.

Qwen3-235B-A22B-Instruct-2507#

Alibaba has been one of the most active contributors to the open-source LLM ecosystem with its Qwen series. Qwen3 is the latest generation, offering both dense and MoE models across a wide range of sizes. At the top of the lineup is Qwen3-235B-A22B-Instruct-2507, an updated version of the earlier Qwen3-235B-A22B’s non-thinking mode.

This model has 235B parameters, with 22B active per token, powered by 128 experts (8 active). Note that it only supports non-thinking mode and does not generate <think></think> blocks. You can try Qwen3-235B-A22B-Thinking-2507 for more complex reasoning tasks.

Why should you use Qwen3-235B-A22B-Instruct-2507:

• State-of-the-art performance: The model sees significant gains in instruction following, reasoning, comprehension, math, science, coding, and tool use. It outperforms models like GPT-4o and DeepSeek-V3 on benchmarks including GPQA, AIME25, and LiveCodeBench.
• Ultra-long context: It features a context length of 262,144 natively and extendable up to over 1 million tokens. This makes it a perfect choice for systems like AI agents, RAG, and long-term conversations. Keep in mind that to process sequences at this scale, you need around 1000 GB of GPU memory (model weights, KV-cache storage, and peak activation memory demands).
• Multilingual strength: Instruct-2507 supports 100+ languages and dialects, with better coverage of long-tail knowledge and stronger multilingual instruction-following than previous Qwen models.

The Qwen team does not stop with Instruct-2507. They note a clear trend: scaling both parameter count and context length for building more powerful and agentic AI. Their answer is the Qwen3-Next series, which focuses on improved scaling efficiency and architectural innovations.

The first release, Qwen3-Next-80B-A3B, comes in both Instruct and Thinking versions. The instruct variant performs on par with Qwen3-235B-A22B-Instruct-2507 on several benchmarks, while showing clear advantages in ultra-long-context tasks up to 256K tokens.

Since Qwen3-Next is still very new, there’s much more to explore. We’ll be sharing more updates later.

Deploy Qwen3-235B-A22B-Instruct-2507Deploy Qwen3-235B-A22B-Instruct-2507

Llama 4 Scout and Maverick#

Meta’s Llama series has long been a popular choice among AI developers. With Llama 4, the team introduces a new generation of natively multimodal models that handle both text and images.

The Llama 4 models all use the MoE architecture:

• Llama 4 Scout: 109B total parameters (17B active), 16 experts.
• Llama 4 Maverick: 400B total parameters (17B active), 128 experts.
• Llama 4 Behemoth (preview): 2T total parameters (288B active), 16 experts. Still in training, Behemoth already outperforms GPT-4.5, Claude Sonnet 3.7, and Gemini 2.0 Pro on several STEM benchmarks.

Why should you use Llama 4 Scout and Maverick:

• Strong performance: Because of distillation from Llama 4 Behemoth, both Scout and Maverick are the best multimodal models in their classes. Maverick outperforms GPT-4o and Gemini 2.0 Flash on many benchmarks (like image understanding and coding), and is close to DeepSeek-V3.1 in reasoning and coding, with less than half the active parameters.

• Efficient deployment:

  • Scout can fit on a single H100 GPU with on-the-fly INT4 quantization (released in BF16).
  • Maverick is officially available in BF16 and FP8 quantized weights. The FP8 version fits on a single H100 DGX host while maintaining strong quality.

• Safety and reliability: Llama 4 Scout and Maverick were released with built-in safeguards to help developers integrate them responsibly. This includes alignment tuning during post-training, safety evaluations against jailbreak and prompt injection attacks, and support for open-source guard models such as Llama Guard and Prompt Guard.

The specific decision between Llama 4 Scout and Maverick depends on your use case:

• If you need to handle long-context scenarios (up to 10M tokens) or your compute is limited, choose Llama 4 Scout.
• If you need best-in-class multimodal performance for reasoning, coding, and visual tasks, choose Llama 4 Maverick.

Also note that it’s been almost half a year since the Llama 4 models were released as of this writing. Newer open-source and proprietary LLMs may already surpass them in certain areas.

Deploy Llama-4-Scout-17B-16E-InstructDeploy Llama-4-Scout-17B-16E-Instruct

GLM-4.5#

The GLM-4.5 series is the latest release from Zhipu AI, built with the goal of creating a truly generalist LLM. The team believes a strong LLM must go beyond just a single domain. It should combine problem solving, generalization, common-sense reasoning, and more capabilities into a single model.

To measure this, they focus on three pillars:

• Agentic abilities: interacting with external tools and the real world.
• Complex reasoning: solving multi-step problems in domains like mathematics and science.
• Advanced coding: tackling real-world software engineering tasks.

The result is the GLM-4.5 series, designed to unify reasoning, coding, and agentic abilities in one model. It comes in two MoE variants:

• GLM-4.5: 355B total parameters (32B active)
• GLM-4.5-Air: 106B total parameters (12B active per token)

Why should you use GLM-4.5:

• Balanced performance: GLM-4.5 ranks well across 12 benchmarks spanning reasoning, agentic tasks, and coding. It delivers near-parity with models like OpenAI o3 and outperforms Claude 4 Opus and Gemini 2.5 Pro in several domains. While not always at the very top, it’s one of the most balanced options available.

  

• Hybrid thinking: Both GLM-4.5 and GLM-4.5-Air support thinking and non-thinking modes. They are flexible for instant chat as well as complex multi-step reasoning and tool use.

• Fully open-source: Zhipu has released base models, hybrid reasoning models, and FP8 versions of both GLM-4.5 and GLM-4.5-Air. Licensed under MIT, they are free to use commercially and for downstream development.

If your chat application involves reasoning, coding, and agentic tasks together, GLM-4.5 is a strong candidate. Note that running the FP8 version of GLM-4.5 needs at least 4× NVIDIA H200 GPUs. For teams with limited resources, GLM-4.5-Air FP8 is a more practical choice, which fits on a single H200.

Ling-1T#

Developed by InclusionAI, Ling-1T is a trillion-parameter non-thinking model built on the Ling 2.0 architecture. It represents the frontier of efficient reasoning, featuring an evolutionary chain-of-thought (Evo-CoT) process across mid-training and post-training stages.

With 1 trillion total parameters and ≈ 50 billion active per token, Ling-1T uses a MoE design optimized through the Ling Scaling Law for trillion-scale stability. The model was trained on more than 20 trillion high-quality, reasoning-dense tokens, supporting up to 128K context length.

Why should you use Ling-1T:

• Efficient reasoning: Ling-1T expands the Pareto frontier between reasoning accuracy and length on tasks like AIME 25. It demonstrates advanced reasoning compression, maintaining high accuracy with fewer generated tokens. Across major math, reasoning and code benchmarks, it outperforms or matches top models like DeepSeek-V3.1-Terminus, GPT-5-main, and Gemini-2.5-Pro.
• Emergent intelligence at trillion-scale: The model exhibits strong emergent reasoning and transfer capabilities. Without extensive trajectory fine-tuning, Ling-1T achieves around 70 % tool-call accuracy (BFCL V3). It can Interpret complex natural-language instructions and transform abstract logic into functional visual components. However, the current release still has room for improvement in multi-turn interaction, long-term memory, and tool use.
• Aesthetic and front-end generation strength: With its hybrid Syntax–Function–Aesthetics reward mechanism, Ling-1T produces not only functional code but also visually refined front-end layouts. It currently ranks first among open-source models on ArtifactsBench. This is especially useful for building applications that combine reasoning and UI generation.

 

---

 

Now let’s take a quick look at some of the FAQs around LLMs.

What is the best open-source LLM now?#

If you’re looking for a single name, the truth is: there isn’t one. The “best” open-source LLM always depends on your use case, compute budget, and priorities.

The open-source LLM space is evolving quickly. New releases often outperform older models within months. In other words, what feels like the best today might be outdated tomorrow.

Instead of chasing the latest winner, it’s better to focus on using a flexible inference platform that makes it easy to switch between frontier open-source models. This way, when a stronger model is released, you can adopt it quickly and apply the inference optimization techniques you need for your workload.

Why should I choose open-source LLMs over proprietary LLMs?#

The decision between open-source and proprietary LLMs depends on your goals, budget, and deployment needs. Open-source LLMs often stand out in the following areas:

• Customization. You can fine-tune open-source LLMs for your own data and workloads. Additionally, you can apply inference optimization techniques such as speculative decoding, prefix caching and prefill-decode disaggregation for your performance targets. Such custom optimizations are not possible with proprietary models.
• Data security. Open-source LLMs can be run locally, or within a private cloud infrastructure, giving users more control over data security. By contrast, proprietary LLMs require you to send data to the provider’s servers, which can raise privacy concerns.
• Cost-effectiveness. While open-source LLMs may require investment in infrastructure, they eliminate recurring API costs. With proper LLM inference optimization, you can often achieve a better price-performance ratio than relying on commercial APIs.
• Community and collaboration. Open-source projects benefit from broad community support. This includes continuous improvements, bug fixes, new features, and shared best practices driven by global contributors.
• No vendor lock-in. Using open-source LLMs means you don’t rely on a single provider’s roadmap, pricing, or availability.

How can I optimize LLM inference performance?#

One of the biggest benefits of self-hosting open-source LLMs is the flexibility to apply inference optimization for your specific use case. Frameworks like vLLM and SGLang already provide built-in support for inference techniques such as continuous batching and speculative decoding.

But as models get larger and more complex, single-node optimizations are no longer enough. The KV cache grows quickly, GPU memory becomes a bottleneck, and longer-context tasks such as agentic workflows stretch the limits of a single GPU.

That’s why LLM inference is shifting toward distributed architectures. Optimizations like prefix caching, KV cache offloading, data/tensor parallelism, and prefill–decode disaggregation are increasingly necessary. While some frameworks support these features, they often require careful tuning to fit into your existing infrastructure. As new models are released, these optimizations may need to be revisited.

At Bento, we help teams build and scale AI applications with these optimizations in mind. You can bring your preferred inference backend and easily apply the optimization techniques for best price-performance ratios. Leave the infrastructure tuning to us, so you can stay focused on building applications.

What should I consider when deploying LLMs in production?#

Deploying LLMs in production can be a nuanced process. Here are some strategies to consider:

1. Model size: Balance accuracy with speed and cost. Smaller models typically deliver faster responses and lower GPU costs, while larger models can provide more nuanced reasoning and higher-quality outputs. Always benchmark against your workload before committing.
2. GPUs: LLM workloads depend heavily on GPU memory and bandwidth. For enterprises to self-host LLMs (especially in data centers), NVIDIA A100, H200, B200 or AMD MI300X, MI350X, MI355X are common choices. Similarly, benchmark your model on the hardware you plan to use. Tools like llm-optimizer can quickly help find the best configuration.
3. Scalability: Your deployment strategy should support autoscaling up or down based on demand. More importantly, it must happen with fast cold starts or your user experience suffers.
4. LLM-specific observability: Apart from traditional monitoring, logging and tracing, also track inference metrics such as Time to First Token (TTFT), Inter-Token Latency (ITL), and token throughput.
5. Deployment patterns: How you deploy LLMs shapes everything from latency and scalability to privacy and cost. Each pattern suits different operational needs for enterprises: BYOC, multi-cloud and cross-region, on-prem and hybrid

Final thoughts#

The rapid growth of open-source LLMs has given teams more control than ever over how they build AI applications. They are closing the gap with proprietary ones while offering unmatched flexibility.

At Bento, we help AI teams unlock the full potential of self-hosted LLMs. By combining the best open-source models with tailored inference optimization, you can focus less on infrastructure complexity and more on building AI products that deliver real value.

To learn more about self-hosting LLMs:

• Read our LLM Inference Handbook
• Benchmark and optimize LLM performance with llm-optimizer
• Join the BentoML Slack community to connect with other builders
• Sign up for our inference platform and deploy your first LLM in the cloud
• Schedule a call with us to discuss your LLM use case