Mistral & Mixtral: Run Open-Weight LLMs

What Is Mistral / Mixtral?

Mistral Large 3 is one of the best permissive open-weight models in the world and Mistral's first mixture-of-experts model since the seminal Mixtral series. The Mistral family encompasses both dense models (7B, 24B, 128B) and sparse Mixture-of-Experts variants. Mixtral 8x7B uses 8 expert networks of 7B parameters each, with only 2 experts activating per token, meaning you get ~13B active parameters while benefiting from 47B total parameters.

The Mixtral model family employs a sparse Mixture-of-Experts (SMoE) architecture where a router selects a subset of expert networks to process each token. This design yields exceptional efficiency: Mixtral 8x7B offers quality closer to a 70B dense model but at the inference cost of a 13B model.

Available Sizes and Variants

The open-weight Mistral / Mixtral lineup includes:

• Mistral 7B - A pretrained generative text model with 7 billion parameters available in base and instruct variants
• Mixtral 8x7B - 46.7B total parameters performing inference at the same speed and cost as models one-third that size, with 32k token context length
• Mixtral 8x22B - Sparse mixture-of-experts with 39B active parameters out of 141B total parameters, featuring 64K token context window
• Mistral Small 3.1 - 24 billion parameters with state-of-the-art vision understanding and 128k token context capabilities
• Mistral Large 3 - State-of-the-art general-purpose Multimodal granular Mixture-of-Experts model with 41B active parameters and 675B total parameters

All open-weight variants are released under the Apache 2.0 license, enabling unrestricted research and commercial use.

Hardware Requirements

VRAM needs vary significantly by model size and quantization:

• Mistral 7B: 6-8GB VRAM with Q4 or Q5 quantization
• Mixtral 8x7B: 24GB VRAM for Q4 quantization (~28.4 GB weights with partial offload)
• Mixtral 8x22B: 48GB+ VRAM for Q4 (~85.9 GB weights, ~97 GB total)

MoE models must hold all experts in VRAM even though only a few activate per token. Mixtral 8x7B in 4-bit quantization needs roughly 28GB and fits on a single NVIDIA RTX 5090 32GB.

For system-level specs: 16 GB RAM minimum (32 GB recommended), any modern 8-core CPU, and an NVMe SSD with 100-500 GB of free space for model files. Mac (Apple Silicon) excels thanks to unified memory and MLX runtime, Windows is best with NVIDIA + CUDA, and Linux offers the widest GPU flexibility including AMD ROCm support.

Benchmark Highlights

Mixtral 8x7B outperformed Llama 2 70B on nine of twelve benchmarks. More specifically, Mixtral significantly outperforms Llama 2 70B in French, German, Spanish, and Italian.

Mixtral 8x7B achieves ~70% on MMLU, which is meaningfully better than Mistral 7B. Mixtral can handle 32k tokens of context, shows strong features in code generation, and with 46.7 billion total parameters but using only 12.9 billion per token, maintains both speed and cost efficiency.

Mistral 8x7B ranks at 43.1 on recent benchmarks, behind Mistral Large 3 (45.7) and Ministral 3 14B Reasoning (47.3). For frontier capabilities, Mistral Large 3 achieves parity with the best instruction-tuned open-weight models on general prompts while demonstrating image understanding and best-in-class performance on multilingual conversations, debuting at #2 in the OSS non-reasoning models category on the LMArena leaderboard.

How to Run It Locally

The simplest path is Ollama, a command-line-first runtime with a lightweight background server and an OpenAI-compatible API. Install from ollama.com, then run:

ollama run mixtral:8x7b

For the 8x22B variant use: ollama run mixtral:8x22b. Ollama's library lists 200+ ready-to-run models spanning Llama, Qwen, DeepSeek, Mistral, Gemma, and OpenAI's open-weight gpt-oss line, each pre-configured with the right prompt template.

Alternative tools also support Mistral / Mixtral models:

• LM Studio - Polished desktop GUI aimed at people who would rather browse and click than type commands, and it also runs a local API server
• Jan - One of the most popular and best-looking local LLM applications, a privacy-first alternative for ChatGPT
• GPT4All - The most beginner-oriented of the three, with a curated, smaller model list and a simpler one-click installation experience

Ollama is built on top of llama.cpp, delivering excellent token-per-second throughput with intelligent memory management and efficient GPU acceleration for NVIDIA (CUDA), Apple Silicon (Metal), and AMD (ROCm) GPUs.

Best Use Cases

Mixtral 8x7B is ideal for text summarization, question and answering, text classification, text completion, and code generation. Mistral 7B supports text summarization, classification, text completion, and code completion.

Code & Technical Tasks: Mistral 7B approaches the performance of CodeLlama 7B on code tasks while remaining highly capable at English language tasks. Mixtral 8x22B uses only 39B active parameters out of 141B and offers unparalleled cost efficiency, with strong maths and coding capabilities and native function calling.

Document Processing & RAG: Mixtral 8x22B's 64K token context window allows precise information recall from large documents, making it excellent for retrieval-augmented generation and long-document summarization.

Multilingual Workflows: Mixtral 8x22B has native multilingual capabilities and strongly outperforms LLaMA 2 70B on HellaSwag, Arc Challenge and MMLU benchmarks in French, German, Spanish and Italian.

Production Inference: Mixtral 8x7B is well-suited for production environments requiring open-weight models for customization or fine-tuning when computational resources or latency are limited.

FAQ

Can I run Mixtral 8x7B on my gaming PC?

Yes, with caveats. Mixtral 8x7B in 4-bit quantization needs roughly 28GB and fits on a single NVIDIA RTX 5090 32GB. Older cards like an RTX 3090 (24GB) require layer offloading techniques. Start with Mistral 7B if you have 8GB; it quantizes to ~4.4 GB.

Which is faster: dense Mistral 7B or sparse Mixtral 8x7B?

Mixtral 8x7B offers quality closer to a 70B dense model but at the inference cost of a 13B model because only 2 of 8 experts activate per token. Mistral 7B is smaller and loads faster, but Mixtral delivers better quality at comparable speed thanks to its sparse architecture.

Are Mistral models safe to use commercially?

Yes. Both base and instruction fine-tuned versions are released under the Apache 2.0 license, providing a strong foundation for further customization across the enterprise and developer communities. No licensing restrictions apply to research or production use.

How does Mixtral 8x22B compare to Mistral Large 3?

Mixtral 8x22B is retired as of March 2025, with Mistral Small 4 recommended for new integrations. Mistral Large 3 is a state-of-the-art general-purpose Multimodal mixture-of-experts model with 41B active parameters and 675B total parameters, offering superior performance but requiring 48GB+ VRAM. For local deployment on consumer hardware, Mixtral 8x7B or Mistral Small 3.1 remain the practical sweet spot.