Best Hardware for Running Local LLMs: Picking a PC, Mac, or GPU by Model Size

Interest in running LLMs (large language models) entirely on your own PC or Mac, instead of relying on cloud services like ChatGPT or Claude, keeps growing. Tools such as Ollama and llama.cpp now let you run open models like Llama 3.1 and Qwen2.5 with no monthly fee and no internet connection. The upsides are real: sensitive data never leaves your machine, and you can experiment as much as you like.

The first hurdle, though, is a simple question: what hardware should you actually buy? Unlike the cloud, running locally means loading the entire model into your own memory, and that is where performance and cost are decided. This guide walks through how to choose among a Mac, an NVIDIA GPU-equipped PC, or a high-memory PC, and in what configuration, based on your intended use and model size, as of July 12, 2026. The technical differences in memory bandwidth between chips are covered in our widely read analysis, “Apple Silicon vs NVIDIA GPU Memory Bandwidth Comparison,” so here we focus on what to buy.

The single most important factor is memory capacity

Two things mostly determine how a local LLM behaves: the capacity of memory to hold the whole model, and the memory bandwidth at which that data moves.

Capacity comes first. If the model does not fit in memory, nothing runs comfortably. If it does not fit in a GPU’s VRAM, it spills into system memory or storage and speed collapses. The basic approach is a two-step one: first secure enough capacity to hold the model you want to run, then think about how fast it runs via bandwidth.

This is where quantization matters. Quantization deliberately lowers the numerical precision inside a model to cut the memory and compute it needs. At the original 16-bit (FP16) precision, roughly 2 GB of memory is needed per 1B (billion) parameters, but the widely used 4-bit (Q4) quantization brings that down to about 0.5-0.6 GB per 1B, sharply reducing memory needs while keeping quality loss small. How quantization works is explained in “Model Compression Explained: Distillation vs Quantization.” The memory figures below assume the commonly used Q4 quantization.

Memory requirements by model size

Model size is expressed in parameter counts like “7B” or “70B” (B means a billion). What that number means is covered in “What Do 7B and 70B Mean? LLM Parameter Counts Explained.” Here we use the actual Q4-quantized file sizes of representative open models as a baseline and lay out the memory you want for comfortable use. The file sizes are the real download sizes from the official Ollama library¹².

Class	Example models	Q4 file size	Recommended memory (RAM/VRAM)
7-8B	Llama 3.1 8B / Qwen2.5 7B	4.7-4.9GB¹²	8GB (minimum) - 16GB (comfortable)
13-14B	Qwen2.5 14B	9.0GB²	16GB
30-34B	Qwen2.5 32B	20GB²	24-32GB
70-72B	Llama 3.1 70B / Qwen2.5 72B	43-47GB¹²	48-64GB or more

The recommended memory is larger than the file size because, at runtime, you need not only the model weights but also headroom for the context that holds conversation history (the KV cache) and for the tool itself. The longer the text you work with, the more that headroom grows.

Running at higher precision needs more memory. At FP16, even the 8B class needs about 16GB, and the 70B class needs the equivalent of roughly 140GB, which is not realistic on consumer machines. Running larger models locally essentially assumes quantization. Decide roughly what size model you want to use day to day, then aim for enough capacity to also hold a model a step or two above it.

Two paths: Apple’s unified memory or NVIDIA’s dedicated VRAM

Hardware for local LLMs largely splits into two camps: Apple Silicon Macs, and Windows/Linux PCs with an NVIDIA GPU. The two hold memory in fundamentally different ways, and that shapes the choice.

Apple’s unified memory shares one memory pool between the CPU and GPU. On a 64GB Mac, for example, most of that can be allocated to the LLM. In other words, memory capacity is close to being the ceiling on the model you can load, which makes it easy to run large models on a single machine. It also draws little power and runs quietly, with no extra power supply or case needed.

NVIDIA’s dedicated VRAM means only the memory on the GPU board is used to run the LLM. VRAM bandwidth is higher than a Mac’s, so the same model tends to generate faster. The CUDA platform is the de facto standard, and many AI tools are optimized for it, which is another advantage. The background to that dominance is touched on in “Why NVIDIA Dominates AI Hardware.” The weakness is capacity: even the consumer flagship RTX 5090 has 32GB³⁴. Running a model larger than that means adding multiple GPUs, which raises the bar on price, power, and physical space.

To summarize the decision axes: if you want to run a large model (70B class) on a single machine, quietly and power-efficiently, Apple’s unified memory suits you. If you prioritize generation speed, also want gaming or image/video generation, or already run Windows/Linux, an NVIDIA GPU fits better. For the concrete bandwidth numbers (M4 Max at 546GB/s, RTX 5090 at 1792GB/s, and so on), see the bandwidth comparison article.

Recommended builds by budget (as of July 12, 2026)

The prices below are rough guides based on official and reported information as of July 12, 2026. As noted later, prices and configurations are unusually fluid right now due to surging memory prices, so check the latest sales pages before buying.

Start here: free with software only, or up to the 16GB class

Before buying hardware, the surest move is to try it on a PC you already own. With 8GB of memory, 7-8B class models will run. If you just want a feel for it on your current machine, there are options like “WebLLM,” which runs an LLM entirely in the browser.

If you are buying something new, a Mac mini with 16GB of memory or a typical laptop is the entry line. The Mac mini’s M4 (120GB/s memory bandwidth) starts at $799 for the base model⁵ and comfortably handles day-to-day use of the 7-14B class.

Mid-range: comfortably running 13-34B

To target a step up, secure 24-48GB of memory. On the desktop, the Mac mini M4 Pro (273GB/s bandwidth, configurable up to 48GB) starts at $1,599⁵⁶ and is a power-efficient, easy-to-handle option.

On the NVIDIA side, the RTX 5080 (MSRP $999) and RTX 5070 Ti (MSRP $749), both with 16GB of VRAM, are the mid-range benchmark⁴. To keep costs down, there are also the 16GB RTX 5060 Ti (MSRP $429) and the 16GB RTX 4060 Ti (MSRP $499), but these use a 128-bit bus with modest bandwidth (448GB/s and 288GB/s respectively), so while capacity is enough, speed trails the higher cards⁴⁷¹⁰. With 16GB of VRAM the 32B class tends to overflow, so if you want to run that tier reliably, a 24-32GB Mac configuration is the cleaner choice.

High end: running the 70B class on one machine

To run a 70B model in the 43-47GB range on a single machine, a Mac with large unified memory is the most realistic option. The Mac Studio M4 Max (410-546GB/s bandwidth, up to 64GB) starts at $2,499, and the higher M3 Ultra (819GB/s bandwidth, 96GB standard) is $5,299⁸. If you need portability, there is also the MacBook Pro with the M5 Max, which supports 128GB of unified memory. The 16-inch M5 Pro model starts at $2,699, but only the higher M5 Max supports 128GB of memory, and a large-memory configuration pushes the price up considerably⁹.

To reach 70B on NVIDIA, even the flagship RTX 5090 at 32GB is not enough on its own, so you would bundle multiple cards such as the 24GB RTX 4090 or the 32GB RTX 5090³⁴⁷. The speed is appealing, but the burden of price, power draw, and setup is heavy, and if single-machine simplicity matters to you, a high-memory Mac has the edge.

A 2026-specific caveat: memory price spikes and fluctuating prices

What deserves particular attention right now is soaring component prices. Demand for AI data centers has tightened supply of memory and storage chips, and the effect has reached consumer machines. Apple raised prices across a wide range of Macs in June 2026, with the Mac mini M4 Pro lifted from $1,399 at its October 2024 launch to $1,599⁵. The company explained the increase by saying it had never seen component prices rise this much, this quickly⁵.

The impact reaches the memory options themselves. The Mac Studio’s ultra-high-capacity 256GB and 512GB configurations, once the go-to for local LLMs, were discontinued in March and May 2026 due to RAM shortages, and as of July 2026 the M4 Max tops out at 64GB and the M3 Ultra at 96GB⁸. The era of “buy one giant Mac and everything fits” has narrowed for the time being, and that is worth factoring into your buying plans.

The same goes for NVIDIA: the RTX 5090’s MSRP is $1,999⁴, but street prices continue to run well above it. If you are considering a GPU, it is safer to budget by the actual retail price rather than the MSRP.

Given all this, if you are not in a hurry, one reasonable move is to pin down the minimum configuration that fits the model size you need and wait for prices to settle. If you do need something now, the basic split holds: Macs are power-efficient and high-capacity, making single-machine setups easy, while NVIDIA wins on speed and versatility. Trying small models on the machine you already have, then investing in capacity once your use case is clear, is the least wasteful way forward.