Your 16 GB Mac
M2 ProConfirmed from your screenshot. Start with a 4B model; an 8B reasoning model is also realistic.
Yes—for the right model. Macs can run useful local reasoning; phones work best with smaller models and bounded tasks.
Explore token generation across five hardware generations, then see which reasoning models fit. A token is a piece of text; tokens per second measure output speed, not answer quality.
Confirmed from your screenshot. Start with a 4B model; an 8B reasoning model is also realistic.
One or two generations newer, as requested. Exact chip tier is unknown; 32 GB is the planning budget.
Assumed from “two generations old.” Start around 1–2B parameters; the Pro benchmark is a reference, not your measured speed.
Choose a hardware family. Numbers are published results, not tests run on your devices. Blank rows mean evidence is missing—not zero performance.
The Mac chart uses Llama 2 7B Q4_0, an older non-reasoning benchmark. M2–M4 use build 8e672ef; M5 submissions use newer software. The iPhone chart uses TinyLlama 1.1B Q4_0; its Air result uses a newer runtime. Neither chart predicts a specific reasoning model’s speed.
Android is represented by five Snapdragon flagship generations, not every Android vendor. Its older rows use different workloads; only the two Qwen3-1.7B GenieX rows form a closely matched pair. Pixel/Tensor, Exynos and MediaTek phones require separate tests.
Coverage uses the five latest announced generations as of the research date: M2–M6, iPhone 14–18, and Snapdragon 8 Gen 2 through 8 Elite Gen 6. M6 MacBook Pro was announced today and is due November 6; announcements are not measured performance.
These examples show what is possible with specific models and software. Different model sizes and test settings make these examples unsuitable for ranking the devices.
Generated tokens per second · shared 0–125 scale.
The 32 GB M2 Pro is a published reference machine, not the assumed newer Mac. Its Qwen3 rate falls from 96.3 to 54.9 tokens/s when context grows from 4K to 16K. Liquid’s examples use 1K input and 100 output tokens. These are short runs, not sustained capacity measurements.
A reasoning model may produce thousands of thinking tokens before its final answer. Explore the generation time for a 256-token answer.
Scenario calculation: (thinking tokens + 256) ÷ tokens/s. Add model loading, prompt processing, queue and network time. It holds speed constant; it does not predict how many tokens a model needs or whether its answer is correct.
My starting choices: Qwen3-1.7B on the iPhone, Qwen3.5-4B or DeepSeek-R1-0528-Qwen3-8B on the 16 GB Mac, and gpt-oss-20b on the 32 GB Mac. These are practical trial candidates, not a quality leaderboard.
B means billion model parameters. Approximate download sizes assume a compact quantized build—usually 4-bit, which stores weights at reduced precision. Sizes vary by package and are not total working memory. Fit judgments are planning recommendations at short context, not measured compatibility certifications.
| Model and tradeoff | Download | iPhone | 16 GB Mac | 32 GB Mac |
|---|---|---|---|---|
| LFM2.5-1.2B-ThinkingCompact reasoning for extraction and narrow tasks; the author cautions against knowledge-heavy work and programming. Custom Liquid license. | ~0.7–1.3 GB | Start here | Easy fit | Easy fit |
| Qwen3-1.7BSwitchable thinking mode. Use a 4-bit build and limit the reasoning budget. Apache 2.0. | ~1–1.5 GB | Start here | Easy fit | Easy fit |
| DeepSeek-R1-Distill-Qwen-1.5BOlder compact reasoning distill; useful for experiments, less capable than larger models. MIT. | ~1–1.5 GB | Try it | Easy fit | Easy fit |
| Qwen3.5-2BThinking is available but off by default. Requires a runtime with Qwen3.5 support. Apache 2.0. | ~2–3 GB | Try it* | Easy fit | Easy fit |
| Qwen3-4B-Thinking-2507Dedicated reasoning model. Phone use needs short context and a compatible app. Apache 2.0. | ~2.5–3 GB | Conditional* | Good starter | Easy fit |
| Qwen3.5-4BThinking enabled by default. Start with a 4-bit build and 4K context. Apache 2.0. | ~3–4 GB | Conditional* | Good starter | Easy fit |
| DeepSeek-R1-0528-Qwen3-8BPublished M2 Pro 16 GB reference: 23.1 generated tokens/s. MIT. | ~5–6 GB | Skip initially | Good fit | Easy fit |
| Qwen3.5-9BLeave memory for macOS, apps and conversation cache. Apache 2.0. | ~6–8 GB | Skip initially | With headroom | Good fit |
| Underdog Saluki 27B 1.0IQ2-mix compression of Qwen3.8-27B, focused on preserving tool use. Local coding evaluation pending; weights alone are not total RAM. Apache 2.0. | 7.90 GB | Skip | Conditional* | Candidate |
| gpt-oss-20bOpenAI reasoning model with low/medium/high effort. Use a supported quantized build; all experts still need storage. Apache 2.0. | ~12–14 GB | Skip | Tight / trial only | Good candidate |
| Qwen3-30B-A3B-Thinking-250730B total parameters, about 3B active per token. Reserve memory for the full model plus cache. Apache 2.0. | ~18–20 GB | Skip | Skip | Conditional* |
* Conditional: start with 2K–4K context, close other apps and confirm the runtime supports that architecture. On phones, app memory limits and heat can stop a model that appears to fit on paper. Larger contexts need additional cache memory. A 32 GB planning budget does not assert that every M3/M4 chip tier ships in that exact configuration.
Qwen models listed here use Apache 2.0; the DeepSeek distills use MIT; gpt-oss uses Apache 2.0. Liquid uses its own license, so check its terms before building a paid service. The model-name links lead to the original cards.
Qwen3-1.7B supports thinking and non-thinking modes. Qwen3.5-2B defaults to non-thinking; Qwen3.5-4B and 9B default to thinking. The dedicated Thinking and DeepSeek R1 variants are reasoning models. For gpt-oss, use the runtime’s supported reasoning-effort and chat-format handling.
Use Ollama for straightforward model management, or MLX LM for an Apple-native inference and benchmark toolkit. Start with one model and one request at a time.
ollama run qwen3.5:4b
# On the 32 GB Mac:
ollama run gpt-oss:20bThese commands download model files. Nothing has been installed or benchmarked on your Macs as part of this page.
PocketPal is an open-source iOS/Android starting point with model downloads and a benchmark screen. Confirm support for the chosen model; a compatible file format alone is not enough.
Begin with Qwen3-1.7B in a 4-bit GGUF package. For Android, Qualcomm’s optimized runtime is a separate integration path; its results should not be expected automatically in PocketPal.
For each device, record the model hash, quantization, app/runtime version, prompt length, generated tokens, time to first token, peak memory and a 20-minute sustained run. Test answer correctness on real tasks. This is the harness needed before advertising throughput or pricing.
Underdog Saluki 27B is an Apache 2.0, 7.90 GB mixed-precision quantization of Qwen3.8-27B. Its focus is retaining tool use in a smaller download. That is different from training a specialist on your own tasks.
| Test | Saluki | Full-size base |
|---|---|---|
| Tool selection · 120 tasks | 88 / 120 | 84 / 120 |
| Parallel tool calls · 100 tasks | 42 / 100 | 35 / 100 |
| SWE-bench Verified subset · 50 issues | 30 / 50 | 33 / 50 |
| MBPP+ coding score | 78.0 | 83.9* |
Source: publisher’s benchmark table. The author notes small-sample variation; *the base MBPP+ score uses a different public harness. The 50-issue result is not the full SWE-bench Verified benchmark. “96% retention” is an aggregate claim, not a guarantee for a particular coding task.
Local evaluation is pending. As of October 27, the complete file is checksum-verified on a download host; transfer to the M2 Pro is underway. A prepared suite covers six Python tasks, two JavaScript tasks and three tool-call checks. No local coding success rate or Saluki token speed is claimed yet.
Start with two candidate services: private structured extraction with schema validation, and small code fixes checked against tests. Compare Saluki with a smaller 4-bit model under the same memory, context and latency budget. Its 54 GB parent is an informative compression baseline, but not the only commercial alternative.
Measure cost per accepted task, including failures, retries, validation and idle capacity. Keep model versions visible for reproducibility, even when customers choose a task rather than a model name. Add task-specific fine-tuning only after repeated failures show what training needs to improve.
Privacy is a property of where the job runs and who can access it. Local-only execution or customer-controlled workers support a clear promise. An unrelated supplier’s Mac can read the request; encrypted networking alone does not make that supplier blind. Any cloud fallback should be explicit and optional.
Memory capacity decides what fits. Memory bandwidth often limits one-at-a-time token generation. Sustained heat, competing apps and the amount of time a device is available affect how much work it can sell. Tokens per second alone cannot establish cost per watt; measure energy at the wall and completed, acceptable answers.
Unlimited phone data does not improve inference speed. It only helps transfer requests and model files; carrier conditions, background execution and heat still matter. A phone is a better first experiment for private on-device work than an always-available server.
For pooling your Macs, send whole jobs to a trusted idle worker and cache model files there. Tailscale protects the connection, but the receiving computer still sees the prompt. A work laptop should only participate within its owner’s device policy.
Download chart data Download research notes
Retrieved October 27, 2026. Community results are self-reported; vendor results use their own configurations. Missing results are intentionally left blank. No throughput estimate is presented as a measurement.