Research · October 27, 2026

Can consumer hardware provide useful compute?

Yes—for the right model. Macs can run useful local reasoning; phones work best with smaller models and bounded tasks.

Explore token generation across five hardware generations, then see which reasoning models fit. A token is a piece of text; tokens per second measure output speed, not answer quality.

Your 16 GB Mac

M2 Pro

Confirmed from your screenshot. Start with a 4B model; an 8B reasoning model is also realistic.

Your 32 GB Mac

M3 or M4 scenario

One or two generations newer, as requested. Exact chip tier is unknown; 32 GB is the planning budget.

Your iPhone

iPhone 16 family

Assumed from “two generations old.” Start around 1–2B parameters; the Pro benchmark is a reference, not your measured speed.

Five generations, at a glance

Choose a hardware family. Numbers are published results, not tests run on your devices. Blank rows mean evidence is missing—not zero performance.

Published generation speedtokens / second · 0–125 →

Why the charts are not a universal device ranking

The Mac chart uses Llama 2 7B Q4_0, an older non-reasoning benchmark. M2–M4 use build 8e672ef; M5 submissions use newer software. The iPhone chart uses TinyLlama 1.1B Q4_0; its Air result uses a newer runtime. Neither chart predicts a specific reasoning model’s speed.

Android is represented by five Snapdragon flagship generations, not every Android vendor. Its older rows use different workloads; only the two Qwen3-1.7B GenieX rows form a closely matched pair. Pixel/Tensor, Exynos and MediaTek phones require separate tests.

Coverage uses the five latest announced generations as of the research date: M2–M6, iPhone 14–18, and Snapdragon 8 Gen 2 through 8 Elite Gen 6. M6 MacBook Pro was announced today and is due November 6; announcements are not measured performance.

What reasoning models actually achieve

These examples show what is possible with specific models and software. Different model sizes and test settings make these examples unsuitable for ranking the devices.

Generated tokens per second · shared 0–125 scale.

The 32 GB M2 Pro is a published reference machine, not the assumed newer Mac. Its Qwen3 rate falls from 96.3 to 54.9 tokens/s when context grows from 4K to 16K. Liquid’s examples use 1K input and 100 output tokens. These are short runs, not sustained capacity measurements.

Fast tokens can still mean a long wait

A reasoning model may produce thousands of thinking tokens before its final answer. Explore the generation time for a 256-token answer.

generation time

Scenario calculation: (thinking tokens + 256) ÷ tokens/s. Add model loading, prompt processing, queue and network time. It holds speed constant; it does not predict how many tokens a model needs or whether its answer is correct.

Reasoning models that fit

My starting choices: Qwen3-1.7B on the iPhone, Qwen3.5-4B or DeepSeek-R1-0528-Qwen3-8B on the 16 GB Mac, and gpt-oss-20b on the 32 GB Mac. These are practical trial candidates, not a quality leaderboard.

B means billion model parameters. Approximate download sizes assume a compact quantized build—usually 4-bit, which stores weights at reduced precision. Sizes vary by package and are not total working memory. Fit judgments are planning recommendations at short context, not measured compatibility certifications.

Local reasoning shortlist · phone column assumes an iPhone 16-class device
Model and tradeoffDownloadiPhone16 GB Mac32 GB Mac
LFM2.5-1.2B-ThinkingCompact reasoning for extraction and narrow tasks; the author cautions against knowledge-heavy work and programming. Custom Liquid license.~0.7–1.3 GBStart hereEasy fitEasy fit
Qwen3-1.7BSwitchable thinking mode. Use a 4-bit build and limit the reasoning budget. Apache 2.0.~1–1.5 GBStart hereEasy fitEasy fit
DeepSeek-R1-Distill-Qwen-1.5BOlder compact reasoning distill; useful for experiments, less capable than larger models. MIT.~1–1.5 GBTry itEasy fitEasy fit
Qwen3.5-2BThinking is available but off by default. Requires a runtime with Qwen3.5 support. Apache 2.0.~2–3 GBTry it*Easy fitEasy fit
Qwen3-4B-Thinking-2507Dedicated reasoning model. Phone use needs short context and a compatible app. Apache 2.0.~2.5–3 GBConditional*Good starterEasy fit
Qwen3.5-4BThinking enabled by default. Start with a 4-bit build and 4K context. Apache 2.0.~3–4 GBConditional*Good starterEasy fit
DeepSeek-R1-0528-Qwen3-8BPublished M2 Pro 16 GB reference: 23.1 generated tokens/s. MIT.~5–6 GBSkip initiallyGood fitEasy fit
Qwen3.5-9BLeave memory for macOS, apps and conversation cache. Apache 2.0.~6–8 GBSkip initiallyWith headroomGood fit
Underdog Saluki 27B 1.0IQ2-mix compression of Qwen3.8-27B, focused on preserving tool use. Local coding evaluation pending; weights alone are not total RAM. Apache 2.0.7.90 GBSkipConditional*Candidate
gpt-oss-20bOpenAI reasoning model with low/medium/high effort. Use a supported quantized build; all experts still need storage. Apache 2.0.~12–14 GBSkipTight / trial onlyGood candidate
Qwen3-30B-A3B-Thinking-250730B total parameters, about 3B active per token. Reserve memory for the full model plus cache. Apache 2.0.~18–20 GBSkipSkipConditional*

* Conditional: start with 2K–4K context, close other apps and confirm the runtime supports that architecture. On phones, app memory limits and heat can stop a model that appears to fit on paper. Larger contexts need additional cache memory. A 32 GB planning budget does not assert that every M3/M4 chip tier ships in that exact configuration.

Model licenses and reasoning modes

Qwen models listed here use Apache 2.0; the DeepSeek distills use MIT; gpt-oss uses Apache 2.0. Liquid uses its own license, so check its terms before building a paid service. The model-name links lead to the original cards.

Qwen3-1.7B supports thinking and non-thinking modes. Qwen3.5-2B defaults to non-thinking; Qwen3.5-4B and 9B default to thinking. The dedicated Thinking and DeepSeek R1 variants are reasoning models. For gpt-oss, use the runtime’s supported reasoning-effort and chat-format handling.

Run the experiment with existing software

Mac: a small private worker

Use Ollama for straightforward model management, or MLX LM for an Apple-native inference and benchmark toolkit. Start with one model and one request at a time.

ollama run qwen3.5:4b
# On the 32 GB Mac:
ollama run gpt-oss:20b

These commands download model files. Nothing has been installed or benchmarked on your Macs as part of this page.

Phone: a bounded local task

PocketPal is an open-source iOS/Android starting point with model downloads and a benchmark screen. Confirm support for the chosen model; a compatible file format alone is not enough.

Begin with Qwen3-1.7B in a 4-bit GGUF package. For Android, Qualcomm’s optimized runtime is a separate integration path; its results should not be expected automatically in PocketPal.

For each device, record the model hash, quantization, app/runtime version, prompt length, generated tokens, time to first token, peak memory and a 20-minute sustained run. Test answer correctness on real tasks. This is the harness needed before advertising throughput or pricing.

Saluki: promising compression, not a coding verdict

Underdog Saluki 27B is an Apache 2.0, 7.90 GB mixed-precision quantization of Qwen3.8-27B. Its focus is retaining tool use in a smaller download. That is different from training a specialist on your own tasks.

Publisher-reported results · not our local measurements
TestSalukiFull-size base
Tool selection · 120 tasks88 / 12084 / 120
Parallel tool calls · 100 tasks42 / 10035 / 100
SWE-bench Verified subset · 50 issues30 / 5033 / 50
MBPP+ coding score78.083.9*

Source: publisher’s benchmark table. The author notes small-sample variation; *the base MBPP+ score uses a different public harness. The 50-issue result is not the full SWE-bench Verified benchmark. “96% retention” is an aggregate claim, not a guarantee for a particular coding task.

Local evaluation is pending. As of October 27, the complete file is checksum-verified on a download host; transfer to the M2 Pro is underway. A prepared suite covers six Python tasks, two JavaScript tasks and three tool-call checks. No local coding success rate or Saluki token speed is claimed yet.

Offer a result, with a model behind it

Start with two candidate services: private structured extraction with schema validation, and small code fixes checked against tests. Compare Saluki with a smaller 4-bit model under the same memory, context and latency budget. Its 54 GB parent is an informative compression baseline, but not the only commercial alternative.

Measure cost per accepted task, including failures, retries, validation and idle capacity. Keep model versions visible for reproducibility, even when customers choose a task rather than a model name. Add task-specific fine-tuning only after repeated failures show what training needs to improve.

Privacy is a property of where the job runs and who can access it. Local-only execution or customer-controlled workers support a clear promise. An unrelated supplier’s Mac can read the request; encrypted networking alone does not make that supplier blind. Any cloud fallback should be explicit and optional.

Useful locally does not yet mean profitable compute

Memory capacity decides what fits. Memory bandwidth often limits one-at-a-time token generation. Sustained heat, competing apps and the amount of time a device is available affect how much work it can sell. Tokens per second alone cannot establish cost per watt; measure energy at the wall and completed, acceptable answers.

Unlimited phone data does not improve inference speed. It only helps transfer requests and model files; carrier conditions, background execution and heat still matter. A phone is a better first experiment for private on-device work than an always-available server.

For pooling your Macs, send whole jobs to a trusted idle worker and cache model files there. Tailscale protects the connection, but the receiving computer still sees the prompt. A work laptop should only participate within its owner’s device policy.

Sources and reusable data

Download chart data Download research notes

  1. llama.cpp M-series benchmarks
  2. llama.cpp A-series benchmarks
  3. Galaxy S23 benchmark and backend comparison
  4. Qualcomm: Llama 2 7B demo, up to 20 tokens/s
  5. Qualcomm AI Hub: Qwen3-1.7B metrics
  6. Liquid AI: LFM2.5-1.2B-Thinking benchmarks
  7. LocalScore: DeepSeek R1 0528 Qwen3 8B
  8. oMLX: Qwen3-1.7B on M2 Pro, 32 GB
  9. Apple: M6 MacBook Pro announcement, October 27, 2026
  10. Apple: iPhone 18 Pro announcement
  11. Qualcomm: Snapdragon 8 Elite Gen 6 announcement
  12. Conway Research: Saluki model card and benchmark limitations
  13. OpenAI: gpt-oss-20b
  14. OpenAI: running gpt-oss locally with Ollama
  15. Ollama: Qwen3.5 model downloads
  16. PocketPal: open-source iOS and Android app
  17. MLX LM: Apple Silicon inference and benchmarking

Retrieved October 27, 2026. Community results are self-reported; vendor results use their own configurations. Missing results are intentionally left blank. No throughput estimate is presented as a measurement.