Qwen3.8-27B Local Install Guide: Ollama, LM Studio, vLLM

Qwen3.8-27B Local Install Guide: Ollama, LM Studio, vLLM

Qwen3.8-27B is Alibaba's open-weight 27.8-billion-parameter model released in August 2026, and it is one of the strongest models you can realistically run on your own computer. It ships under the Apache 2.0 licence with 256K native context and multimodal input, so running it locally means no API bills and no data leaving your machine. This guide covers the exact memory requirements per quantization, how to install it with Ollama, LM Studio, or vLLM, and which hardware it actually fits.

How much VRAM does Qwen3.8-27B need?

This is the question that decides everything. The model itself is 27.8B dense parameters, and the download size depends on quantization:

Realistic hardware picks: a 24 GB card (RTX 3090, RTX 4090, or RTX 5090) is the sweet spot for the Q4 model, while a Mac with 48 GB of unified memory handles it comfortably. A 32 GB Mac works if you keep the context shorter. A 16 GB GPU cannot hold the whole model — layers spill into system RAM, which is much slower. If you only have 12 GB or 8 GB of VRAM, skip to the 2-bit quant section below; community builds like the 7.89 GB Underdog Saluki 27B make it possible with 9-11 GB of combined VRAM and RAM.

Step 1: Install Ollama and pull the model

Ollama is the simplest path on Windows, Mac, and Linux. It handles downloading, quantization selection, and serving behind a local API.

On Linux or WSL, install Ollama with the official script:

curl -fsSL https://ollama.com/install.sh | sh

On Mac, use Homebrew:

brew install ollama

On Windows, download the installer from ollama.com and run it — no manual PATH setup needed.

Then pull the model (about 18 GB at the default Q4_K_M quantization, so this takes a while):

ollama pull qwen3.8:27b

Confirm it landed with ollama list — you should see qwen3.8:27b in the table.

Step 2: Run it and turn off thinking mode

Start an interactive chat:

ollama run qwen3.8:27b

The first prompt is slow because Ollama loads the weights into memory — watch the terminal logs if you started ollama serve separately. One quirk worth knowing: Qwen3.8-27B's thinking (reasoning) mode is ON by default. If you want fast, direct answers instead of a chain of thought before every reply, disable it for the session:

/set think no

Type that inside the Ollama prompt and it takes effect immediately. Leave thinking on for math, code, and logic tasks — it measurably helps there.

To check it works, ask something grounded like "List the files in a Python project as a tree" and verify the output is formatted correctly. If the model stutters or repeats itself at the default quantization, that usually means your machine is offloading layers to RAM — check GPU memory with nvidia-smi while it runs.

Step 3: The GUI path — LM Studio with GGUF quants

If you prefer a desktop app over the terminal, LM Studio is the friendliest option, and it unlocks the community's extra-small quants.

1. Download LM Studio from lmstudio.ai and install it.

2. In the search panel, look for Qwen3.8-27B GGUF files — you will see builds from Unsloth, bartowski, and others.

3. Pick a quantization that fits your VRAM: Q4_K_M (~18 GB) for 24 GB GPUs, IQ3_S (~11.8 GB, matches the original on AIME25 and LiveCodeBench per its publisher) for tighter budgets, or IQ2_XS (~8.4 GB) for 12 GB cards.

4. Click Download, then Load Model, and start chatting in the app.

LM Studio exposes an OpenAI-compatible server (enable it in Developer mode), so any tool that talks to the OpenAI API — OpenCode, Aider, Cline — can point at http://localhost:1234/v1 and use your local Qwen3.8-27B instead of a paid API.

Step 4: Mac setup

Apple Silicon Macs are genuinely good Qwen3.8-27B machines because unified memory acts as one big pool. Sizing guidance from tested setups:

• 48 GB+ Mac: run the full Q4 model with long context, no compromises.
• 32 GB Mac: works, but keep context moderate (32K or below) to avoid swap slowdowns.
• 16 GB Mac: too small for the standard quants — use a 2-bit build instead (see below).

The Ollama path on Mac is just brew install ollama then ollama run qwen3.8:27b. If you want maximum speed, community MLX-based launchers like oMLX (from the macOS menu bar, serves on port 8000 with an OpenAI-compatible API) or MTPLX are purpose-built for Apple Silicon and benchmark faster than the generic Ollama build for coding workloads.

Step 5: Serve it as an API for agents and apps

Once the model is running locally, the real payoff is plugging it into developer tools. Ollama serves an OpenAI-compatible endpoint at http://localhost:11434/v1 by default. In an agentic coding tool like OpenCode, you can launch with:

ollama launch opencode --model qwen3.8:27b

This opens OpenCode's terminal UI with Qwen3.8-27B as the agent model — give it a task like "build a small Python app" and it will write, run, and fix code without touching a cloud API.

For production-ish serving on a Linux server, vLLM is the standard choice. A typical Docker command for the FP8 build:

docker run -d --name qwen --gpus all \
  -v /srv/llm/models:/models:ro \
  vllm/vllm-openai:v0.30.0 \
  --model /models/Qwen3.8-27B-FP8 \
  --host 127.0.0.1 --port 8000 \
  --max-model-len 65536 --gpu-memory-utilization 0.90

Two security habits worth adopting: always bind to 127.0.0.1 (vLLM listens on all interfaces by default), and never pass --trust-remote-code unless you have audited the model files — it lets code bundled with the model execute on your machine.

Tight on VRAM? Try the 2-bit builds

If you have a 12 GB or 8 GB GPU, the standard Q4 model will not fit, but community 2-bit quantizations exist. The most cited is Underdog Saluki 27B, a 7.89 GB GGUF build published in October 2026 using ISTA-DASLab's GSQ-RCO quantization method, which assigns per-tensor precision instead of a flat bit depth. It runs in stock Ollama, LM Studio, and llama.cpp under Apache 2.0.

On a 12 GB card it runs fully on the GPU; on an 8 GB card it holds most layers in VRAM and spills the rest into system RAM — slower, but usable. Expect a quality trade-off: the publisher's own tests put it level with the full model on tool calling and instruction following, a few points behind on coding, and at roughly 82-85 percent of full-model scores on competition math. Verify on your own tasks before relying on it for anything critical.

Troubleshooting common issues

• Slow generation despite enough VRAM: check that all layers are actually on the GPU (nvidia-smi); partial RAM offload is the usual cause.
• Out-of-memory on load: reduce the context window (Ollama: /set parameter num_ctx 16384) or drop to a smaller quant.
• Images are ignored: the standard Ollama build is text-only; vision needs the mmproj projector file (--mmproj) plus a client that actually sends the image.
• Garbled tool calls: Qwen3.8-27B ships with tool calling enabled; if your client formats tools differently, try the Hermes tool-call parser settings from the official model card.

Key takeaways

• Qwen3.8-27B is Alibaba's 27.8B open-weight model (Apache 2.0, 256K context, August 2026) — one of the strongest models that fits on a single machine.
• Budget ~21-22 GB of GPU memory for the Q4 build; a 24 GB NVIDIA card or a 48 GB Mac is the comfortable tier.
• Install with curl -fsSL https://ollama.com/install.sh | sh then ollama pull qwen3.8:27b and ollama run qwen3.8:27b.
• Thinking mode is on by default — disable it with /set think no in Ollama for snappier answers.
• LM Studio gives you a GUI plus smaller community quants (IQ3_S, IQ2_XS); 2-bit builds like the 7.89 GB Saluki fit 8-12 GB cards.
• Serve it at http://localhost:11434/v1 to use it as a free, private coding-agent backend, and bind any server deployment to 127.0.0.1.