IBM releases Modular Granite Switch 4.1 models in 8B and 30B sizes – Unite.AI

IBM announced Granite Switch 4.1 on October 15, 2026, releasing two modular language models, with 8 billion and 30 billion parameters, comprising a base model, a general-purpose decision adapter, a security adapter, and a text-to-speech encoder embedded in a single hosted endpoint. The IBM Granite blog post announcing the release was written by Kate Soule, Subhajit Chaudhury, Hendrik Strobelt, and Luis A Lastras.

The Configuration 4.1

The release includes Granite-Switch-4.1-8B and Granite-Switch-4.1-30B, based on the basic granite-4.1-8b and granite-4.1-30b models, carrying 8 billion and 30 billion parameters, respectively, and a 128K context window. The Model 8B sheet and Model 30B sheet list a release date of October 14, 2026, an Apache 2.0 license, and Hugging Face Transformers and vLLM backend.

Each checkpoint composes three components with the basic model: an activated LoRA decision (aLoRA) for general decision making; guardian-core, an aLoRA from IBM’s Guardian library that detects security risks including harm, jailbreak, profanity, violence, sexual content, social bias, and unethical behavior; and Granite Speech 5.0 TurboCTC, a non-autoregressive, English, 470 million parameter conformal CTC encoder built-in for transcription, which IBM describes as the fastest ASR model in the OpenASR ranking. IBM says that all three technologies, along with the base model, require approximately the same GPU memory as the base model itself.

The cards also list preview variants without sound (granite-switch-4.1-8b-preview, granite-switch-4.1-30b-preview, and granite-switch-4.1-3b-preview) which provide the complete Granite Libraries adapter package. IBM introduced the Granite Switch project in spring 2026 as a modular approach to model development, and its previous version of Granite Libraries provides the repository of specialized decision and task adapters that can be composed into custom Switch models.

How the gearbox works

Granite Switch uses what model boards call coarse-grained expert switching, where a single adapter remains active on all decoder levels for a contiguous range of tokens. Callers activate an adapter by passing its name to the chat model, which automatically inserts the corresponding control token. A standard attention-based lightweight switching layer detects such tokens and produces adapter indices per location, directing each decoding layer to apply the corresponding LoRA weights for that range.

The cards list four design properties: each adapter reads only the KV cache of the base model and never the internal state of another adapter; adapters are developed, tested and published independently, without joint training; a single checkpoint works with both Transformers and vLLM; and adapter selection requires no code changes. IBM says the cache-efficient Activated LoRA design allows the KV cache of a base model generation to be reused for subsequent adapter-based decisions, and credits its recent contributions to the vLLM kernel and a co-located ASR encoder for quickly switching between chat, decision, and voice requests.

The decision adapter receives a state and a question with labeled options, then produces a softmax probability distribution over those labels in a single forward pass. The incoming audio is transcribed by the built-in speech encoder, and the resulting transcription replaces the audio within the chat context, so the transcription runs on the same chat and decision endpoint without a separate voice server and without any changes to how the model is called. IBM also hosts a Speech-to-Decisions demo space where the 8B model transcribes incoming audio, produces summaries, checks for malicious content, and evaluates 15 additional user-defined flags, all in real time on an endpoint.

Benchmarks and comparisons reported by IBM

IBM contrasts its approach with closed decision models like Jev, which it says focuses on generality, and says the guardian-core adapter on average outperforms Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions, and Clef-Flash on security-related decision tasks. In IBM’s published comparison of its specialized Granite Libraries adapters and generalist decision models, the best Granite adapter came ahead on OOD Safety by 2.3 points, IFEval by 17.0, BIO F1 by 11.0, and RAGTruth by 11.8, while finishing behind on LM-AggreFact by 2.6, InfoBench-GPT4 by 3.2, and InfoBench-Human by 0.8.

The model cards report decision index scores for the decision adapter, measured on the 1,068-line public fast subset of the edition-v0.3 score benchmark with the adapter served behind a hosted vLLM endpoint. Adapter 8B achieved an overall score of 42.9, with 29.1 for Knowledge, 48.9 for Language, 49.8 for Retrieval, 59.8 for Tools, and 18.0 for Arts. Adapter 30B achieved an overall score of 50.1, with 32.4 for Knowledge, 58.6 for Language, 49.8 for Retrieval, 76.5 for Tools and 26.4 for Arts. The papers state that the local and hosted service of the same adapter agree within about 0.6 points.

IBM also reports a race in which Granite-Switch-4.1-30B, a frozen Granite base with four specialized aLoRAs, completed a five-step real-world RAG workflow much faster than Jev-27B, a frozen Qwen base with a generic LoRA decision. In a second demonstration, IBM says that a custom 3 billion parameter Granite Switch model playing the game Doom made 60 to 100 decisions per second, compared to the 10 decisions per second reported for Jev, and that unlike Jev its model can narrate and answer questions during gameplay.

Availability, requirements and limitations

Both checkpoints are licensed under Apache 2.0. Requirements are Python 3.9 or later, PyTorch 2.11.0 or later, Transformers 5.16 to 5.17.0, and optionally vLLM 0.26.0 up to and including 0.28.0. The dense decoder-only core architecture uses grouped query attention (GQA), RoPE, SwiGLU, and RMSNorm; supports 131,072 context tokens; and lists English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese as supported languages.

Anyone can compose a custom Granite Switch model by combining adapters from the project’s adapter catalog with related composition tools. IBM positions Mellea as the preferred application layer for running Granite Switch adapters, and Mellea already packages the Granite Libraries adapter families, but the Decision Adapter does not yet have Mellea intrinsic, with support planned. IBM also plans an Adapter SDK for training custom and third-party adapters in Mellea, along with tutorials and support for Granite Switch, and says it will release additional details and data on Granite Switch in the weeks following the announcement.

The papers document several limitations. Speech cascade works only in English and returns lowercase, unpunctuated transcriptions without decoding time control, and because the ASR encoder is not jointly trained with the language model or adapters, an incorrect transcription can silently change what the model or an adapter responds to. vLLM limits decision option labeled logprob requests to 128 token IDs, so questions with multiple options must be split into block requests. The papers also state that guardian-core is not a replacement for application-level security testing, that checkpoints override the Guardian Library’s factuality detection, factivity correction, and policy guardrail adapters, and that decision adapter probabilities should be read as a precedent informing a decision rather than a certified classification for domains outside of its training.