Microsoft on Wednesday open-sourced Phi-4-reasoning-vision-15B, a 15-billion-parameter multimodal model that can process images and text, handle math and science problems, and navigate graphical user interfaces. The model ships with a feature most competitors lack: the ability to activate or suppress its own chain-of-thought reasoning depending on the task. Microsoft trained the entire system on roughly 200 billion tokens of multimodal data, about one-fifth of what rivals like Alibaba's Qwen family and Google's Gemma3 consumed during training.
The release, available immediately on Microsoft Foundry, HuggingFace, and GitHub under a permissive license, represents Microsoft's clearest bet yet that smaller models built on carefully curated data can match systems many times their size where it counts.
The Breakdown
- Microsoft open-sourced Phi-4-reasoning-vision-15B, a 15B-parameter multimodal model that selectively activates chain-of-thought reasoning.
- Trained on 200 billion tokens, about one-fifth of what Qwen3 VL and Kimi-VL consumed during training.
- Scores 88.2 on ScreenSpot v2 for UI grounding and 75.2 on MathVista, trailing larger Qwen3-VL-32B but at a fraction of the compute cost.
- Designed as a perception layer for computer-use agents that navigate interfaces from screenshots alone.
A model that knows when to shut up
Most reasoning models operate in binary. You turn thinking on, or you turn it off. The model follows that instruction regardless of whether the task actually benefits from multi-step reasoning. Ask a reasoning model to caption a photo and it will dutifully generate a chain-of-thought trace before telling you it sees a dog. That burns compute and adds latency for zero accuracy gain.
Microsoft built Phi-4-reasoning-vision-15B to break that pattern. The team trained it on a hybrid data mixture, roughly 20 percent reasoning traces and 80 percent direct responses, tagged with explicit mode tokens. Upload a calculus problem and the model fires up structured step-by-step reasoning. Hand it a receipt to read and it just answers. No internal monologue, no wasted tokens. Research has shown that chain-of-thought reasoning actually degrades performance on perception tasks like OCR and image captioning. Selective activation is not a convenience feature. It is a performance one.
Override is still possible. Developers can force either mode with explicit tokens. But the hybrid default performed better on average across Microsoft's own benchmarks than forcing thinking or non-thinking for every query. The model supports three modes. Hybrid lets it decide on its own. Think forces reasoning on every prompt. Nothink skips it entirely.
"For tasks such as image captioning and optical character recognition, reasoning is often unnecessary and can even be harmful," the Microsoft Research team wrote, "while mathematical and scientific problem-solving benefit from multi-step reasoning."
One-fifth the training data, competitive results
The training efficiency numbers stand out. Phi-4-reasoning-vision-15B consumed about 200 billion tokens of multimodal data, stacked on top of the Phi-4-Reasoning language backbone at 16 billion tokens and the base Phi-4 model at 400 billion unique tokens. Competitors like Qwen3 VL and Moonshot AI's Kimi-VL each used more than one trillion tokens. That gap translates directly to training cost and energy consumption.
Microsoft's researchers credit data curation, not scale. Team members manually reviewed samples from each dataset, spending five to ten minutes per source to classify quality. Bad answers got re-generated using GPT-4o and o4-mini. Unsalvageable questions with good images got recycled as seeds for new training data. The team also reported finding "a surprisingly large number of formatting and logical errors across widely used open-source datasets," a detail that should worry anyone building on public training data without quality checks.
The whole model trained in four days on 240 Nvidia B200 GPUs. That's fast by industry standards, and it makes the economics approachable for organizations that want to fine-tune or reproduce the work.
Where it wins, and where it doesn't
Benchmark results tell a mixed story. On Microsoft's own evaluations across ten benchmarks, Phi-4-reasoning-vision-15B scored 84.8 on AI2D for science diagrams, 83.3 on ChartQA, 75.2 on MathVista, and 88.2 on ScreenSpot v2 for UI element grounding. It beat Google's Gemma3-12B by 17 percent on MathVista_Mini.
Join 10,000+ AI professionals
Strategic AI news from San Francisco. No hype, no "AI will change everything" throat clearing. Just what moved, who won, and why it matters. Daily at 6am PST.
No spam. Unsubscribe anytime.
But it trails the larger Qwen3-VL-32B models on most measures. Qwen3-VL-32B scored 81.8 on MathVista versus 75.2 for Phi-4, and 70.6 on MMMU, a broad multimodal understanding test, versus 54.3. On the hardest math benchmarks with forced thinking enabled, the gap widens further: 78.2 for Qwen3-VL-32B on MathVerse compared to 53.1 for Microsoft's model.
Microsoft was transparent about its methodology. All evaluations ran with temperature at zero, greedy decoding, and a 4,096 maximum output token limit. No custom prompting or parameter tuning. The team committed to releasing all evaluation logs publicly, a practice that remains rare in the field. "These numbers are provided for comparison and analysis rather than as leaderboard claims," they wrote. Refreshing honesty, given how many labs cherry-pick evaluation conditions.
The real value shows up when you plot accuracy against compute time. Phi-4-reasoning-vision-15B sits at what researchers call the Pareto frontier, delivering competitive accuracy at a fraction of the inference cost. For developers who need good-enough results at low latency, that tradeoff matters more than winning every benchmark.
Built for agents, not just benchmarks
Microsoft pushed hard on one application in particular. Computer-use agents that navigate desktop, web, and mobile interfaces. Feed the model a screenshot and it picks out buttons, menus, text fields, then spits back bounding box coordinates. A bot could work through an entire e-commerce checkout that way. Click the size dropdown. Add to cart. Fill in the shipping address. All from pixels, no API access required. That makes it a perception layer for software agents that run without anyone watching.
Microsoft paired this with a mid-fusion architecture combining a SigLIP-2 vision encoder and the Phi-4-Reasoning language backbone. Only some layers process multimodal data, trading marginal output quality for hardware efficiency. The team tested four approaches to image resolution handling, including multi-crop and dynamic tiling methods, and settled on SigLIP-2's Naflex variant with up to 3,600 tokens. That corresponds to roughly 720p resolution and delivered particularly strong results on fine-grained visual tasks like ScreenSpot-Pro.
For agent developers, the model's compact footprint matters. Fifteen billion parameters runs on modest hardware. Low inference latency means real-time interaction with live interfaces. And the selective reasoning keeps the model from burning compute on simple perception tasks that just need a fast, direct answer.
What this signals
Microsoft is betting that the AI market will split. Frontier models for the problems that demand brute force. Smaller, cheaper models for the vast majority of tasks that do not. A year ago the Phi family was a research curiosity. Now it spans language, vision, and robotics, and Microsoft looks emboldened by the pace. Still, the 20/80 reasoning split is a heuristic. The team acknowledged as much, noting it may not hold across every domain. Whether the model correctly decides when to reason remains what they called "an open problem."
The economics point in one direction regardless. A 15-billion-parameter model that delivers 80 to 90 percent of a frontier model's accuracy at a tenth of the inference cost opens deployment scenarios that trillion-parameter systems cannot touch. Edge devices. Interactive applications. On-premise servers where data cannot leave the building.
The weights, fine-tuning code, and benchmark logs are all public. Microsoft is giving away the model and betting developers will build on Azure. The leaderboard, as Microsoft noted, is open. So is the question of whether selective thinking actually works better than always-on reasoning when millions of real users start pushing the boundaries.
Frequently Asked Questions
What does selective reasoning mean in Phi-4-reasoning-vision-15B?
The model automatically decides whether to use chain-of-thought reasoning based on the task. Math and science problems trigger multi-step reasoning traces. Perception tasks like image captioning and OCR get direct answers without internal deliberation. Developers can override this with explicit think or nothink tokens.
How does mid-fusion architecture differ from early fusion?
Mid-fusion uses a separate pretrained vision encoder (SigLIP-2) that converts images into tokens, which are then projected into the language model's embedding space. Only some layers handle multimodal data. Early fusion processes images and text together in a single transformer, producing richer representations but requiring significantly more compute and training data.
Why did Microsoft train on so much less data than competitors?
Microsoft credits meticulous data curation over scale. Team members manually reviewed datasets, re-generated bad answers using GPT-4o, recycled good images with poor questions, and fixed formatting errors in open-source datasets. The result was 200 billion high-quality tokens versus the trillion-plus tokens competitors used.
Can Phi-4-reasoning-vision-15B run on consumer hardware?
At 15 billion parameters, the model is compact enough for modest hardware setups. Microsoft optimized it for low inference latency, making it suitable for edge devices, interactive applications, and on-premise servers. The four-day training run on 240 B200 GPUs also suggests the economics are approachable for fine-tuning.
How does Phi-4-reasoning-vision-15B compare to Qwen3-VL-32B?
Phi-4 trails Qwen3-VL-32B on most raw accuracy benchmarks. Qwen3 scored 81.8 on MathVista versus 75.2 for Phi-4, and 70.6 on MMMU versus 54.3. But Phi-4 delivers those results at a fraction of the inference cost and compute time, sitting on the Pareto frontier of the accuracy-efficiency tradeoff.
