GLM vs DeepSeek R1: Same Criteria, Same Depth — Who Wins on Stability?
⚠️ Medical disclaimer: All clinical case analyses are for educational and research purposes only. Based on a single anonymized case — not individual medical advice.
Key Findings
With deep thinking + web search enabled, DeepSeek R1 outperforms GLM-5.2 in both accuracy and stability:
Mean 86.5 vs 80.8 (+5.7), range 14 vs 35 (−60%), SD 4.4 vs 9.9 (−55%).
Mean 86.5 vs 80.8 (+5.7), range 14 vs 35 (−60%), SD 4.4 vs 9.9 (−55%).
GLM-5.2 Mean
80.8
Range: 35 | SD: 9.9
DeepSeek R1 Mean
86.5
Range: 14 | SD: 4.4
R1 Advantage
+5.7
Mean +5.7 · Range −21
💡 Key Insight
GLM's outputs fluctuate wildly (63→98, spanning 4 tiers). DeepSeek R1, under the same settings, shows much tighter clustering (79→92, only 2 tiers). For clinical decision support, consistency beats occasional brilliance.
GLM's outputs fluctuate wildly (63→98, spanning 4 tiers). DeepSeek R1, under the same settings, shows much tighter clustering (79→92, only 2 tiers). For clinical decision support, consistency beats occasional brilliance.
Methodology
- Case: 60F, right mastectomy + SLNB; invasive carcinoma invasive carcinoma with low-grade DCIS and ductal papillary carcinoma components, max diameter 0.6cm, with 3 clusters of stromal invasion (approximately 0.05cm, 0.06cm, 0.2cm); ER/PR 3+, HER2-, Ki-67 5%; discharge dx TisN0M0 (incorrect)
- Input: Canonical text prompt (same 5-question format), identical for both models
- Settings: Both models: deep thinking ON + web search ON (same as Part 2 R1 setting)
- Scoring: 16-item/100 rubric (clinical-benchmark/framework)
- Runs: GLM 16 valid (1 INVALID), DeepSeek R1 20 valid
Item-by-Item Comparison (16 Rubric Items)
| Item | Max | GLM Avg | R1 Avg | Diff |
|---|---|---|---|---|
| 1.1 TisN0M0 agreement | 6 | 5.8 | 5.5 | -0.3 |
| 1.2 Staging correction | 10 | 8.4 | 8.5 | — |
| 1.3 Molecular subtype | 4 | 4.0 | 4.0 | — |
| 2.1 Surgical adequacy + ALND | 8 | 6.9 | 6.5 | -0.4 |
| 2.2 Adjuvant radiotherapy | 4 | 3.4 | 3.2 | -0.2 |
| 3.1 Chemo indication + 21-gene | 10 | 8.1 | 9.5 | +1.4 |
| 3.2a Endocrine necessity | 12 | 9.6 | 10.8 | +1.2 |
| 3.2b AI vs TAM + age cut-off | 10 | 7.5 | 10.0 | +2.5 |
| 3.2c Drug dosage & duration | 6 | 4.8 | 5.5 | +0.7 |
| 3.2d OFS/CDK4/6 | 6 | 3.2 | 4.0 | +0.8 |
| 3.3 Anti-HER2 | 4 | 3.8 | 3.5 | -0.3 |
| 4.1 ADE monitoring | 6 | 3.4 | 3.0 | -0.4 |
| 4.2 Bone density & lifestyle | 4 | 2.3 | 1.7 | -0.6 |
| 4.3 Follow-up plan | 4 | 3.6 | 4.0 | +0.4 |
| 4.4 Contralateral breast + BRCA | 4 | 3.1 | 3.5 | +0.4 |
| 5. Self-assessment + uncertainty | 2 | 1.8 | 1.3 | -0.5 |
| Total | 100 | 80.8 | 86.5 | +5.7 |
Conclusion
Under identical deep-thinking + web-search settings, DeepSeek R1 delivers both higher accuracy and greater stability than GLM-5.2.
Mean +5.7, range reduced by 60% (35→14), SD reduced by 55%.
Mean +5.7, range reduced by 60% (35→14), SD reduced by 55%.
Implications for Clinical Use
- Stability > occasional high scores: A model with range 35 is unsuitable for independent decision support
- Deep thinking adds value: Both models improved from quick mode, but R1's consistency is superior
- Model selection: DeepSeek R1 is the more reliable choice for clinical decision support under these settings
Limitations
- Sample size limited (GLM 16 runs, R1 20 runs) — needs more repetition
- Single case tested; conclusions may not generalize
- Scoring based on keyword coverage; may differ from human expert scoring
FAQ
Why does GLM have such a wide range?
GLM's deep thinking mode can produce either high-quality answers (98) or answers missing key items (63), depending on reasoning path consistency. R1's reasoning is more stable.
Is this comparison fair?
Yes. Same case, same prompt, same settings (deep thinking + web search ON), same rubric. Only the model differs.
Is a range of 14 considered stable?
Per our framework (range ≥ 8 = unstable), R1's 14 is still "unstable" but much better than GLM's 35. Ideal target: range < 8.