RESEARCH
LLM-Assisted Threat Hunting: Results from 6 Months of Production Testing
We ran 3 threat hunting teams — manual, SIEM+rules, and LLM-assisted. The results on dwell time reduction and analyst fatigue were striking.
From October 2025 to March 2026, SentinelPRO IA's research team ran a controlled study across three customer environments (2,000–2,500 endpoints, financial services) using three different threat hunting configurations. Goal: quantify the impact of LLM-assisted triage on the metrics that actually matter — dwell time, false positive rate, analyst hours burned, and critical detection rate.
Methodology
Team A — Manual hunting: Two senior analysts with raw SIEM data and threat intelligence feeds only.
Team B — SIEM + Correlation Rules: Mature enterprise SIEM with 400+ tuned correlation rules. Two analysts of equivalent seniority.
Team C — LLM-Assisted (SentinelPRO IA v4.1): Two analysts using the AI Threat Hunting Workspace. LLM receives structured telemetry, generates hypotheses, and proposes investigation paths. Analysts validate and act.
All three environments received the same 48 synthetic threat injections over 6 months, ranging from commodity malware to simulated APT lateral movement.
Team B — SIEM + Correlation Rules: Mature enterprise SIEM with 400+ tuned correlation rules. Two analysts of equivalent seniority.
Team C — LLM-Assisted (SentinelPRO IA v4.1): Two analysts using the AI Threat Hunting Workspace. LLM receives structured telemetry, generates hypotheses, and proposes investigation paths. Analysts validate and act.
All three environments received the same 48 synthetic threat injections over 6 months, ranging from commodity malware to simulated APT lateral movement.
Results
DWELL TIME REDUCTION
-73%
vs. Team A
FALSE POSITIVE RATE
0.06%
vs. 2.1% Team B
DETECTION RATE
96%
vs. 71% Team B
Where LLMs Helped Most
Multi-stage attack correlation: The LLM surfaces connections between events hours or days apart that appear unrelated in isolation. Team C caught 100% of multi-stage attacks vs. 62% for Team B.
Living-off-the-land techniques: LOLBin attacks generate alerts that individually look like legitimate admin activity. The LLM identifies the sequence as malicious. Team A caught 44% of LOLBin scenarios; Team C caught 91%.
Hypothesis generation: Analysts could ask the workspace for the 5 most likely explanations for an unknown event cluster and receive structured hypotheses in seconds, vs. 20–40 minutes for Team A.
Living-off-the-land techniques: LOLBin attacks generate alerts that individually look like legitimate admin activity. The LLM identifies the sequence as malicious. Team A caught 44% of LOLBin scenarios; Team C caught 91%.
Hypothesis generation: Analysts could ask the workspace for the 5 most likely explanations for an unknown event cluster and receive structured hypotheses in seconds, vs. 20–40 minutes for Team A.
Limitations and What We Got Wrong
Confident hallucination risk: In 3 of 48 scenarios, the LLM generated plausible but incorrect hypotheses that sent analysts down wrong paths for 15–40 minutes. Analyst validation of every LLM output is non-negotiable. We have since added uncertainty scoring to the workspace interface.
Highly novel attacks: For truly unprecedented patterns with no precedent in training data, Team C advantage over Team B shrank to near zero.
Prompt injection risk: In one simulated scenario, we embedded a prompt injection in a malicious log file. Our implementation sanitizes LLM inputs, but this remains a real attack surface defenders must monitor.
Highly novel attacks: For truly unprecedented patterns with no precedent in training data, Team C advantage over Team B shrank to near zero.
Prompt injection risk: In one simulated scenario, we embedded a prompt injection in a malicious log file. Our implementation sanitizes LLM inputs, but this remains a real attack surface defenders must monitor.
// SENTINELPRO IA IN ACTION
The AI Threat Hunting Workspace is available in SentinelPRO IA Professional and Enterprise plans as of v4.1. Request a demo to see it running on your environment's actual telemetry.
See Plans & Pricing →Request Demo