Case study 01
Cracker Breaker
LLMs that turn raw SIEM alerts into decisions non-specialists can act on
- Period
- September 2024 – December 2025
- Context
- University of Siena (DISPOC) with industrial partners Winet, Commit, Pragma, and Ambrogio
- Role
- Responsible for specifying and developing the AI module
- ~90%
- false positives filtered out
- <1 min
- real-time response on critical alerts
- 3
- role-differentiated views
Problem
Small and medium-sized enterprises are critical cybersecurity targets, but face barriers that enterprise tooling ignores:
- Security logs (SIEM) generate thousands of alerts in complex formats such as JSON, unreadable for non-specialist staff.
- A high false-positive rate produces alert fatigue: benign events flagged as threats teach people to ignore real danger.
- SMEs cannot afford costly cloud infrastructure or a dedicated Security Operations Center.
- Without specific training, employees remain the weakest link in the security chain — phishing, weak passwords.
Target user
Non-specialist stakeholders inside SMEs: IT managers, operators, and administrators who need to understand threats and respond to incidents without a SOC behind them. The system serves three distinct roles — Super Admin, IT Manager, and Operator — each with a differentiated view.
My role
I was responsible for specifying and developing the AI module:
- AI architecture design — a hybrid cloud/edge system balancing performance and cost.
- LLM research and development — selection, benchmarking, and implementation of models for log analysis and security training.
- SIEM integration — the communication pipeline between Wazuh (open-source SIEM) and the AI engines via REST APIs.
- Data science and evaluation — dataset curation, statistical analysis, and validation of model performance.
Process
The project followed an iterative, evidence-driven approach:
- Model selection — evaluation of cloud and lightweight LLMs to balance latency, cost, and explanation quality for each task in the product.
- Prompt engineering and XAI — chain-of-thought techniques to obtain interpretable explanations, aligned with the NIST framework and the NIS2 directive.
- Data pipeline — logs extracted from Wazuh, parsed to JSON, semantically analyzed by the LLM, then rendered as structured output plus short training capsules.
Key decisions
- Dual architecture: Google Cloud (Gemini Flash/Pro) for real-time analysis of critical alerts with response times under one minute; Lightning.ai (Meta Llama 4) for asynchronous work such as generating training plans from historical data and questionnaires.
- Open-source first: Wazuh as the central SIEM, eliminating license costs and maximizing integration flexibility.
- Cost optimization: small models (1B–8B parameters) for local deployment on low-power machines, making serious security affordable for SMEs.
Final result
- Behavioral pattern analysis identified and filtered up to 90% of false positives.
- The system generates natural-language explanations and operational recommendations — “update the firewall”, “block this IP” — directly usable by non-technical staff.
- It also produces remediation plans and 300-character training capsules to improve company security posture in line with the NIS2 directive.
What I learned
- Latency is a security requirement. We moved real-time analysis from Lightning.ai to Google Cloud after discovering that virtual-machine cold starts (six-minute timeouts) were incompatible with incident-response needs.
- Data quality beats data volume. In the SME context, the semantic richness of logs matters more than mass: a curated 30-day window proved sufficient for reliable analysis.
- AI is only as effective as its interface. Role-differentiated views for Super Admin, IT Manager, and Operator were fundamental to the system’s adoption.