Large Language Models for Cybersecurity: A Careful Starting Point

Large language models are becoming useful in cybersecurity, but their value is easy to misstate. They can summarize evidence, map incident language to a shared vocabulary, draft queries for an analyst and expose connections across long reports. They do not automatically know whether a security claim is true, current or safe to act on.

The most useful starting point is to assign the model a bounded role in a workflow whose evidence and authority remain visible.

1. Where an LLM can help

An LLM can support several parts of a security operation:

• translating alerts into an analyst-readable incident summary,
• extracting entities, indicators and relationships from reports,
• proposing search queries for threat-intelligence repositories,
• comparing an alert with approved playbooks,
• asking for missing context before a response is proposed,
• drafting a post-incident report with links to its evidence.

These roles are assistive. They are strongest when the output is a structured intermediate artifact rather than an opaque final decision.

2. Why cybersecurity is a difficult domain

Security data is heterogeneous, time-sensitive and adversarial. A hostname can be benign in one environment and suspicious in another. An indicator may be obsolete. A report may describe a possibility rather than an observed fact. Prompt injection can also arrive through the very documents the model is asked to summarize.

For that reason, the context sent to the model should preserve provenance:

$$e_i = (x_i, s_i, t_i, c_i, q_i),$$If the formula does not fit, focus it and use the left and right arrows, or scroll horizontally.

where $x_i$ is the observation, $s_i$ its source, $t_i$ its time window, $c_i$ its local context and $q_i$ a quality or confidence signal. A prose answer without this tuple is difficult to audit.

3. A safer interaction contract

An operational prompt should define what the model may do and what it must not do. It should require:

1. a separation between observed facts and hypotheses;
2. citations or identifiers for every material claim;
3. an explicit uncertainty statement when evidence is incomplete;
4. no execution of commands or policy changes without an external approval step;
5. a machine-readable record of the request, context and response.

This contract is more important than a clever prompt. It makes the model one component of a controlled system instead of an unreviewable operator.

4. Evaluation beyond fluent text

Fluency is not a security metric. Evaluation should test whether the model preserves indicators, avoids inventing attack stages, distinguishes a recommendation from an action and remains useful when the evidence is contradictory.

For a response $y$ and evidence set $E$, a simple evaluation record can combine task utility and evidence discipline:

$$J(y,E) = \lambda_u U(y) + \lambda_g G(y,E) - \lambda_h H(y) - \lambda_a A(y),$$If the formula does not fit, focus it and use the left and right arrows, or scroll horizontally.

where $U$ measures usefulness, $G$ grounding, $H$ unsupported claims and $A$ unsafe authority. The terms should be measured separately, not hidden behind a single score.

5. A practical first deployment

Start with a read-only assistant for a narrow queue. Store the input evidence, retrieved documents, model version, response and analyst correction. Review failures weekly and expand the role only when the correction pattern is understood.

The goal is not to make the LLM sound like an expert. It is to make the analyst's reasoning faster while keeping the evidence, uncertainty and decision authority inspectable.

Choose a SOC task with an acceptance criterion

For an initial deployment, I propose a verifiable task: turn an alert and its associated events into a summary containing the asset, timeline, evidence and unknowns. An acceptable answer preserves identifiers, distinguishes attempted access from confirmed access and connects each conclusion to an event. A persuasive summary inventing privilege escalation must fail.

Compare the assistant with a template that uses no LLM. Use the same incidents and measure review time, omissions and corrections. Alternate presentation order to reduce analyst familiarity effects. This comparison reveals whether the model adds value over simpler automation.

What changes when tools are added

A summarizer and an agent with query or action access have different failure surfaces. The latter can choose incorrect parameters, recipients or scope even when its explanation sounds correct. Log generated text, proposed calls, authorized calls and observed effects separately. Permissions must be enforced by the service executing the tool.

The systematic review by Xu and colleagues places agents among emerging directions in LLM-based security. AgentDojo offers an environment for evaluating tasks and prompt injection. Neither replaces testing with your own data, tools and authorization rules.

When to expand the scope

Expand when remaining errors are understood, regressions can be detected and a return to the previous workflow is available. Define these criteria before the pilot. Analyst approval does not automatically make a response a correct retraining label.

Continue with RAG for cybersecurity, evaluation golden sets and prompt-injection testing.