Splunk Interview Q&A
Q: What's the actual difference between tstats and stats, and when does it matter?
stats retrieves and decompresses raw events before aggregating — expensive at scale. tstats reads directly from Splunk's tsidx index structures, skipping raw-event retrieval entirely, but only works on indexed fields (not fields extracted later via rex/eval). At real data volume the difference is often an order of magnitude. Use tstats (optionally against an accelerated data model) for any frequently-run search or dashboard where the fields involved are already indexed.
Q: Explain replication factor vs. search factor in an indexer cluster.
Replication factor (RF) is the total number of copies of each data bucket kept across the cluster — a durability guarantee. Search factor (SF) is how many of those copies are kept immediately searchable rather than just stored — an availability/performance guarantee, always ≤ RF. A common misconfiguration is setting RF high for durability while leaving SF too low — the data survives a node failure, but search performance/availability during that failure doesn't, because the "extra" copies aren't query-ready.
Q: Why does a correlation search sometimes never fire even when the condition was clearly met?
Two common causes. First, the search itself may have been skipped by the scheduler — if the previous run was still in progress or the search head was under load, Splunk deprioritizes rather than queues it, and a skipped run produces silently no output. Second, a scheduling-window gap — a search checking earliest=-15m on a 15-minute schedule assumes the prior run completed on time; if it didn't, there's a real gap between what consecutive runs actually covered, and an event landing in that gap is never evaluated by any run.
Q: What is CIM (Common Information Model) and why does it matter for Splunk Enterprise Security?
CIM is a standardized field-naming schema — normalizing differently-sourced data (Windows auth logs, Linux auth logs, cloud IAM events) to consistent field names (like user, src, action) so ES's built-in correlation searches and dashboards work across all of them without a custom mapping per source. Without CIM compliance, onboarding a new log source to ES means either writing custom correlation searches for it or manually normalizing its fields — CIM is what lets ES's out-of-the-box content actually apply broadly.
Q: How does Risk-Based Alerting change the alerting model compared to a traditional one-search-one-alert setup?
Traditional correlation searches decide independently whether to fire, which produces alert fatigue from many individually low-confidence signals. RBA instead has individual events contribute risk scores to specific entities (a user, host, or IP), accumulating across multiple searches and over time — a single failed login isn't independently alertable, but that user's risk score crossing a threshold because of five different low-signal events across a day is. This lets genuinely correlated but individually-weak signals actually drive detection instead of requiring one search to somehow capture the whole pattern.
Q: A forwarder shows as running and connected, but one specific monitored file's data never appears. What do you check?
Two independent things, since either failing alone produces this exact symptom: file permissions (the forwarder process's user needs read access — a file created with different ownership than other logs on the same host is a common cause) and the inputs.conf monitor stanza's path pattern actually matching the file's real location, including any log-rotation naming pattern the app uses. Checking sudo -u and the actual configured stanza separately is faster than guessing which one is wrong.
Q: What happens when a Splunk deployment exceeds its license limit?
Historically, exceeding an ingest-based license triggers a violation warning, and accumulating enough violations within a rolling enforcement window can trigger search restriction — Splunk keeps indexing (no data loss) but blocks the ability to search until the violation clears (needs verification — recheck against current source, since license enforcement mechanics have varied across versions/license types). The specific risk worth naming: this can mean a fully-ingesting deployment that a SOC analyst can't query during exactly the high-volume event (a real incident) when they need it most, which is why proactively monitoring license usage via the internal license-usage index matters operationally, not just for cost control.
Q: Why does Splunk remain dominant in security operations specifically, when cheaper alternatives like the ELK stack exist?
Security teams typically can't dictate log format to every vendor appliance, legacy system, and application already deployed — Splunk's core design (index heterogeneous data as-is, extract and correlate at search time via SPL rather than requiring a common schema up front) fits that constraint directly, and Splunk Enterprise Security adds purpose-built SIEM tooling (correlation searches, CIM, RBA, Notable Events) on top. ELK is often meaningfully cheaper and a strong choice where an organization has more control over log formats or is optimizing for full-text search — the tradeoff is real, not a case of one tool being unconditionally better.
Q: What's the difference between summary indexing and data model acceleration?
Both exist to make expensive searches fast to re-run, but solve slightly different problems. Summary indexing pre-computes and stores the result of one specific, fixed aggregation on a schedule (via collect) — later queries read the small summary index instead of re-running the expensive search. Data model acceleration maintains a tsidx-backed accelerated structure behind an entire data model automatically, which any tstats query against that model can benefit from — a general-purpose accelerated structure multiple different queries can reuse, versus one specific pre-computed result.
Q: Walk through the SPL for a basic security investigation starting from one suspicious IP.
Start broad but time-bounded to confirm which indexes show any activity at all: index=* ". Narrow to the relevant sources and build a normalized, time-sorted timeline using eval/case to reconcile each source's differently-named fields into consistent columns. If reconstructing a session (not just isolated events), transaction groups related events spanning multiple log lines — like a login session — into one logical unit. This progressive narrowing (confirm → normalize → group) is the concrete shape of "Splunk correlates across formats," not an abstract claim.