Lakewatch: Databricks' Agentic SIEM Built on the Security Lakehouse
- Miguel Diaz
- Jun 17, 2026
- 07 Mins read
- Databricks
The problem with traditional SIEMs
Modern security teams face a growing tension: more data, more threats, limited budgets, and SIEMs that charge per byte ingested.
Volume-based licenses force teams to choose which logs to drop. Critical sources like EDR, firewall, and proxy are excluded due to cost — creating dangerous blind spots.
Data is trapped in proprietary formats. Switching providers means migrating petabytes of historical telemetry with no portability guarantees.
Hot storage costs force short retention windows. Retroactive investigations are limited to days or weeks — not the 365+ days required by most compliance frameworks.
Lakewatch solves all three simultaneously: storage in your own cloud at object storage economics, open formats that eliminate lock-in, and 365+ days of hot retention with full-text search in the Bronze layer.
What is Lakewatch?
Lakewatch is Databricks’ native agentic SIEM, built on the security lakehouse. It unifies security telemetry, IT logs, and business data on open storage (Delta/Iceberg), normalized to the OCSF (Open Cybersecurity Schema Framework) standard, with enterprise-grade governance and cost control on the Data Intelligence Platform.
Petabytes of telemetry in your own cloud object storage (S3, ADLS, GCS) in open formats (Delta/Iceberg, OCSF). You pay for storage, not ingestion.
AI-assisted detection rule creation, automatic parsing, query assistance, investigation workflows, natural-language threat hunting, and entity mapping and resolution.
All data flows through Unity Catalog. Permissions, lineage, audit trails, and cost control — integrated in the same platform that already governs the rest of the organization’s data.
Data stays in your cloud in open formats (Delta, Iceberg, OCSF). Switch tools without migrating data. Combine with any analytics or BI platform in your organization.
Medallion architecture for security
Lakewatch applies Databricks’ Medallion architecture to the security domain — three layers that transform raw log data into actionable intelligence:
Raw log with full-text search. 365+ day retention at object storage economics. Ingestion via Lakeflow Connect and Auto Loader.
Enriched data. AI-powered parsing, partial normalization, correlation with business context and identity data.
OCSF-aligned, analytics-ready for detections, queries, threat hunting, and SOC dashboards.
Data already in Unity Catalog can be used directly at any layer, or ingested from scratch via Lakeflow Connect and Auto Loader
Use cases
In this phase, Lakewatch complements the existing SIEM. The goal is not to replace it overnight but to extend its capabilities with long retention and advanced analytics.
Move high-volume, cost-prohibitive sources (EDR, firewall, proxy, cloud logs) off the legacy SIEM. Significantly reduce TCO with 365+ day hot retention.
Monitor audit logs and telemetry from your Databricks workspace directly with Lakewatch. Detect anomalies in data access, jobs, and model activity.
Ingest any source including unstructured data for complete context in threat hunting and investigations — without having to choose what to drop due to budget.
Genie enables natural-language threat hunting that democratizes access to the SOC. Agent Bricks runs investigation workflows and automatic alert triage without human intervention.
All data stays in open formats (Delta, Iceberg, OCSF) in your own cloud infrastructure. Portable, auditable, and combinable with any tool in the organization.
Lakewatch vs. traditional SIEMs and other solutions
- Ingest-based license — exponential costs
- Limited hot retention (30–90 days typical)
- Proprietary formats — total lock-in
- No native integration with business data
- Pay for compute/storage, not ingestion
- 365+ day hot retention in Bronze
- Delta/Iceberg/OCSF — open formats
- Business, security, and IT data unified
Lakewatch vs. Data Intelligence for Cybersecurity (DI4C): Lakewatch is a native, turnkey app ready to use out of the box. DI4C is a broader, DIY partner-oriented framework for building custom solutions on top of the lakehouse.