Make more out of your data by enriching it with metadata. But there are a couple of tradeoffs:
- When: Index- vs runtime — while Elasticsearch has historically always favored index-time, you can do both now. Which has an impact on performance and sometimes also correctness.
- Where: Edge vs central vs in-cluster — having an impact both on performance and operational aspects.
- How: Logstash vs Beats vs Agent (with Fleet) vs OpenTelemetry Collector vs Elasticsearch ingest pipeline vs Elasticsearch runtime fields — you are spoilt for choice.
AI / ML, “de-richment” for personal data, and more complex use-cases all also have their space in this talk.