Observability Engineer

Cardinality Management

Vì sao: You detect a label explosion, relabel at ingestion and downsample long-term data, so one team adding a user id label does not take the metrics backend down.

Log Retention & Cost Optimization

Vì sao: You set tiered retention and sampling per log class and legal need, so the logging bill stops growing faster than traffic while audit data is kept.

Service Dependency Mapping

Vì sao: You generate the dependency graph from trace data and estimate blast radius, so before a risky migration you know which services call the component being changed.

Sampling Strategies

Vì sao: You choose head or tail sampling in the Collector so errors and slow traces are always kept, while routine traffic is cut to fit the storage budget.

SLO-Based Alerting

Vì sao: You standardize SLO and burn rate alerting across teams, so release and freeze decisions are made from the same error budget view everywhere.

Runbook Automation

Vì sao: You encode a recurring incident fix as an automated runbook triggered by an alert, so a known failure heals itself and time-to-remediation can be measured.

Observability Pipeline Design

Vì sao: You design Collector pipelines that enrich, route and fan out telemetry reliably, so switching or adding a backend does not require changing every application.

eBPF & Kernel Observability

Vì sao: You use eBPF tools such as Pixie or bpftrace to see network and syscall behavior without code changes, so you can debug a legacy service nobody can instrument.

Observability Culture & Adoption

Vì sao: You set golden signals and onboarding standards and measure adoption, so a team that ships uninstrumented code is noticed and helped before its first incident.

Cost Management for Observability

Vì sao: You forecast telemetry volume and build chargeback by team, so a surprise ingestion spike is traced to its owner and discussed with numbers.