Open Telemetry
NOTE
all traces, metrics and logs include labels for extra meta-data
Telemetry is fully supported to help debug what is going on
Metrics
NOTE
In preview
Metrics are there to help develop dashbaords, and you can use them to see what is going on inside laters
There are a few metrics collected inside the laters
library
metric | type | Use |
---|---|---|
laters.jobs.enqueue | counter | when a job is enqueued |
laters.jobs.process | counter | when a job is processed |
laters.jobs.process.errors | counter | when a job is processed with errors |
laters.jobs.ready | gauge | number of jobs that are ready to be processed |
laters.jobs.scheduled | gauge | number of jobs that are scheduled to be processed |
laters.jobs.deadletter | gauge | number of jobs that are deadlettered |
laters.process_time | histogram | time to process the message |
Code
Collect Metrics using the OpenTelemetry library, and export them via Prometheus
- 1️⃣ - Add OpenTelemetry to your services
- 2️⃣ - Add
laters
metrics - 3️⃣ - Add other metrics
var serviceName = "todoApp"; //name of your service
builder.Services.AddOpenTelemetry() // 1️⃣
.WithMetrics(b =>
{
b.ConfigureResource(r => r.AddService(serviceName))
.AddLatersInstrumentation() // 2️⃣
// 3️⃣
.AddProcessInstrumentation()
.AddAspNetCoreInstrumentation()
.AddPrometheusExporter();
})
Traces
As jobs are linked with other actions and other jobs, traces are supported to visualize this.
context
When a job is enqueued/created and when it is eventually processed, you will see the trace for both these actions
action | span.kind | Use |
---|---|---|
ForLater | producer | when a job is enqueued |
Handler | consumer | when a job is processed |
here is an example of the consumer (a job being processed)
Code
This example uses OpenTelemtry Tracing along with its associated exporter.
- 1️⃣ - Add OpenTelemetry to your services
- 2️⃣ - Add
laters
tracing - 3️⃣ - Add other tracing
var serviceName = "todoApp"; //name of your service
builder.Services.AddOpenTelemetry() // 1️⃣
.WithTracing(b =>
{
b.ConfigureResource(r => r.AddService(serviceName))
.AddLatersInstrumentation() // 2️⃣
// 3️⃣
.AddAspNetCoreInstrumentation()
.AddNpgsql()
.AddHttpClientInstrumentation()
.AddOtlpExporter(opts =>
{
opts.Endpoint = new Uri("http://otel-collector:4317");
});
});
Logs
For detailed output of information, laters
supports logging (with no Personal Information being captured by the core library)
NOTE
logs and traces share trace-id so you can correlate them.
Code
The example shows how to setup logging using the Serilog library, along with exporting the logs via OpenTelemetry
- 1️⃣ - Add Serilog
- 2️⃣ - Optionally export logs to the console (stdout)
var serviceName = "todoApp"; //name of your service
// 1️⃣
builder.Host.UseSerilog((context, config) =>
{
config
.Enrich.FromLogContext()
.Enrich.With(new PropertyEnricher("service_name", serviceName))
//.Filter.ByIncludingOnly(Matching.FromSource("Laters"))
.WriteTo.OpenTelemetry(opt =>
{
opt.Endpoint = "http://otel-collector:4317";
opt.IncludedData = IncludedData.SpanIdField |
IncludedData.TraceIdField |
IncludedData.TemplateBody;
})
.WriteTo.Console() // 2️⃣
.MinimumLevel.Information();
});