Fix: self-hosted n8n failed executions are hard to track

    n8n's execution history shows individual runs, but it does not give you an instance-wide failure picture by itself. Add metrics first, then traces for executions that need node-level investigation.

    Last updated 2026-09-21~6 min read
    n8nPrometheusGrafanaOpenTelemetryTempoDocker

    Last updated: 2026-06-08 · Tested on: self-hosted n8n 1.x with Prometheus, Grafana, OpenTelemetry, and Tempo · ~7 min read

    Symptoms — what you're seeing

    • You run several self-hosted n8n workflows but have no single view of total, successful, and failed executions
    • You only discover failures by opening individual executions in the n8n editor
    • A workflow is slow or failing intermittently, but the execution list does not show the broader pattern
    • You know an execution failed, but you have to inspect every node manually to find where it stopped
    • You cannot compare workflow failure rates, execution duration, or latency over time

    Why this happens

    n8n's built-in execution history is useful for inspecting individual runs. It is not an observability system for the whole instance. Once several workflows are active, the important questions become aggregate questions: which workflows fail most often, whether latency is increasing, and whether a recent deployment affected every workflow.

    Metrics and traces answer different questions:

    • Metrics answer “what is happening overall?” Prometheus can store execution counts, failure counts, duration, and other time-series measurements. Grafana turns those measurements into dashboards.
    • Traces answer “what happened inside this execution?” OpenTelemetry provides the telemetry model and export path. Tempo stores and queries the traces, while Grafana displays them so you can inspect the execution path and its spans.

    The fix is to build the stack in that order: establish reliable aggregate metrics first, then add tracing for the executions that need node-level diagnosis. Do not start with a dashboard full of panels before confirming that the source data is complete.

    There is one important version check. n8n's metrics and OpenTelemetry capabilities vary by release and deployment mode, so confirm the exact metrics endpoint, environment variable, and tracing support for the n8n version you are running before wiring up the rest of the stack. If a metric or span is not emitted by n8n, Grafana cannot recover it later.

    The fix, step by step

    1. Define the signals before deploying dashboards. Start with total executions, successful executions, failed executions, success rate, execution duration, and workflow-level execution counts. For a production instance, also collect host, process, database, queue, and worker health metrics where those components exist.
    2. Enable and verify n8n metrics. Configure the metrics option supported by your n8n release, expose the metrics endpoint only to your monitoring network, and scrape it with Prometheus — confirm the supported metrics configuration and endpoint path for your deployed version before applying it. Then open the endpoint directly and confirm that it returns current samples rather than an empty response or an authentication error.
      # Prometheus configuration pattern — adapt the target to your network
      scrape_configs:
        - job_name: n8n
          scrape_interval: 15s
          metrics_path: /metrics
          static_configs:
            - targets: ["n8n:5678"]
      The target name, port, path, and authentication settings are deployment-specific — replace the target with the address reachable from your Prometheus container and confirm that the endpoint is actually enabled. A green target in Prometheus is not enough if the returned series do not change while test executions run.
    3. Build the first Grafana dashboard around failure triage. Add panels for active workflows, total executions, successful executions, failed executions, success rate, execution activity over time, and execution duration. Add a workflow table only when the underlying metrics include a stable workflow label. If the label is missing or inconsistent, show instance-level values instead of presenting a misleading workflow ranking.
    4. Validate every panel with controlled executions. Run one known-successful execution, one deliberate test failure, and one slow execution in a non-production workflow. Confirm that the expected counters increase, the failure appears in the failure series, and the duration changes. Validate the complete result set from Prometheus or Grafana; do not test only the first returned item and assume the rest of the workflow data is correct.

    Stuck on the observability setup halfway through? Book a 30-min call and I'll inspect the metrics, dashboard queries, or trace path with you. Email Basit if you prefer to send the deployment details first.

    1. Add OpenTelemetry only after the metrics path works. Configure an OpenTelemetry Collector as the boundary between n8n or your instrumentation and the trace backend. Export traces to Tempo, then connect Grafana to Tempo as a data source. OpenTelemetry is the telemetry protocol and SDK layer; Tempo is the trace storage and query layer. They do not replace each other.
      # Collector pipeline shape — configure receivers and exporters for your deployment
      receivers:
        otlp:
          protocols:
            grpc:
            http:
      
      service:
        pipelines:
          traces:
            receivers: [otlp]
            processors: [batch]
            exporters: [otlp/tempo]
      Add the actual Tempo exporter endpoint, TLS settings, and authentication required by your Collector deployment before running this. Keep the Collector on the internal network and send only the telemetry fields you need.
    2. Confirm the trace boundary and span coverage. Open one known execution in Grafana and follow its trace ID. Confirm that the trace maps to the correct workflow and execution, that spans have usable names, and that failures carry error status or exception information. Do not claim node-level visibility until you can see node-level spans in a real test trace. If your n8n version does not emit those spans natively, instrument the execution boundary or add supported custom instrumentation instead of treating a workflow-level span as a node trace.
    3. Link the dashboard to the investigation path. Use a failure or latency panel to identify the affected workflow and time window. Open the corresponding n8n execution, then use the trace to locate the slow or failed span. This is the useful path: aggregate signal in Prometheus, workflow context in Grafana, and execution detail in Tempo.

    Common variations

    • “I only need to know when n8n is down.” Add an instance health check and alert before building workflow-level dashboards. Uptime monitoring and execution observability solve different problems.
    • “Prometheus shows failures but not the failing node.” That is expected when you have metrics without node-level tracing. Metrics identify the pattern; a correctly instrumented trace identifies the execution path.
    • “Grafana is showing a workflow table with unreliable values.” Check the metric labels and aggregation. A workflow name that changes, is missing, or is not attached consistently will produce misleading groupings.
    • “Tracing creates too much data.” Sample traces deliberately, keep full traces for errors where supported, and avoid putting contact details, credentials, request bodies, or other sensitive payloads into span attributes.
    • “I want to compare latency across workflows.” Use p50, p90, and p99 only after confirming that the duration series has enough samples and consistent units. An average alone hides timeout and tail-latency problems.

    If a specific workflow is failing because an upstream API is returning 429 responses, use the n8n OpenAI 429 rate-limit diagnostic alongside the observability dashboard. For trigger timing problems, compare the execution timestamps against the n8n schedule trigger timezone checks.

    When this isn't the issue

    If the dashboard is empty or the values do not match the n8n execution list, work through these forks before changing the dashboard:

    • Prometheus target is down or unauthorized. Check the target status, network route, container port, reverse proxy, and metrics authentication. Query the endpoint from the Prometheus container, not only from your laptop.
    • The metric exists but does not change. Run a controlled execution and inspect the raw series. The scrape interval, metric name, label set, or n8n version may not match the dashboard query — confirm the metric names and labels against the actual output from your deployment.
    • The trace is missing or has no node spans. Check the OpenTelemetry export logs, Collector receiver, Tempo endpoint, trace sampling, and correlation data. A Grafana data-source connection does not prove that n8n is producing the spans you expect.
    • The workflow itself has a node-level error. Observability will help locate it, but it does not fix invalid credentials, a rate-limited API, an incorrect timezone, or a broken memory configuration. For the latter, see the n8n Simple Memory node error diagnostic.

    Stuck on this for more than 30 minutes? Book a 30-min call. I'll get on Zoom, look at your self-hosted n8n telemetry, and either fix the monitoring path on the call or scope what it'll take. No upsell, no decks.

    Email Basit

    Related pages

    Need this workflow customized or debugged? Book a 30-min scope call →

    Stuck configuring this workflow?

    I build and fix n8n workflows for agencies and SaaS teams. 30-minute scope call — bring the problem, leave with an estimate.

    Related pages