Migrating from OpenTracing.NET to OpenTelemetry.NET

Background

OpenTracing is an interesting project that allows for the collection of various trace sources to be correlated together to provide a timeline of activity that can span services for reporting in a central system. They were competing with OpenCenus but have now merged to form OpenTelemetry.

I was recently brought in as a consultant to help migrate an existing system that used OpenTracing in .NET that recorded trace data into Jaeger so that they might migrate to the latest OpenTelemetry libraries. I thought it would be useful to document what I learnt as the migration process is not particularly clear.

The first thing to note is that OpenTracing and OpenTelemetry are both multi-platform systems and so support many languages and services. While this is great from a standardization process it does mean that a lot of information you find doesn't necessarily relate to the library you are using.

In this particular case we are using the .NET/C# library so moving from OpenTracing API for .NET to OpenTelemetry .NET and while there are plenty of signs that OpenTracing is deprecated and that all efforts are now in OpenTelemetry it's worth noting that as of time of writing OpenTelemetry .NET isn't quite done - many of the non-core libraries are in beta or release-candidate status.

How OpenTracing is used

The OpenTracing standard basically worked through the ITracer interface. You can register it as a singleton via DI and let it from through where it is needed or access it via the GlobalTracer static.

This ITracer has a IScopeManager with am IScope being basically an active thread and the ISpan being a unit of work which can move between threads.

Typically usage looks a little something like this:

class Runner {
  private ITracer tracer;

  public Runner(ITracer tracer) {
    this.tracer = tracer;
  }

  public void Run(string command) {
      using (var scope = tracer.BuildSpan("Run").StartActive()) {
        // ...
      }
  }
}

With scopes capable of having sub-scopes, tagging and events within them to provide further detail. The moment the StartActive is called the clock starts and the moment the using goes out of scope the clock ends providing you the detail levels you need.

How OpenTelemetry changes things

Now OpenTelemetry changes things a little breaking Tracing, Metrics and Logging into separate things.

While OpenTelemetry uses much of the same terminology as OpenTracing when it comes to the .NET client they decided to take a different approach and rather than implement Span again they use .NET's built in ActivitySource as the replacement so there's less to learn.

So we create an ActivitySource in the class and then use it much as we would have used ITracer.

class Runner {
  static readonly AssemblyName assemblyName = typeof(Runner).Assembly.GetName();
  static readonly ActivitySource activitySource = new ActivitySource(AssemblyName.Name, AssemblyName.Version.ToString());

  public void Run(string command) {
      using (var activity = activitySource.BuildActivity("Run")) {
        // ...
      }
  }
}

This has the advantage of not needing to pass around iTracer and ActivitySource is optimized to immediately return nulls if no tracing is enabled (which the using keyword handles just fine).

If, however, you are using multiple systems and want to stick the the OpenTelemetry terminology of Tracer and Span instead of ActivitySource and Activity then check out the OpenTelemetry.API shim (not to be confused with the OpenTracing shim covered below).

There is a full comparison available of how OpenTelemetry maps to the .NET Activity API available too.

WARNING using and C# 8+

Some code analysis/refactoring tools may recommend changing the using clause from the traditional using (var x...) to the brace-less using var x... of C# 8.

Do not do this with BuildSpan or BuildActivity unless you have only one of them and you're happy for the timing to consider the end when the method exits.

This is because removing the braces removes the defined end of the span or activity. This means that they will continue on beyond their original intended scope and carry on their timings until the end of the method.

How to migrate

It is possible to migrate in steps by switching your application over from OpenTelemetry to OpenTracing while still temporarily supporting ITracer until you have moved although the code to set this up correctly is easy to get wrong (and if you do you will see duplicate spans in your viewer).

In ASP.NET Core you want to register your AddOpenTelemetryTracing with all your necessary instrumentation. For example:

services.AddOpenTelemetryTracing(builder => {
    builder
        .SetResourceBuilder(ResourceBuilder.CreateDefault().AddService("MyServiceName"))
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddSqlClientInstrumentation(o => {
            o.SetDbStatementForText = true;
            o.RecordException = true;
        })
        .AddJaegerExporter(o => {
            o.AgentHost = openTracingConfiguration?.Host ?? "localhost";
            o.AgentPort = openTracingConfiguration?.Port ?? 6831;
        });
});

Backwards compatible support for ITracer & GlobalTracer

If you still need to support ITracer/GlobalTracer in the mean time you can use OpenTelemetry.Shims.OpenTracing to forward old requests on. Simply add this block of code to the end of the AddOpenTelemetryTracing covered above:

    services.AddSingleton<ITracer>(serviceProvider => {
        var traceProvider = serviceProvider.GetRequiredService<TracerProvider>();
        var tracer = new TracerShim(traceProvider.GetTracer(applicationName), Propagators.DefaultTextMapPropagator);
        GlobalTracer.RegisterIfAbsent(tracer);
        return tracer;
    });

Note that other combinations of trying to get this working for us resulted in duplicate spans. It is quite possible that duplicate trace providers can cause this.

Jeager

With OpenTracing Jaeger required the use of it's own C# Client for Jaeger however this library is now being deprecated in favor of OpenTelemetry.Exporter.Jaeger.

There are some limitations with this - specifically right now the Jaeger support is for tracing only and uses the UDP collector only which means no authentication and limited message sizes.

Jaeger looked at directly supporting the OpenTelemetry collector system but that experiment has been discontinued and is now planned for Jaeger 2.

In the mean time if these limitations are a problem it's possible the ZipKin exporter might be a better choice given that Jaeger also supports that.

Logs on a Span

OpenTracing helpfully correlated logs from ILogger to the appropriate span so they could appear alongside what they related to in, for example, Jaeger.

OpenTelemetry wants to keep logs separate and instead relies on .NET's ActivityEvent for this purpose but if you already have lots of ILogger usage instead then the OpenTelemetry.Preview NuGet package has your back and will re-dispatch ILogger entries as ActivityEvent ensuring they end up in the right place as they did with OpenTracing.

services.AddLogging(o => {
  o.AddOpenTelemetry(t => {
    c.AttachLogsToActivityEvent();
  })
})

Dealing with pre-release NuGet packages

At time of writing a number of the OpenTelemetry libraries are pre-release. If you are using .NET Analyzers they will prevent a successful build because of this. If you are absolutely sure you want to proceed then add NU5104 to the <NoWarn> section of your .csproj, e.g.

<NoWarn>NU5104</NoWarn>

Additional tracing

OpenTracing has a popular Contrib package that provides support for a variety of sources. Much of this is replaced by additional OpenTelemetry .NET packages, specifically the following packages which will need to be added to your assembly and the necessary .

There are many additional services that were not natively or contrib-supported as well including Elasticsearch, AWS, MassTransit, MySqlData, Wcf, Grpc, StackExchangeRedis etc.

Hope that helps!

[)amien