In this series I've been focusing on the new minimal hosting APIs built using WebApplication and WebApplicationBuilder. These provide a simpler model for building web apps, while maintaining the same overall functionality as the generic-Host-based applications of .NET Core 3.x/5.

However, this simplification has challenges. The more complex startup code in earlier versions, typically split between Program.cs and Startup, had advantages, in that it provided well-known hooks that tools could use to hijack the application start process.

A classic example of this is the EF Core tooling. If you've ever used EF Core, you may be familiar with the problems that arise if you try and change your startup code. So when the framework changes its default startup code you know it's going to cause issues!

EF Core tools in ASP.NET Core 3.x/5

EF Core includes various tools for generating migrations and running them against your database, but to do this, it needs to understand your code. Essentially, it needs to be able to run of your application's startup code, so that all the configuration and dependency injection services you've configured are used.

In previous versions of ASP.NET Core, EF Core hooked into the CreateWebHostBuilder or CreateHostBuilder method in your Program class. The EF Core tools would look for this "magic" method to get access to the IWebHostBuilder or IHostBuilder used to build your application.

public static IHostBuilder CreateHostBuilder(string[] args) =>
    Host.CreateDefaultBuilder(args)
        .ConfigureWebHostDefaults(webBuilder =>
        {
            webBuilder.UseStartup<Startup>();
        });

This always felt very hacky; if you renamed the method, or changed its signature, then the EF Core tools would break.

By using this well-known method, the EF Core tools could load your application's assembly using reflection, execute the method, grab the returned IHostBuilder, call Build on it to create an IHost, and then grab the IServiceProvider from the IHost.Services property! And from this provider, EF Core can introspect pretty much anything about your application. Pretty nifty, but very dependent on specific conventions.

So what happens when ASP.NET Core throws out all these conventions in .NET 6 with the minimal hosting APIs? Well, the EF Core tools broke😄

Supporting the EF Core tools in .NET 6

Of course, the issue was fixed, and this post looks briefly at how David Fowler achieved that. The tl;dr; is that he added new diagnostic source events during app startup!

If you're not familiar with DiagnosticSource and DiagnosticListener, I have an introductory post about it here, which describes where they fit in to the various logging primitives in .NET Core.

That post is over 4 years old now (it even contains references to project.json!) but it still serves as a good reference to how DiagnosticSource works.

DiagnosticSource is intended primarily as a high performance way of logging rich data, similar to Event Tracing for Windows (ETW), but entirely in-process. By emitting DiagnosticSource events as part of the host building process, David provided a way for the EF Core tools to get access to the IHostBuilder and IHost objects. In the next section we'll look at the new events, and in the subsequent section we'll see how the EF Core tooling uses them.

Emitting new events as part of the host building process

In the past two posts in this series, I showed that the new WebApplicationBuilder and WebApplication are just wrappers around the generic host introduced in .NET Core 3.0. The additional DiagnosticSource events were added to the generic HostBuilder, so they apply both to the minimal hosting WebApplication and to the "traditional" generic host constructs.

That means that if you continue to use the Program.cs and Startup split in .NET 6, and you rename the CreateHostBuilder() method, the EF Core tools should no longer break!

The following code shows the HostBuilder.Build() method (as of .NET 6 RC1, somewhat adpated for readability). The changes made are relatively simple:

• A new DiagnosticListener is created for the duration of the Build() method.
• If anything is listening for the Microsoft.Extensions.Hosting.HostBuilding event, the HostBuilder passes itself as a parameter to the listener.
• After the host has been built, and before the method returns, the HostBuilder checks if anything is listening for the Microsoft.Extensions.Hosting.HostBuilt event. If it is, the newly built host is passed as a parameter to the listener.

public class HostBuilder : IHostBuilder
{
    public IHost Build()
    {
        using var diagnosticListener = new DiagnosticListener("Microsoft.Extensions.Hosting");
        const string hostBuildingEventName = "HostBuilding";
        const string hostBuiltEventName = "HostBuilt";

        if (diagnosticListener.IsEnabled() && diagnosticListener.IsEnabled(hostBuildingEventName))
        {
            diagnosticListener.Write(hostBuildingEventName, this);
        }

        // Normal build process
        BuildHostConfiguration();
        CreateHostingEnvironment();
        CreateHostBuilderContext();
        BuildAppConfiguration();
        CreateServiceProvider();

        var host = _appServices.GetRequiredService<IHost>();
        if (diagnosticListener.IsEnabled() && diagnosticListener.IsEnabled(hostBuiltEventName))
        {
            diagnosticListener.Write(hostBuiltEventName, host);
        }

        return host;
    }
}

Adding these DiagnosticSource events won't have much utility on user applications, as in your own application you obviously already have access the HostBuilder and the built IHost. Where things come in handy is when you need to run the app using reflection, as in EF Core's tools.

Updates to the HostFactoryResolver

The EF Core tools have a tricky job. They need to be able to use your application's configuration—including connection strings, DI service configuration, and DB provider configuration—without actually running your application. To achieve that, they use a helper class called HostFactoryResolver.

HostFactoryResolver lives in a source package (i.e. the source code is copied around and included in the consuming projects), and provides helpers for finding one of the well known entry points for ASP.NET Core apps in an assembly.

Prior to .NET 6, HostFactoryResolver would look for an entrypoint called one of the following:

• BuildWebHost
• CreateWebHostBuilder
• CreateHostBuilder

If it found a method with the required name on the assembly entrypoint class (i.e. Program), that returns the correct type, and takes a string[] parameter, then it would invoke this method using reflection to obtain the IHostBuilder (or similar) instance. This is how the EF Core tools were able to use your app's configuration.

In .NET 6, a new resolution mechanism was added that uses the new DiagnosticSource events. The code below shows the ResolveHostFactory method, including all the comments, as they're quite enlightening:

// This helpers encapsulates all of the complex logic required to:
// 1. Execute the entry point of the specified assembly in a different thread.
// 2. Wait for the diagnostic source events to fire
// 3. Give the caller a chance to execute logic to mutate the IHostBuilder
// 4. Resolve the instance of the applications's IHost
// 5. Allow the caller to determine if the entry point has completed
public static Func<string[], object>? ResolveHostFactory(
    Assembly assembly, TimeSpan? waitTimeout = null, bool stopApplication = true,
    Action<object>? configureHostBuilder = null, Action<Exception?>? entrypointCompleted = null)
{
    if (assembly.EntryPoint is null)
    {
        return null;
    }

    try
    {
        // Attempt to load hosting and check the version to make sure the events
        // even have a chance of firing (they were added in .NET >= 6)
        var hostingAssembly = Assembly.Load("Microsoft.Extensions.Hosting");
        if (hostingAssembly.GetName().Version is Version version && version.Major < 6)
        {
            return null;
        }

        // We're using a version >= 6 so the events can fire. If they don't fire
        // then it's because the application isn't using the hosting APIs
    }
    catch
    {
        // There was an error loading the extensions assembly, return null.
        return null;
    }

    return args => new HostingListener(
        args, 
        assembly.EntryPoint, 
        waitTimeout ?? s_defaultWaitTimeout, 
        stopApplication, 
        configureHostBuilder, 
        entrypointCompleted).CreateHost();
}

This method does some initial checks to make sure that we've referenced the .NET 6+ version of the Microsoft.Extensions.Hosting assembly, and if we have, creates an instance of HostingListener, which is a nested class in HostFactoryResolver where most of the work happens.

You can see the full source code for HostingListener on GitHub, but as it's 150 lines, I'll walk through the important parts below.

The bulk of the work is in the CreateHost() method, shown below. This code is responsible for:

• Creating the DiagnosticListener subscription, by registering the HostingListener itself to receive callbacks. We'll take a look at this shortly.
• Creating a new thread and starting the entrypoint. This starts your application running in a background thread, and assumes that at somepoint your app will call HostBuilder.Build()
• Waiting for one of 3 three things to happen:
  • Our diagnostic listener code throws a StopTheHostException after completing all its setup (we'll come to this one shortly)
  • Some other exception is thrown in your application
  • Application startup takes too long, and so CreateHost times out.
• Once one of the above has happened, the result is propagated to the caller, either an object is returned (which will be the IHost created by your app), or an exception will be thrown:

private readonly TaskCompletionSource<object> _hostTcs = new();
private readonly MethodInfo _entryPoint;
private static readonly AsyncLocal<HostingListener> _currentListener = new();
private readonly Action<Exception?>? _entrypointCompleted;

public object CreateHost()
{
    using var subscription = DiagnosticListener.AllListeners.Subscribe(this);

    // Kick off the entry point on a new thread so we don't block the current one
    // in case we need to timeout the execution
    var thread = new Thread(() =>
    {
        Exception? exception = null;

        try
        {
            // Set the async local to the instance of the HostingListener so we can filter events that
            // aren't scoped to this execution of the entry point.
            _currentListener.Value = this;

            var parameters = _entryPoint.GetParameters();
            if (parameters.Length == 0)
            {
                _entryPoint.Invoke(null, Array.Empty<object>());
            }
            else
            {
                _entryPoint.Invoke(null, new object[] { _args });
            }

            // Try to set an exception if the entry point returns gracefully, this will force
            // build to throw
            _hostTcs.TrySetException(new InvalidOperationException("Unable to build IHost"));
        }
        catch (TargetInvocationException tie) when (tie.InnerException is StopTheHostException)
        {
            // The host was stopped by our own logic
        }
        catch (TargetInvocationException tie)
        {
            exception = tie.InnerException ?? tie;

            // Another exception happened, propagate that to the caller
            _hostTcs.TrySetException(exception);
        }
        catch (Exception ex)
        {
            exception = ex;

            // Another exception happened, propagate that to the caller
            _hostTcs.TrySetException(ex);
        }
        finally
        {
            // Signal that the entry point is completed
            _entrypointCompleted?.Invoke(exception);
        }
    })
    {
        // Make sure this doesn't hang the process
        IsBackground = true
    };

    // Start the thread
    thread.Start();

    try
    {
        // Wait before throwing an exception
        if (!_hostTcs.Task.Wait(_waitTimeout))
        {
            throw new InvalidOperationException("Unable to build IHost");
        }
    }
    catch (AggregateException) when (_hostTcs.Task.IsCompleted)
    {
        // Lets this propagate out of the call to GetAwaiter().GetResult()
    }

    return _hostTcs.Task.GetAwaiter().GetResult();
}

With this code alone, it's not clear how the EF Core tools are able to get hooks into the IHostBuilder build process, or how they access the IHost. The answer lies in the DiagnosticSource events that the HostingListener receives thanks to calling DiagnosticListener.AllListeners.Subscribe(this) in the previous code.

private sealed class HostingListener : IObserver<DiagnosticListener>, IObserver<KeyValuePair<string, object?>>
{
    private static readonly AsyncLocal<HostingListener> _currentListener = new();

    public void OnNext(DiagnosticListener value)
    {
        if (_currentListener.Value != this)
        {
            // Ignore events that aren't for this listener
            return;
        }

        if (value.Name == "Microsoft.Extensions.Hosting")
        {
            _disposable = value.Subscribe(this);
        }
    }
    
    public void OnCompleted()
    {
        _disposable?.Dispose();
    }
    // ...
}

When a new DiagnosticListener is initialized, the hosting listener checks that

1. We're in the same async local context (to avoid concurrency issues), and if not, ignores the listener. We won't receive any events from this listener.
2. Checks that the listener is called "Microsoft.Extensions.Hosting", so that we only subscribe to events from HostingBuilder.

After subscribing to the hosting events listener, we will receive events when HostBuilder calls diagnosticListener.Write(), as we saw earlier in the post. The OnNext() method is called with a KeyValuePair, containing the name of the event, and the object written. This is how the HostingListener can access the IHostBuilder and the IHost! the IHostBuilder is customised prior to finishing the build, and the final IHost object is passed back to the EF Core tools by setting the result on the TaskCompletionSource

private readonly bool _stopApplication;
private readonly TaskCompletionSource<object> _hostTcs = new();
private readonly Action<object>? _configure;

public void OnNext(KeyValuePair<string, object?> value)
{
    if (_currentListener.Value != this)
    {
        // Ignore events that aren't for this listener
        return;
    }

    if (value.Key == "HostBuilding")
    {
        // run the action to customise the IHostBuilder passed in value.Value
        _configure?.Invoke(value.Value!);
    }

    if (value.Key == "HostBuilt")
    {
        // store the IHost passed in value.Value
        _hostTcs.TrySetResult(value.Value!);

        // The EF Core tools don't actually want to _run_ the application
        if (_stopApplication)
        {
            // Stop the host from running further
            throw new StopTheHostException();
        }
    }
}

private sealed class StopTheHostException : Exception { }

To prevent the application running properly (and listening on a port for example), the listener throws a StopTheHostException. As the listener runs in-line in the application, this effectively bails out of the running app. The HostingListener catches this exception, holds on to the IServiceProvider, and continues as in previous versions of the framework.

With these changes, the EF Core tools are once again able to retrieve an instance of your application's IServiceProvider, by basically running your Program.Main and bailing out once the IHost has been built.

It feels like it's abusing DiagnosticSource somewhat, but it works!

Summary

In this post I described how the EF Core tools work with the new minimal hosting WebApplicationBuilder and WebApplication. With these types and top-level programs, the previous convention-based approaches EF Core used to load an IServiceProvider for your app will no longer work.

To work around these changes, new DiagnosticSource events were added to HostBuilder. These events allow subscribers access to the HostBuilder just before it is built, as well as access to the IHost instance immediately after it's built.

The EF Core tools use these events to run your application and to retrieve the IHost instance (and associated IServiceProvider). They then throw an exception so that your app doesn't actually run.