I recently used the previous posts in this series to build integration tests for a website that I've been working on. For this project, I was required to use xUnit and FluentAssertions, two tools that I hadn't worked with before. Anyway, although some basic tests were working fine, I stumbled upon an issue when testing my api endpoints. The test was supposed to be simple: "My news api endpoint returns news articles":
[Fact]
public async Task Get_ValidRequest_Returns200OKWithJsonObject()
{
// arrange
// act
var response = await Client.GetAsync(_endpoint + "?skip=0&take=10");
// assert
response.StatusCode.Should().Be(HttpStatusCode.OK, because: "the request is valid");
var result = await DeserializeResponseAsync<GetNewsResponse>(response);
result.Should().NotBeNull(because: "this endpoint always returns an object");
result!.Total.Should().BePositive(because: "results exist");
result.Items
.Should().HaveCountLessThanOrEqualTo(10, because: "up to 10 results have been requested")
.And.HaveCountGreaterThan(0, because: "results exist");
result.Items.Should().NotContain(e => e.Title == null || e.SubTitle == null || e.Id == 0);
}
I was only able to get 0 articles in my integration test though, while I knew for sure that I should be receiving 3 articles.
So I went searching...
Discovering the issue
The first clue that I got was a bunch of warnings from uSync, stating that it couldn't reconstruct my content. Some communication in the Umbraco Discord revealed that this was a bug in uSync and it was quickly patched in version 10.3.2 (so go update if you haven't already 😛)
That was only the start though...
The examine rebuilder
I discovered that I could recreate the behaviour manually if I deleted the TEMP index files. For about 1 minute I got 0 results, but then it fixed itself. Using the debugger, I discovered the ExamineIndexRebuilder class and the RebuildOnStartupHandler. As it turns out, these are the key classes that handle index initialisation. There were a few changes that I had to make in order to get my integration tests to work:
- I needed to know when the RebuildIndex method in the ExamineIndexRebuilder returns so I could send a signal to the integration tests.
- Since I couldn't monitor that method directly, I needed to force the method to the foreground by manipulating the useBackgroundThread parameter.
- Static properties cause interference between tests, so I had to eliminate the static fields in the RebuildOnStartupHandler.
- To keep my sanity, I had to get rid of the hardcoded call to Thread.Sleep().
I started writing...
The solution
In order to solve the issues, I had to replace both the ExamineIndexRebuilder and the RebuildOnStartupHandler implementation. Although replacing only the handler might've also worked, I feared that unseen references may cause issues later if I didn't change the index rebuilder as well.
Fixing ExamineIndexRebuilder
First I made a small helper object:
public class ExamineWaitContext
{
private TaskCompletionSource? _tcp = null;
public Task Set()
{
_tcp?.TrySetCanceled();
_tcp = new TaskCompletionSource();
return _tcp.Task;
}
public void Fire()
{
_tcp?.TrySetResult();
}
}
This helper allowed me to asynchronously wait for the index rebuild to finish.
Next, I created a new implementation of the ExamineIndexRebuilder:
internal class CustomExamineIndexRebuilder : ExamineIndexRebuilder
{
private readonly ExamineWaitContext _waitContext;
public CustomExamineIndexRebuilder(IMainDom mainDom,
IRuntimeState runtimeState,
ILogger<ExamineIndexRebuilder> logger,
IExamineManager examineManager,
IEnumerable<IIndexPopulator> populators,
IBackgroundTaskQueue backgroundTaskQueue,
ExamineWaitContext waitContext)
: base(mainDom, runtimeState, logger, examineManager, populators, backgroundTaskQueue)
{
_waitContext = waitContext;
}
public override void RebuildIndexes(bool onlyEmptyIndexes, TimeSpan? delay = null, bool useBackgroundThread = true)
{
// Force rebuilding of indexes on the current thread
// so that the integration tests can wait for the rebuild to complete.
base.RebuildIndexes(onlyEmptyIndexes, delay, false);
// After rebuilding, send a signal to the wait context that the rebuild is done
// This way, the integration tests know that rebuilding is finished and testing can begin
_waitContext.Fire();
}
}
Then, I registered these services in my WebApplicationFactory:
private void ConfigureServices(IServiceCollection services)
{
// First remove the original implementations
services.RemoveAll<ExamineIndexRebuilder>();
services.RemoveAll<IIndexRebuilder>();
// Then add the new implementations
services.AddSingleton<ExamineIndexRebuilder, CustomExamineIndexRebuilder>();
services.AddSingleton<IIndexRebuilder, CustomExamineIndexRebuilder>();
services.AddSingleton<ExamineWaitContext>();
}
Fixing RebuildOnStartupHandler
This fix was unfortunately not as graceful as the fix for the ExamineIndexRebuilder, but it was a key step to make my integration tests work.
Firstly, I made a state object as replacement for the static fields:
public class CustomRebuildOnStartupHandlerState
{
public bool _isReady;
public bool _isReadSet;
public object? _isReadyLock;
}
Then, I made a copy of the RebuildOnStartupHandler and made a couple changes:
public sealed class CustomRebuildOnStartupHandler
: INotificationHandler<UmbracoRequestBeginNotification>
{
// The static private fields have been removed
// and replaced with the _state field,
// which is injected in the constructor
private readonly ExamineIndexRebuilder _backgroundIndexRebuilder;
private readonly IRuntimeState _runtimeState;
private readonly CustomRebuildOnStartupHandlerState _state;
private readonly ISyncBootStateAccessor _syncBootStateAccessor;
public CustomRebuildOnStartupHandler(
ISyncBootStateAccessor syncBootStateAccessor,
ExamineIndexRebuilder backgroundIndexRebuilder,
IRuntimeState runtimeState,
CustomRebuildOnStartupHandlerState state)
{
_syncBootStateAccessor = syncBootStateAccessor;
_backgroundIndexRebuilder = backgroundIndexRebuilder;
_runtimeState = runtimeState;
_state = state;
}
// This method should be a copy of the method in the original handler,
// except using the singleton state object instead of static fields
public void Handle(UmbracoRequestBeginNotification notification)
{
if (_runtimeState.Level != RuntimeLevel.Run)
{
return;
}
// This method call now uses the fields in the state object,
// rather than the static fields
LazyInitializer.EnsureInitialized(
ref _state._isReady,
ref _state._isReadSet,
ref _state._isReadyLock,
() =>
{
SyncBootState bootState = _syncBootStateAccessor.GetSyncBootState();
// This method now passes TimeSpan.Zero to this method, so that we don't have to wait that 1 minute at the start.
_backgroundIndexRebuilder.RebuildIndexes(
bootState != SyncBootState.ColdBoot,
TimeSpan.Zero);
return true;
});
}
}
Of course, these new classes were also registered in the dependency injection container in the website factory:
// The umbraco rebuild on startup handler gets removed, because the static fields inside of it break unit tests when running multiple at once.
// If examine related tests start breaking, check if the implementation of this type has changed.
// The service is replaced with a custom notification handler which uses a singleton class instead of static fields
var sd = services.First(s => s.ImplementationType == typeof(RebuildOnStartupHandler));
services.Remove(sd);
services.Add(new UniqueServiceDescriptor(typeof(INotificationHandler<UmbracoRequestBeginNotification>), typeof(CustomRebuildOnStartupHandler), ServiceLifetime.Transient));
services.AddSingleton<CustomRebuildOnStartupHandlerState>();
Fixing the test
The last thing I had to do was to trigger the initialisation of the website before running any tests. To trigger initialisation, I made a request to the homepage:
IntegrationTestBase.cs
public abstract class IntegrationTestBase
: IDisposable
{
// ... the usual fields
private readonly Task _initializationTask;
public IntegrationTestBase()
{
WebsiteFactory = new UmbracoApplicationFactory();
_initializationTask = WebsiteFactory.StartAsync();
var ssf = WebsiteFactory.Services.GetRequiredService<IServiceScopeFactory>();
Scope = ssf.CreateAsyncScope();
Client = CreateHttpClient();
}
protected Task PrepareAsync()
{
return _initializationTask;
}
// ... the rest of my integration test base class
}
UmbracoApplicationFactory.cs
public class UmbracoWebsiteFactory
: WebApplicationFactory<Program>
{
// ... The usual implementation of the website factory
private bool _started = false;
public async Task StartAsync()
{
if (!_started)
{
var waitContext = Services.GetRequiredService<ExamineWaitContext>();
var waitHandle = waitContext.Set();
await CreateClient().GetAsync("/");
await waitHandle;
_started = true;
}
}
}
To finish it up, I had to add one extra line to each test:
// arrange
await PrepareAsync();
After that, I finally got the green checkmarks on all my integration tests! 🎉
Conclusion
It was surprisingly challenging to get the integration tests working with examine. I'm very happy that I managed to get it working, though I have mixed feelings about the solution. Replacing core services from Umbraco should be avoided as much as possible as there is no guarantee that they stay the same. This little journey did teach me a lot about the nature of Umbraco and it's services and about the possibilities with .NET 6.
What do you think of this solution? How do you feel about replacing core Umbraco services? Let me know with a comment and also feel free to leave a like and some feedback on this post. Thanks for reading!
Oldest comments (0)