Service Fabric is a distributed systems platform for packaging, deploying, and managing stateless and stateful distributed applications and containers at large scale. Service Fabric runs on Windows and Linux, on any cloud, any datacenter, across geographic regions, or on your laptop.
Introduction
In this post we will learn two things that can make life easier to support advanced scenarios when using Service Remoting for communication between services and actors. We will do this by making use of a simple library I created. It provides you with the ability to:
- Intercept service fabric remoting messages so you can take action when the call is about to be made and after it has been delivered.
- Add custom headers to the remoting messages. This can be used to, for example, add a trace identifier for call tracing & logging purposes.
Message Interception
Messages can be intercepted on both the sending side and the receiving side
Client-side message interception
On the receiving side messages can be intercepted using the BeforeHandleRequestResponseAsync and AfterHandleRequestResponseAsync extension points when creating a service listener:
protected override IEnumerable<ServiceInstanceListener> CreateServiceInstanceListeners()
{
yield return new ServiceInstanceListener(context =>
new FabricTransportServiceRemotingListener(context,
new ExtendedServiceRemotingMessageDispatcher(context, this)
{
// Optional, log the call before being handled
BeforeHandleRequestResponseAsync = reqInf =>
{
var sw = new Stopwatch();
sw.Start();
ServiceEventSource.Log.ServiceMessage(Context,
$"BeforeHandleRequest {reqInf.Service} {reqInf.Method}");
return Task.FromResult<object>(sw);
},
// Optional, log the call after being handled
AfterHandleRequestResponseAsync = respInf =>
{
var sw = (Stopwatch) respInf.State;
ServiceEventSource.Log.ServiceMessage(Context,
$"AfterHandleRequest {respInf.Method} took {sw.ElapsedMilliseconds}ms");
return Task.CompletedTask;
}
}));
}
Server-side message interception
On the sending side messages can be intercepted using the BeforeSendRequestResponseAsync and AfterSendRequestResponseAsync extension points when creating the ExtendedServiceRemotingClientFactory on constructor of the ServiceProxyFactory:
var proxyFactory = new ServiceProxyFactory(handler => // or ActorProxyFactory in case of actors
new ExtendedServiceRemotingClientFactory(
new FabricTransportServiceRemotingClientFactory(remotingCallbackMessageHandler: handler),
customHeadersProvider)
{
// Optional, log the call before being handled
BeforeSendRequestResponseAsync = reqInf =>
{
var sw = new Stopwatch();
sw.Start();
Console.WriteLine($"BeforeSendRequest {reqInf.Method}");
return Task.FromResult<object>(sw);
},
// Optional, log the call after being handled
AfterSendRequestResponseAsync = respInf =>
{
var sw = (Stopwatch)respInf.State;
var duration = sw.ElapsedMilliseconds;
Console.WriteLine($"AfterSendRequest {respInf.Method} took {duration}ms");
return Task.CompletedTask;
}
});
Custom Headers
Custom headers can be used to pass data between the sender and the receiver like tracing information or security context data. Using the BeforeHandleRequestResponseAsync and AfterHandleRequestResponseAsync actions additional logging can be applied monitor the flow between remoting calls.
Prepare the Reliable Service
Modify the service and create a listener that can handle the requests:
protected override IEnumerable<ServiceInstanceListener> CreateServiceInstanceListeners()
{
yield return new ServiceInstanceListener(context =>
new FabricTransportServiceRemotingListener(context,
new ExtendedServiceRemotingMessageDispatcher(context, this)));
}
Adding custom headers (the caller)
var customHeaders = new CustomHeaders
{
{"Header1", DateTime.Now},
{"Header2", Guid.NewGuid()}
};
var serviceUri = new Uri("fabric:/ServiceFabric.Remoting.CustomHeaders.DemoApplication/DemoService");
var proxyFactory = new ServiceProxyFactory(handler =>
new ExtendedServiceRemotingClientFactory(
new FabricTransportServiceRemotingClientFactory(remotingCallbackMessageHandler: handler),
customHeaders));
var proxy = proxyFactory.CreateServiceProxy<IDemoService>(serviceUri);
There is an overload of the Create method that accepts a Func. This is useful in scenarios where the created proxy factory or proxy is reused. Since creating a proxy factory is expensive this is the preferred way if you need dynamic header values. The func is invoked on every request made using the proxy.
Reading custom headers (the callee)
The receiving service or actor can extract the values in the custom headers using the RemotingContext class:
public async Task<string> SayHello()
{
var remotingContext =
string.Join(", ", RemotingContext.Keys.Select(k => $"{k}: {RemotingContext.GetData(k)}"));
ServiceEventSource.Log.ServiceMessage(Context,
$"SayHelloToActor got context: {remotingContext}");
return Task.FromResult($"Got the following message headers: {remotingContext}")
}
What is next?
More documentation and a demo Service Fabric application can be found at this repository:
Expecho
/
ServiceFabric-Remoting-CustomHeaders
This package allows injecting custom message headers into remoting messages (Actors and Reliable Services, V2 remoting only) at runtime. The headers are available client side to read. It also provides message interception using BeforeHandleRequestResponseAsync and AfterHandleRequestResponseAsync to act on remoting events.
ServiceFabric.Remoting.CustomHeaders
This package allows injecting custom headers into remoting messages (Actors and Reliable Services, V2 remoting only) at runtime. The headers are available client side to read It also provides message interception using BeforeHandleRequestResponseAsync and AfterHandleRequestResponseAsync to act on remoting events.
Common used classes:
NuGet
Examples
This repository includes a Service Fabric application for demonstration purposes. A Console Application is used to access the application and shows the usage of the package.
Usage scenarios
Custom headers can be used to pass data between the sender and the receiver like tracing information or security context data. Using the BeforeHandleRequestResponseAsync and AfterHandleRequestResponseAsync actions additional logging can be applied monitor the flow between remoting calls.
How to use
Prepare Reliable Services
Modify the service and create a listener that can handle the requests
protected override IEnumerable<ServiceInstanceListener> CreateServiceInstanceListeners()
{
yield return new ServiceInstanceListener…
Top comments (3)
This is great, as it opens up some opportunities to extend the functionality of a given service without having to necessarily update the contract. Could also add some opportunities for a more 'aspect-oriented' approach to adding general purpose functionality around services without cluttering the service contract. Obviously overuse of that as a technique would make the code quite ugly, but everything in moderation 👍
Yeah I mainly used it for tracing purposes. I wouldn't use it to pass all kind of information to services instead of using proper variables for instance.
This is a great piece of functionality, just POC'd the Remoting piece in my own sandbox. Any chance you could split the Reliable Services and Actor pieces out into separate libraries with a lower dependency level ? I've gotten the code working as early as SDK 3.3.664