computers are cool

mookid on code

Castle Windsor resolution logic: Handler filters

August 10th, 2011 by mookid

One of the features I’m looking forward to in the upcoming Castle Windsor 3.0 (“Wawel”) release is handler filters! I’m especially looking forward to this feature because it solves a problem I have had quite a few times now, and also because it’s a feature that I’ve contributed to the Windsor project.

Please note that handler filter are NOT available before Windsor 3.0, which will probably be out sometime around middle August 2011.

As you can see, handler selectors is a hook in Resolve that sorts out the situation by choosing one handler among all the available handlers. Pretty much related to this, are handler filters, which is a hook in ResolveAll, that sorts out the situation by choosing and ordering several handlers from all the available handlers. Let’s take a look at a tiny example….

Example: Task processing pipeline

A golden use case for this feature is when you rely on CollectionResolver to inject a collection for you, e.g. in chain of responsibility-like scenarios like so:

public class TaskProcessor
{
    readonly IEnumerable<ITask> tasks;
    readonly IErrorReporter errorReporter;
 
    public TaskProcessor(IEnumerable<ITask> tasks, IErrorReporter errorReporter)
    {
        this.tasks = tasks;
        this.errorReporter = errorReporter;
    }
 
    public void ProcessTasks()
    {
        foreach(var task in tasks)
        {
            try
            {
                task.DoIt();
            }
            catch(SomeDomainException ex)
            {
                errorReporter.ReportErrorProcessingTask(task, ex);
                break;
            }
        }
    }
}

Usually, when doing this kind of task processing, you care about the order in which the tasks are processed. Let’s pretend we have some tasks:

public class FinishTheJob : ITask
{
    public void DoIt()
    {
        Console.WriteLine("Finishing stuff");
    }
}
 
public class PrepareSomething : ITask
{
    public void DoIt()
    {
        Console.WriteLine("Preparing stuff");
    }
}
 
public class CarryItOut : ITask
{
    public void DoIt()
    {
        Console.WriteLine("Carrying out some important logic");
    }
}

and the tasks are registered “dynamically” (allowing us to add new tasks just by dropping new implementations of ITask into the project), like so:

container.Register(AllTypes.FromThisAssembly().BasedOn<ITask>().WithService.Base());

Now, how do we make sure that the PrepareSomething and FinishTheJob tasks are run at the right time, i.e. before and after CarryItOut? Handler filters to the rescue!

Let’s register a handler filter like so:

container.Kernel.AddHandlersFilter(new TaskHandlersFilter());

and the filter is implemented like this:

class TaskHandlersFilter : IHandlersFilter
{
    readonly Dictionary<Type, int> sortOrder = new Dictionary<Type, int>
                                                   {
                                                       {typeof (PrepareSomething), 1},
                                                       {typeof (CarryItOut), 2},
                                                       {typeof (FinishTheJob), 3},
                                                   };
 
    public bool HasOpinionAbout(Type service)
    {
        return service == typeof(ITask);
    }
 
    public IHandler[] SelectHandlers(Type service, IHandler[] handlers)
    {
        // come up with some way of ordering implementations here
        // (cool solution coming up in the next post... ;))
        return handlers
            .OrderBy(h => sortOrder[h.ComponentModel.Implementation])
            .ToArray();
    }
}

Now, when my program does this:

foreach(var task in container.ResolveAll<ITask>())
{
    task.DoIt();
}

I get this output:

Preparing stuff
Carrying out some important logic
Finishing stuff

just as expected, which in turn means that CollectionResolver‘s call to ResolveAll will also yield an ordered list of tasks.

One could also imagine that only certain handlers were returned, thus allowing tenants in multi-tenant scenarios to have different task processing pipelines.

Nifty, huh? In the next post, I will show a simple yet sophisticated way of ordering the handlers in the SelectHandlers method of a handler filter.

Castle Windsor resolution logic: Handler selectors

August 8th, 2011 by mookid

Do you know what handler selectors are? It’s a way for you to register some logic in Windsor that will choose among all the available handlers when some particular component is resolved.

E.g. if you have a multi-tenant application with multiple implementations of IUserNotifier, let’s call them NotifyUserBySms and NotifyUserByEmail, and you need to make sure that one particular tenant whose employees are equipped with smartphones get notified by email, you can let a handler selector make this selection transparent to the rest of your application by adding a handler selector, like so:

class UserNotificationHandlerSelector : IHandlerSelector
{
    public bool HasOpinionAbout(string key, Type service)
    {
        return service == typeof (IUserNotifier);
    }
 
    public IHandler SelectHandler(string key, Type service, IHandler[] handlers)
    {
        // somehow determine whether employees of current tenant can receive emails
        Console.WriteLine("Does the current tenant' s employees have smartphones? (y/n)");
 
        switch (char.ToLower(Console.ReadKey().KeyChar))
        {
            case 'y':
                return handlers.Single(h => h.ComponentModel.Implementation == typeof (NotifyUserByEmail));
            default:
                return handlers.First(h => h.ComponentModel.Implementation != typeof (NotifyUserByEmail));
        }
    }
}

thus making it possible for all other services to depend on IUserNotifier, not caring under which tenant’s context they’re running.

As you can see, this handler selector only has an opinion when an IUserNotifier is being resolved. And then, when its SelectHandler method is invoked, it gets to choose among all assignable handlers for that particular service type, effectively providing a way to make sure that some particular implementation is used, given some condition (here simulated by some Console.ReadKey() action).

Nifty huh? Only sad thing though, is that handler selectors are not pulled from the container – you have to supply an instance to the kernel, so if you need some services in the selector you need to pull them manually from the container. This can of course be overcome by supplying the handler selector with a reference to the container, allowing it to Resolve stuff at will – just remember – as always – to Release what you Resolve.

This was a short introduction to handler selectors – next post will be about something new, namely handler filters which is a new feature in Windsor 3.0.

Frictionless Persistence in .NET with MongoDB is available on Channel 9

June 24th, 2011 by mookid

Microsoft were cool enough to record my talk at this year’s Goto Cph, even though the talk was on the Cloud and NoSQL track.

You can watch it here.

Afterwards, go check out all the other great presentations – it’s so cool they’ re made available!

On the bus with MassTransit #1

June 3rd, 2011 by mookid

This is the first post in a series of posts on the open source service bus implementation, MassTransit. In order to push myself through learning it, I will do these posts as I go and document stuff I learn on the way.

This first post will be about one thing, that you’ll probably end up doing quite a few times while building your awesome messaging-based distributed system: Creating a service.

Check out TopShelf and build it

I’ve described how to use TopShelf before, but a lot has changed since then. TopShelf, at the time of writing, is now in version 2.2.1.0, and since 2.0 it has a feature called “shelving”, which is the ability to host any number of services beneath one Windows Service – just like IIS hosts web apps. This is pretty awesome, but today I’ll just show “the old API”, which IMO is still pretty slick!

First, make sure you have a working Ruby installation with Albacore (gem install albacore). You’ll need these to build Topshelf.

Next, git clone git://github.com/phatboyg/Topshelf.git and build.bat. That should leave a bunch of binaries in the NET35 and NET40 folders beneath build_output. This is open source, baby!

Put your app on the TopShelf

Create a new “Console Application” and make sure to target the full .NET 4 framework (and not that bastard .NET 4 Client Profile thingie, which is default…)

Now add a reference to Topshelf.dll and log4net.dll. Punch in something like this:

class Program
{
    static void Main()
    {
        HostFactory.Run(x =>
        {
            x.SetDescription("MyBackend-description");
            x.SetDisplayName("MyBackend-display-name");
            x.SetServiceName("MyBackend-service-name");
        });
    }
}

This is all the necessary stuff to create a fully functional program, that can be run and debugged by running the resulting .exe – either from the command line, or with F5 inside Visual Studio.

Moreover, it can be installed as a Windows Service by going to the command line (as an administrator!) and executing NameOfYourApp.exe install. On my machine, I got this output:

C:\temp\TransitDemo\Backend\bin\Debug>Backend.exe  install
 
Running a transacted installation.
 
Beginning the Install phase of the installation.
Installing service MyBackend-service-name..
Service MyBackend-service-name has been successfully installed.
Creating EventLog source MyBackend-service-name in log Application...
 
The Install phase  completed successfully, and the Commit phase is beginning.
 
The Commit phase completed successfully.
 
The transacted install has completed.

Now I can net start MyBackend-service-name and net stop MyBackend-service-name on the command line, or I can go to Windows’ service control snap-in and pull the usual levers to control it. Nifty!

Now I have a perfectly functioning Windows Service that does nothing at all! Let’s output some stuff to see where we’ll put our code in a short while. Try extending the x => { ... } lambda to make the program look like this:

class Program
{
    static void Main()
    {
        HostFactory.Run(x =>
        {
            x.Service<MyBackend>(c =>
            {
                c.SetServiceName("MyBackend");
                c.ConstructUsing(() => new MyBackend());
                c.WhenStarted(d => d.StartMyBackend());
                c.WhenStopped(d => d.StopMyBackend());
            });
 
            x.SetDescription("MyBackend-description");
            x.SetDisplayName("MyBackend-display-name");
            x.SetServiceName("MyBackend-service-name");
        });
    }
}

and a class that looks something like this:

class MyBackend
{
    public void StartMyBackend()
    {
        Console.WriteLine("Started!");
    }
 
    public void StopMyBackend()
    {
        Console.WriteLine("Stopped!");
    }
}

Now, when I press F5, wait a little, and press Ctrl + C, I get this output on the console:

Started!
Stopped!
Press any key to continue . . .

Awesome, huh?

I really like how TopShelf now uses a Func<TService> to construct its service instance via ConstructUsing, instead of depending on the common service locator as the old TopShelf did. This way, I can pull the instance from an IoC container if I want.

In the next post I will take a look at how to send a message from a web application when someone tries to register as a user.

Free geek night about MongoDB

May 30th, 2011 by mookid

Goto ConferenceAt Goto Copenhagen this May, I gave a talk about MongoDB, which is a nifty document-oriented database that I find pretty interesting.

So, because I like to talk about MongoDB so much, I’ll give my talk again as a free Trifork geek night on Tuesday the 21th of June at the Trifork HQ in Aarhus (this time in Danish though).

If you’re a .NET person, possibly developing big enterprisey stuff and/or you’re interested in MongoDB or NoSQL in general, you should come to this one.

Tinkering with MongoDB and sharding

May 12th, 2011 by mookid

After my MongoDB presentation today, I was asked a few questions about MongoDB’s sharding capabilities. Like my interest so far, my talk was completely focused on the frictionless aspects of using MongoDB, so I have never tried any of the sharding and replica set configurations that MongoDB can run in.

That has got to end!

So, let’s try spinning up the simplest possible sharding scenario that we can think of: Two durable MongoDB instances on port 27017 and 27018 with one collection sharded across them:

mkdir c:\data\db1
mongod --shardsvr --port 27017 --journal --dbpath c:\data\db1
mkdir c:\data\db2
mongod --shardsvr --port 27018 --journal --dbpath c:\data\db2

MongoDB sharding requires that you spin up a special configuration server that stores the configuration of which shards are available – let’s spin this up on port 27019:

mkdir c:\data\configdb
mongod --configsvr --port 27019 --dbpath c:\data\configdb

- and finally, we spin up one instance of mongos on port 27020, pointing it towards the configuration server:

mongos --configdb localhost:27019 --port 27020

To finalize the setup, we let the configuration database know of the two shards we have started by connecting to the admin database using the Mongo shell:

C:\mongodb\bin>mongo localhost:27020
MongoDB shell version: 1.8.1
connecting to: localhost:27020/test
> use admin
switched to db admin
> db.runCommand({addShard:"localhost:27017"})
> db.runCommand({addShard:"localhost:27018"})

Now, let’s see if it understood this:

> db.runCommand({listShards: 1})
{
        "shards" : [
                {
                        "_id" : "shard0000",
                        "host" : "localhost:27017"
                },
                {
                        "_id" : "shard0001",
                        "host" : "localhost:27018"
                }
        ],
        "ok" : 1
}

As you can see, the config database correctly stored information about our two shards – that was easy!

Finally, we need to enable sharding for one particular database and make sure that our collection of unicorns is sharded by the name field:

> db.runCommand({enableSharding: "fairytale"})
{ "ok" : 1 }
> db.runCommand({shardCollection: "fairytale.unicorns", key: {name: 1}})
{ "collectionsharded" : "fairytale.unicorns", "ok" : 1 }

At this point, even though the fairytale database and unicorns collection did not exist, they have been created for me, and the required index has been created for the shard key. I can verify this like so:

> use fairytale
> db.unicorns.getIndexes()
[
        {
                "name" : "_id_",
                "ns" : "fairytale.unicorns",
                "key" : {
                        "_id" : 1
                },
                "v" : 0
        },
        {
                "ns" : "fairytale.unicorns",
                "key" : {
                        "name" : 1
                },
                "name" : "name_1",
                "v" : 0
        }
]

Now, let’s go to C# with mongo-csharp and hammer 10 million randomly named unicorns, each carrying a payload of up to 8 KB of fairy dust in there:

class Program
{
    static readonly Random Random = new Random();
 
    static void Main()
    {
        var server = MongoServer.Create("mongodb://localhost:27020");
        var database = server.GetDatabase("fairytale");
        var unicorns = database.GetCollection("unicorns");
 
        using (database.RequestStart())
        {
            var batch = Enumerable.Range(0, 10000000)
                .Select(_ =>
                        new
                            {
                                name = GenerateRandomUnicornName(),
                                horns = 1,
                                likes = new[] {"eggs", "beer"},
                                fairy_dust = GenerateRandomBytes(Random.Next(8192))
                            });
 
            unicorns.InsertBatch(batch);
 
            server.GetLastError();
        }
 
        server.Disconnect();
    }
 
    static byte[] GenerateRandomBytes(int length)
    {
        var bytes = new byte[length];
        Random.NextBytes(bytes);
        return bytes;
    }
 
    static string GenerateRandomUnicornName()
    {
        Func<char> charFactory = () => (char)('a' + Random.Next(25));
        var funcs = Enumerable.Repeat(charFactory, 10 + Random.Next(30));
        return new string(funcs.Select(f => f()).ToArray());
    }
}

now, let’s go to the Mongo shell and see if they’re there:

> db.unicorns.count()
10000000

Great! 10 million unicorns in there. Let’s check out the disk and see if data was somehow distributed among the shards:

That seems to be pretty well balanced if you ask me. Let’s see what Mongo can tell us:

> db.printShardingStatus()
--- Sharding Status ---
  sharding version: { "_id" : 1, "version" : 3 }
  shards:
      { "_id" : "shard0000", "host" : "localhost:27017" }
      { "_id" : "shard0001", "host" : "localhost:27018" }
  databases:
        { "_id" : "admin", "partitioned" : false, "primary" : "config" }
        { "_id" : "fairytale", "partitioned" : true, "primary" : "shard0000" }
                fairytale.unicorns chunks:
                                shard0001       66
                                shard0000       65
                        too many chunksn to print, use verbose if you want to force print

66 chunks on the 1st shard, and 65 chunks on the 2nd shard – it is in fact pretty well balanced.

Conclusion so far

It seems to be pretty easy to begin sharding data, which is perfectly in line with the usual MongoDB feeling. It’s definitely a subject I need to look more into though, so if you want to read more about it, I can really recommend Kristina Chodorow‘s blog: Snail In A Turtleneck.

I’ll be speaking about MongoDB at GotoCph

April 1st, 2011 by mookid

I started playing around with MongoDB about a year ago, and since then I have grown to like it so much that I am going to try to tell what I know at the first ever Goto Copenhagen.

The talk is scheduled on the “Cloud and NoSQL” track on Thursday at 14:05.

A link directly to the abstract can be found here.

(PS: This is not an April fool’s day prank… :) )

How to set the current culture in NServiceBus

March 21st, 2011 by mookid

Today we were experiencing some weird behavior when running an integration test with DillPickle, where – apparently – values of doubles would lose their decimal point when they were transferred in messages from our test to an NServiceBus service.

Stopping the service and inspecting the message in the queue quickly revealed a message that looked somewhat like this:

<Messages>
    <SetCurrentValueMessage>
        <Value>13.56</Value>
    </SetCurrentValueMessage>
</Messages>

which is all fine and dandy.

Now, I’m used to being Danish, so I know that we’re somewhat deviant in regards to our decimal point – “,” – so we quickly diagnosed the problem: Our integration tests were running with the invariant culture, to allow us to parse Gherkin files in English and use “.” as the decimal point – but Windows and everything else was running with da-DK, so 13.56 would be improperly deserialized to the value 1356 when it reached our NServiceBus service.

Solution: Normalize the culture of all the processes of our system.

Our first attempt was to modify the culture in our endpoint configuration like so:

public class EndpointConfiguration : IConfigureThisEndpoint, AsA_Server
{
    public EndpointConfiguration()
    {
        // does NOT work!
        Thread.CurrentThread.CurrentCulture = CultureInfo.InvariantCulture;
        Thread.CurrentThread.CurrentUICulture = CultureInfo.InvariantCulture;
    }
}

but obviously this did not work, because NServiceBus does not deserialize messages on this thread!

Our solution was to create a message module, which seems to get called before transport messages are deserialized, setting the culture in there – like so:

public class SetCurrentCultureMessageModule : IMessageModule
{
    public void HandleBeginMessage()
    {
        // works!
        Thread.CurrentThread.CurrentCulture = CultureInfo.InvariantCulture;
        Thread.CurrentThread.CurrentUICulture = CultureInfo.InvariantCulture;
    }
 
    public void HandleEndMessage()
    {
    }
 
    public void HandleError()
    {
    }
}

In the future I’ll make sure that the culture is explicitly set in all processes of systems I am building. It’s kind of scary that errors could happen where stuff like “debit account 100.00″ could be mis-interpreted as “debit account 10000″!! :-o

Little nifties #1

March 18th, 2011 by mookid

Almost always, the first extensions methods I add to a system are these:

public static class EnumerableExtensions
{
    public static string JoinToString<T>(this IEnumerable<T> enumerable, Func<T, object> selector, string separator)
    {
        return enumerable.Select(selector).JoinToString(separator);
    }
 
    public static string JoinToString<T>(this IEnumerable<T> enumerable, Func<T, object> selector)
    {
        return enumerable.Select(selector).JoinToString(", ");
    }
 
    public static string JoinToString(this IEnumerable enumerable, string separator)
    {
        var strings = enumerable.Cast<object>()
            .Select(e => e == null ? "(null)" : e.ToString())
            .ToArray();
 
        return string.Join(separator, strings);
    }
}

which allows me to format any sequence of stuff sensibly, easily, and inline, e.g. like so:

Log.WarnFormat("Some stuff happened to stuff with these names: {0}", names.JoinToString(", "));

and so:

Log.WarnFormat("Some stuff happened to things with these names: {0}", things.JoinToString(t => t.Name));

and so:

catch(ReflectionTypeLoadException exception)
{
    Log.ErrorFormat(@"ZOMG!!111something happened!!
 
 
{0}", exception.LoaderExceptions.JoinToString(Environment.NewLine));
}

Nifty!

Tell, Don’t Ask

March 7th, 2011 by mookid

or “How big is an interface?”… Uuuh, what?

Well, consider these two interfaces:

public interface ISomeService1
{
    void SetStuff(double value);
    void SetSomeOtherStuff(string anotherValue);
    void SetMoreStuff(bool moreStuff);
    void SetEvenMoreStuff(bool eventMoreStuff);
}
 
public interface ISomeService2
{
    SomeValue GetValue();
}

- which is bigger?

If you think that ISomeService1 is the bigger interface, then there’s a good chance you’re wrong! Why is that?

It’s because an interface is not just the signatures of the methods and properties it exposes, it also consists of the types that go in and out of its methods and properties, and the assumptions that go along with them!

This is a fact that I see people often forget, and this is one of the reasons why everything gets easier if you adhere to the Tell, Don’t Ask principle. And by “everything”, I almost literally mean “everything”, but one thing stands out in particular: Unit testing!

Consider this imaginary API:

public interface ISomeKindOfValueStore
{
    Values GetValues(string name); 
}

Now, in our system, on various occasions we need to change the values for a particular name. We start out by doing it like this:

public class BusinessLogic
{
    ISomeKindOfValueStore someKindOfValueStore;
    public BusinessLogic(ISomeKindOfValueStore someKindOfValueStore)
    {
        this.someKindOfValueStore = someKindOfValueStore;
    }
 
    public void Run()
    {
        var values = someKindOfValueStore.GetValues("MegaCorp");
 
        values.Value1 = 2.3; //< secret biz constants
        values.Value2 = 4.5;
 
        // more logic in here
    }
}

Obviously, if GetValues returns a reference to a mutable object, and this object is the one kept inside the value store, this will work, and the system will chug along nicely.

The problem is that this has exposed much more than needed from our interface, including the assumption that the obtained Values reference is to the cached object, and an assumption that the Value1 and Value2 properties can set set, etc.

Now, imagine how tedious it will be to test that the Run method works, because we need to stub an instance of Values as a return value from GetValues. And when multiple test cases need to exercise the Run method to test different aspects of it, we need to make sure that GetValues returns a dummy object every time – otherwise we will get a NullReferenceException.

Now, let’s improve the API and change it into this:

public interface ISomeKindOfValueStore
{
    void SetValues(string name, double newValue1, double newValue2); 
}

adhering to the “Tell, Don’t Ask” principle, allowing a potential usage scenario like this:

public class BusinessLogic
{
    ISomeKindOfValueStore someKindOfValueStore;
    public BusinessLogic(ISomeKindOfValueStore someKindOfValueStore)
    {
        this.someKindOfValueStore = someKindOfValueStore;
    }
 
    public void Run()
    {
        someKindOfValueStore.SetValues("MegaCorp", 2.3, 4.5);
 
        // more logic in here
    }
}

As you can see, I have changed the API from a combined query/command thing to a pure command thing which appears to be much cleaner to the eye – it actually reveals exactly what is going on.

And testing has suddenly become a breeze, because our mocked ISomeKindOfValueStore will just record that the call happened, allowing us to assert it in the test cases where that is relevant, ignoring it in all the other test cases.

Another benefit is that this coding style lends itself better to stand the test of time, as it is more tolerant to changes – the implementation of ISomeKindOfValueStore may change an in-memory object, update something in a db, send a message to an external system, etc. A command API is just easier to change.

Therefore: Tell, Don’t Ask.

« Previous Entries Next Entries »