Trifork Geek Nights

Posted on by mookid8000 3 Comments

Just wanted to say that I will be speaking at a couple of Trifork Geek Nights in November and December, about Castle Windsor and NServiceBus respectively.

First,

Advanced Windsor-Tricks

A quick introduction followed by some of the more advanced features of my favorite IoC container. If you’re new to IoC, or you are wondering what the fuzz is about, or you are interested in letting your container do some more work for you, you should come to this one 🙂

Will be held on

from 4:30 pm to 6:30 pm.

And then,

Distributed systems in .NET with NServiceBus

Introduction to messaging in .NET with NServiceBus. Will give an introduction to some fundamental messaging patterns and go on to show how these can be put to use with NServiceBus.

If you’re new to IoC, you can probably benefit from showing up at the “Advanced Windsor-Tricks” geek night before going to this one, because NServiceBus is relying heavily on having a container.

Will be held on

from 4:30 pm to 6:30 pm as well.

Hope to see some enthusiastic coders there.

PS: The Geek Nights will be held in Danish 🙂

Scheduling recurring tasks in NServiceBus

Posted on by mookid8000 6 Comments

A while ago, on a project I am currently involved with which is based on NServiceBus, we needed to publish certain pieces of information at fixed intervals. I was not totally clear in my head on how this could be implemented in an NServiceBus service, so I asked for help on Twitter, which resulted in a nifty piece of advice from Andreas Öhlund: Set up a timer to do a bus.SendLocal at the specified interval.

That’s exactly what we did, and I think we ended up with a pretty nifty piece of code that I want to show off 🙂

PS: bus.SendLocal(message) effectively does a bus.Send(((UnicastBus)bus).Address, message) – i.e. it puts a message, MSMQ and all, in the service’s own input queue.

First, we have an API that looks like this (looking a little funny, I know – wait and see…):

– which is implemented like this (registered as a singleton in the container):

The System.Timers.Timer is a timer which uses the thread pool to schedule callbacks at the specified interval. It’s pretty easy to use, and it fits nicely with this scenario.

Now, in combination with this nifty class of extension goodness:

– we can schedule our tasks like so:

Now, why is this good? It’s good because the actual task will then be carried out by whoever implements IHandleMessages<PublishRealTimeDataMessage> in the service, processing the tasks with all the benefits of the usual NServiceBus message processing pipeline.

Nifty, huh?

Looking over the simplicity and elegance of this solution, I’m kind of embarassed to tell that my first take on this was to implement the timer almost exactly like above, except instead of bus.SendLocal in the Elapsed-callback, we had a huge event handler that simulated most of our message processing pipeline – including NHibernateMessageModule, transactions, and whatnot….

Please note that ScheduleRealTimeDataPublishing is not re-entrant – in this form its Every method should only be used from within the Run and Stop methods of implementors of IWantToRunAtStartup, as these are run sequentially.

More checking out MongoDB: Updating

Posted on by mookid8000 Leave a comment

In MongoDB, there’s no way to lock a database, collection, or document. The ability to work without locking is a requirement for any db that wishes to be horizontally scalable, and obviously this imposes some limitations and/or possibilities (depending on your point of view :)).

If you want all the goodness that document-orientation brings, it seems we need to cope with this non-locking database.

So how DO you update stuff in MongoDB? And, more importantly: how do you update stuff without race conditions?

In one of my previous posts on MongoDB, I mentioned that the unit of atomicity is a document – i.e., either a document gets saved/updated/deleted or it doesn’t. That must mean that we can count on updating one document only (or not), so we should build our applications so they can work without requiring multiple documents to be updated to be consistent ([1. Which is good practice anyway! In my experience, long and wide db transactions are often used, not to enforce a strict consistency as much as to allow scenarios like: “when this happens, this should also happen”. But that kind of logic can often be handled by something else, e.g. by a publishing events reliably to other processes (logically and/or physically), that handles the side-effects.]).

First, let’s take a look at how to actually update a document.

Naïve attempt to update a document

Well, we could do this:

That is, if you go and save a document that already has an ID, any existing document with that ID will be updated.

This would work if we were the only client on the db. But what if someone was editing the post in that same moment, adding another tag as well? Well, if he was unfortunate enough to save his edits when we were out for coffee, his changes would be lost.

One way to actually do it

By using the update function!

update accepts the following four arguments:

  1. criteria – document selector that specifies which document to be updated
  2. objNew – document to save
  3. upsert – bool to specify auto-insert if document does not exist (“update if present, insert if missing”)
  4. multi – bool to allow updating multiple documents that match the criteria (default is only first document)

Actually, as you can now probably see, save(doc) is just a shorthand for update({}, doc, true, false) – an upsert with the document we’re saving.

This way, we could easily add an incrementing version field to our documents to make sure that the version we’re saving is the version we retrieved.

Let’s try it out:

– good thing he didn’t get through with that one! 🙂

As you can see, we can easily implement optimistic concurrency on each document by constraining updates to the version we checked out. But, as I will show next, you can actually do a lot of things on the server.

Server-side document updates

Instead of retrieving an entire document, modifying it, and saving it back again with the risk of overwriting someone else’s edits, we can ask the server to make edits as smaller operations. E.g. our attempt to add a missing ‘test’ tag to the document could have been done like this:

See how the code $push modifier was used to push a value into the array… this stuff is great. But here, we have a race condition again – what if someone added the ‘test’ tag almost the same time as we did? Then two ‘test’ tags would be present in the array.

One way is to constrain the update by id and the absence of ‘test’ in the tags array:

– another is to use the $addToSet function, which makes MongoDB treat the array as a set:

Nifty!

My conclusion (so far) is that an application can get a huge benefit from using the various modifier operations – performance-wise (obviously), but probably also UX-wise as well… It’s a step in another direction from the usual CRUD scenarios that I usually compulsively associate with the word “update”, and I imagine it could be made to reflect the user’s interactions with the system.

I am thinking that the majority of the user’s interactions with the system could (and probably should) be put on the form

– and then one could implement a more-or-less generic mechanism on the client side to handle unsuccessful updates (e.g. by asking the user what to do then, reloading some data to allow [assumed state before] to be something else, or by handling certain situations with some kind of merging function, etc.).

How to perform updates across multiple documents

My first thought is that this situation should be avoided when working with a document-oriented db. I think most people will agree with this one.

I am pretty unsure of this, actually… The rest of this post is just a few thoughts on my first take on this, should I need to do this. Comments are greatly appreciated!

The problem in updating multiple documents is that we can perform an update on one document at a time, each time checking if the update went well or not. But there’s no way to (consistently) roll back update #1 if update #2 fails. So this means that there’s only one way: Forward! But how to proceed then, when an update failed?

How do we usually do stuff reliably across boundaries of multiple things that may or may not succeed, allowing us to handle errors as gracefully as possible and proceed thereafter?

I’m thinking that asynchronous reliable one-way messaging is the answer to this.

So if an application ever needs to update multiple documents in the most reliable way possible, it should probably perform one document update per “transaction” – in NServiceBus terminology that would be updating one document per message handler. And then the handler should throw an exception if an update unexpectedly fails.

But again: I’m thinking that this situation should be avoided at all costs with a document-oriented db. If ACID is required, the application should probably have a RDBMS on the side, or implement some kind of transactional mechanism in the document store.

Conclusion

That concludes my little learning series of MongoDB posts.

I must say that I am intrigued by all the NoSQL discussions currently going on in the communities, and I think it is always a sign of health that we question the technologies we use.

I am entirely convinced that document dbs could and should have been used for some parts of systems that I have experienced, and I am blown away by the lack of friction when starting up a project on top of a schemaless db.

As a .NET dude, I am convinced that the future will see more .NET systems built with more than one db underneath – e.g. with MongoDB for all the “soft parts” of the system, NHibernate on SQL Server for the few things that by nature require ACID, and then some NHibernate Search/Lucene/Solr.NET for full-text indexing and searching capabilities etc.

NServiceBus for dummies who want to be smarties 6

Posted on by mookid8000 Leave a comment

NServiceBus for dummies who want to be smarties 6… sixth and last post in the series will show an example using the publish/subscribe functionality provided by NServiceBus.

One could image all kinds of cool integration scenarios where interested parties somehow receive events from our system regarding stuff that may be interesting to them. Let’s imagine that our user registration from the fifth post is an extremely interesting event to the sales department and the Ãœber Boss of our enterprise.

To accomodate this, I create a separate assembly meant for holding messages that are not private to our application. This way, I can easily distinguish between messages that I can change whenever I feel like it, and messages that I must put more care and effort into crafting, because they will most likely live forever when a dozen applications in our enterprise are suddenly depending on them.

This means that my saga from the fifth post will handle an email confirmation like this:

As you can see, I Bus.Publish(...) a message saying that someone with a particular email has registered.

To be able to publish stuff, some kind of subscription storage must be configure for the endpoint. Therefore, I change my backend’s EndpointConfig to be configured with this:

– which means my backend will be storing subscriptions in a message queue.

If you are somehow unclear on the difference between sending and publishing, I can tell you that the conceptual difference is this: sending is like saying “do this stuff”, whereas publishing is like saying “this stuff happened” – see the difference in tense?

Functionally, the difference is that sending will put a message in the queue specified as recipient for that particular message whereafter one recipient will consume that message off its input queue (possibly competing with others), whereas publishing will put a message in the input queue of each one of whoever subscribed to that particular message (possibly many – possibly zero).

Now, to simulate the sales department and the über boss, I create two new projects containing an app.config that looks like this:

– and similarly for the über boss.

Note how a subscriber must have a message endpoint mapping telling where messages will be published – in the example above, my mapping specifies that all messages from my PublicMessages assembly can be subscribe to on the endpoint named “backend”. And obviously, a subscriber must have an input queue, through which subscribed messages will be received.

The EndpointConfig can look like this:

– and similarly for the über boss as well.

Note how the service subscribes to the messages that it wants to receive by calling Bus.Subscribe<...> in the Run() method. What happens, is that NServiceBus somehow communicates to whoever is configured as the publisher of messages of type SomeoneActuallyRegistered, that messages should be sent to this service’s input queue.

And then, I implement a message hander in each service… first, the sales department:

and then, the über boss:

To make this run, I go to “Set StartUp projects…” of my solution and make all my services start up when I press F5.

Picture showing how to configure multiple startup projects

Now, we can check things out by pressing F5, thus running the backend, the sales department, and the über boss at once. On my machine, it looks like this:

Running three services at once

Now it doesn’t take too much imagination to come up with all kinds of cool solutions to integration scenarios.

Conclusion

That concludes the sixth and last post in my “NServiceBus for dummies who want to be smarties” series. I hope the series can provide some information on how to get started using NServiceBus, and perhaps fill out some of the gaps that comes from documentation in the form of sparsely available blog posts from different sources.

NServiceBus for dummies who want to be smarties 5

Posted on by mookid8000 1 Comment

NServiceBus for dummies who want to be smarties 5… fifth post, this time with an example on how sagas can be used to implement a workflow in ASP.NET MVC.

One of the typical workflows in web applications is when someone signs up for an account, but the web site wants to check if the email address is valid. Let’s look at an example that goes like this:

  1. User enters email address as a signup request
  2. Web application sends email with a “secret” confirmation link
  3. User visits the secret link, thereby activating the account

To keep things simple, I have made a simple page containing two forms: one for the email address, and one that simulates visiting a secret link by letting us enter a “ticket”. The view looks like this:

And then I have made this simple controller to handle the posts from the registration page:

Now, to model this workflow we need some kind of persistence on our backend, which is where sagas come into the picture. Sagas is the built-in mechanism in NServiceBus that helps in building stateful services. Stateful services are, as the word implies, services that preserve some kind of state between receiving messages.

The NServiceBus saga is a nifty way to declare what that state should contain (by letting a class implement ISagaEntity) and – given a message that the saga can handle – how to retrieve that saga (by overriding ConfigureHowToFindSaga and setting up which properties to compare).

Lets start out by specifying that NServiceBus should take care of persisting the saga entity (which will by done through Fluent NHibernate/NHibernate/SQLite under the hood)… that can easily be achieved by changing our backend’s endpoint configuration to this:

This will make NServiceBus persist ongoing sagas in an SQLite db file called “NServiceBus.Sagas.sqlite” inside the backend’s execution directory. If you omit the NHibernateSagaPersisterWithSQLiteAndAutomaticSchemaGeneration() thing, NServiceBus will store ongoing sagas by using its InMemorySagaPersister, which – surprise! – stores sagas in memory.

I had some problems with the in-memory persister however, as I could not make it correlate messages with my saga unless I correlated with interned strings only, by implementing getters on my messages like so:

I imagine this has to do with a deserializer somewhere, somehow generating strings that are not interned, although I have not verified this – I am only guessing! No biggie though, as long as the SQLite saga persister is so easy to use.

In my example the saga is initiated by the RequestRegistration message which contains only an email. When the saga receives that message, the email is stored, and a secret ticket is generated, emailed to the user, and stored in the saga data. The saga is completed when it receives a ConfirmRegistration message with the right ticket.

Let’s specify what will constitute the saga data:

The first three properties come from the ISagaEntity interface – and then come my two properties for storing the email and the ticket.

Please do remember to mark all the properties of your saga data as virtual! Otherwise, Fluent NHibernate will throw some stupid exception, saying that “Database was not configured through Database method” (wtf?) (thanks to this post for sorting that one out!). The exception is fair though, as NHibernate would complain about not being able to create proxies if there were un-interceptable properties on the data class – the error message is just weird…

To create a saga, you let a class inherit from Saga&lt;TSagaEntity&gt; where TSagaEntity would be UserRegistrationSagaData in my example… and then we implement ISagaStartedBy<TMessage> and IMessageHandler<TMessage> where the two TMessage type parameters should be filled out with RequestRegistration and ConfirmRegistration respectively. Like so:

– and then – given the two kind of messages my saga can handle – how to correlate the messages with my saga:

Even though my saga is initiated by RequestRegistration, I set up a mapping that ensures that a new saga will not be created if someone requests registration twice with the same email.

Lastly, the actual logic carried out by the saga – the two message handlers (note that MailSender and NewUserService are application services that are automagically injected becuase they’re public properties of the saga):

Note how I mark the saga as complete by calling MarkAsComplete(), thus allowing the saga data to be deleted. This could also be a nifty place to save the time of when the last registration email was sent to a particular address in order to disallow sending another registration email for the next hour or so, to avoid people spamming each other by using our web site.

Now, if I fire up my backend and request registration with the email [email protected], it looks like this:

Submitting email as a registration request
The result on the backend showing that the message was properly received

Then, if I request registration with [email protected] followed by [email protected], I can verify that a new saga is only created for [email protected]:

Submitting the same email twice results in the same ticket being sent out

And then, when I confirm a registration request, it looks like this:

Submitting one of the 'secret' tickets
The result on the backend after having submitted on of the tickets

Isn’t that great? I cannot imagine a framework with an API more elegant and terse than this.

Conclusion

That was the fifth post in my “NServiceBus for dummies who want to be smarties” series. Sixth and last post will be about our backend publishing messages whenever interesting stuff happens. This will provide a message based interface for interested parties to subscribe to.

NServiceBus for dummies who want to be smarties 4

Posted on by mookid8000 Leave a comment

NServiceBus for dummies who want to be smarties 4… fourth post, this time on how to incorporate NServiceBus as a backend in an ASP.NET MVC application.

How to create the projects

First, create a new ASP.NET MVC project. I assume you know how to use an IoC container to instantiate controllers, and that you know how to configure your container (otherwise, there are plenty of information available – e.g. here and here), so I will just show this simple snippet on how to build the bus and put it in the container:

In Application_Start:

– and then the container can be created like this:

Notice the call to the extension method CastleWindsorBuilder? It comes from the assembly NServiceBus.ObjectBuilder.CastleWindsor.dll which is included with NServiceBus – it provides the adapters necessary for NServiceBus to a) register everything it discovers (like e.g. all your implementations of IHandleMessages&lt;&gt;), and b) pull instances when needed thus supplying dependencies during object activation. This way of supporting multiple IoC containers just plain ROCKS!

At this point NServiceBus has registered itself and registered all message handlers found in the current web application’s bin directory ( Configure.WithWeb() as opposed to Configure.With()) in the supplied IoC container, and since I am pulling all my controllers from Windsor, I can jump directly to implementing my HomeController:

As you can see, I implemented two actions responding to a HTTP post, both sending a message to the backend, but DoSomethingBad throws an exception, in which case i do not want the message to actually be sent.

My view is just a trivial (Spark) view with two forms:

And then I made a simple backend with one message handler:

Now, to check if things work, I navigate to /home/index, which on my computer looks like this:

dosomething

– and then I enter the text “WHEE!” into the first text field and press “Send”, yielding the following in my backend’s console window:

fantastic

Which is great! My backend receives messages from my web application, so now I can put all kinds of heavy calculations and processesing and whatnot in there, to be computed outside of web requests.

Now, what if something bad happens during the web request, and we do not want any messages to actually be sent? Well, if I enter a text into the other text field and submit it (to the DoSomethingBad action of my HomeController), nothing happens on the backend! Why is that?

Simply because I have started a TransactionScope in my controller layer supertype, TxBaseController, ensuring that all controller actions are run inside a (possibly distributed) transaction! It looks like this:

– and since message queueing plays along nicely with the currently ongoing ambient transaction (which e.g. the incredible NHibernate ALSO does), this transaction scope will make sure that everything just works!

Conclusion

That concludes today’s “NServiceBus for dummies who want to be smarties”. This one was on how to actually put NServiceBus to use as a simple backend for an ASP.NET MVC web application. Freaking easy, right?! Next time I will show how to put sagas to use by implementing one of the workflows of a typical web application.

NServiceBus for dummies who want to be smarties 3

Posted on by mookid8000 Leave a comment

NServiceBus for dummies who want to be smarties 3… third post on how to do some more stuff with NServiceBus.

But where do the messages go?

Well, where did you tell the framework to send them?

The answer to this question is deferred until the last meaningful moment, i.e. the moment when the generic host starts up and reads its app.config.

How do I receive messages?

The answer is simple: use the MsmqTransportConfig section to tell NServiceBus where this service can be reached. If the specified queue does not exist, it will be created. E.g. like so:

How do I send messages?

This question is a bit harder to answer. First, it can be specified in the UnicastBusConfig/MessageEndpointMappings element, for each message type, or for an entire assembly at a time. E.g. like so:

In the example above, I have configured that all messages from the assembly Messages except SomeSpecificMessage will be routed to someService, whereas SomeSpecificMessage has its own destination in the queue anotherService.

Note that this section is not required if the service is a publisher, because then it will know where to send messages because other services have subscribed, and in the process they leave the name of the queue where they can be found.

Conclusion

That concludes today’s “NServiceBus for dummies who want to be smarties”. This was about where messages go. Next post will be an example with a simple infrastructure for an ASP.NET MVC backend running on NServiceBus, accepting subscriptions from other services. It’s going to be hot!

NServiceBus for dummies who want to be smarties 2

Posted on by mookid8000 2 Comments

NServiceBus for dummies who want to be smarties 2… second post, this time on how to do some more stuff with NServiceBus.

How to do stuff when the service starts

Make a class implement IWantToRunAtStartup, e.g. like so:

How to speed up delivery of messages that are transient in nature

If your message contents get stale after a short period of time, it makes no sense to use reliable messaging where messages are written to disk before the message queueing service attempts to deliver the messages to their destination. Therefore, to be true to the transient nature of the messages and speed up delivery, you can decorate some of your messages with the [Express] attribute. E.g. like so:

How to make messages expire after some amount of time

If you know your message content does not make sense after some period of time, say one hour, just decorate your message with the [TimeToBeReceived] attribute – e.g. like so:

Note that the argument to the attribute must be Parseable – unfortunately C# does not allow for calling e.g. TimeSpan.FromHours(1) in the attribute initializer.

How to stop logging all kinds of noise to the console

Logging is good, but not all the time. To disable all of NServiceBus’ logging, just make your endpoint config implement the interface IWantCustomLogging and do nothing in the Init method – e.g. like so:

Conclusion

That concludes today’s “NServiceBus for dummies who want to be smarties”. This was about how to add some more life to your service. Next post will be about the question: where do the messages go?

NServiceBus for dummies who want to be smarties 1

Posted on by mookid8000 Leave a comment

NServiceBus for dummies who want to be smarties 1… first post: on how to get started creating services with NServiceBus.

How to create messages

Creating messages with NServiceBus consists of performing the following steps:

  1. Create a new project of type “Class Library”
  2. Make the project reference NServiceBus.DLL (“Get it now”) from the binaries folder
  3. Create a class for each message, “tagging” each message class with the IMessage interface – see example below

Note that IMessage is just a marker interface, so it does not require you to implement anything.

How to create a service

Creating a service with NServiceBus consists of performing the following steps:

  1. Create a new project of type “Class Library”
  2. Make the project reference NServiceBus.dll, NServiceBus.Core.dll, log4net.dll, and NServiceBus.Host.exe
  3. Make the project reference the message DLL from above
  4. Make a class implement IConfigureThisEndpoint – see example below
  5. Make the endpoint configuration class implement either AsA_Client, AsA_Server, or AsA_Publisher

How to run that service

Make each service project run the referenced NServiceBus.Host.exe inside its bin\Debug directory – e.g. CustomerService should, as its debug action, run the external program SolutionDir\CustomerService\bin\Debug\NServiceBus.Host.exe.

NServiceBus.Host.exe, referred to as “the generic host”, is actually a Windows service, that uses reflection to pick up all the assemblies that surround it, and use them to build a service.

This is why you need to implement IConfigureThisEndpoint – it’s a marker interface that tells the generic host to look for configuration in that class.

A nifty thing to do at this point, is to set your solution to debug multiple projects when you press F5 – do this by right clicking on the top node in solution explorer, select “Set startup projects…”, and make all your service projects start up when debugging.

How to do stuff in a service

Make a class in a service that implements IHandleMessages&lt;TMessage&gt;. This requires you to implement one function, Handle. See example below:

But how does the service get access to the bus?

NServiceBus defaults to using Spring.NET for DI, so dependencies are automagically resolved – which also includes the bus! Which in turn means that the following code snippet can send messages by using the bus:

Ok, but where does the message go?

Well, that depends – for this stuff to run, we need to do a tad more configuration by supplying each service with an app.config. E.g. like so:

In this XML file, I am configuring the following things:

  • The MsmqTransportConfig section tells the generic host that this service can be reached at client, which is just the name of a private message queue, which the framework will automatically create for us
  • The UnicastBusConfig/MessageEndpointMappings section tells the framework that the message types from the assembly Messages are owned by the service behind the server endpoint

Why do I say that the messages “are owned by the service behind the server endpoint”? Why not just say that the message endpoint mappings configure the destination? Consider this table:

Operation Who performs the operation Who has IHandleMessages&lt;TMessage&gt;
bus.Send client server
bus.Publish server client

As you can see, there’s more to the endpoint mappings than just specifying the message destination – a more careful and precise description is that the endpoint mappings specify who owns the messages.

Please pay some respect and awe to the MaxRetries, ErrorQueue, and NumberOfWorkerThreads attributes – how cool is that?! It means that all exceptions that may happen inside your message handlers should not be handled by you, just let them bubble up to the framework and let it retry delivery up to 5 times until successful, or else the message will go to the error queue. That totally rids you of handling those pesky exceptions coming from your database transaction (hopefully rarely, or else your database is a bottleneck) being selected as the deadlock victim, temporarily unavailable external services unable to communicate, etc. Handling those things is truly the job of a framework.

Conclusion

That concludes today’s “NServiceBus for dummies who want to be smarties”. This was how to go about creating a simple service which is capable of receiving a message, and in the message handler it can send a message.