Querying in MongoDB works by sending a document to the server, e.g. in the following snippet I create a document with a post ID

var somePost = db.posts.findOne({"_id": ObjectId("00112233445566778899aabb")})

- which can actually be even shorter, as the find and findOne functions accept an ObjectId directly as their argument, like so:

var somePost = db.posts.findOne(ObjectId("00112233445566778899aabb"))

But how can I find a post with a specific slug? Easy, like so:

var somePost = db.posts.find({slug: "this-slug-probably-comes-from-a-url"})

But how does this perform? It’s easy to examine how queries are executed with the explain() function, like so:

> db.posts.find({slug: "this-slug-probably-comes-from-a-url"}).explain()
{
        "cursor" : "BasicCursor",
        "startKey" : {
 
        },
        "endKey" : {
 
        },
        "nscanned" : 10000,
        "n" : 1,
        "millis" : 11,
        "allPlans" : [
                {
                        "cursor" : "BasicCursor",
                        "startKey" : {
 
                        },
                        "endKey" : {
 
                        }
                }
        ]
}

- yielding some info about the execution of the query. I don’t know exactly how to interpret all this, but I think I get that "nscanned": 10000 means 10000 documents were scanned – and in a collection with 10000 documents, that’s not entirely optimal as it implies a full table scan. Now, let’s make sure that our query will execute as fast as possible by creating and index on the field (_id is always automatically indexed):

db.posts.ensureIndex({slug:1})

Now lets explain() again:

> db.posts.find({slug: "post-no-454"}).explain()
{
        "cursor" : "BtreeCursor slug_1",
        "startKey" : {
                "slug" : "post-no-454"
        },
        "endKey" : {
                "slug" : "post-no-454"
        },
        "nscanned" : 1,
        "n" : 1,
        "millis" : 0,
        "allPlans" : [
                {
                        "cursor" : "BtreeCursor slug_1",
                        "startKey" : {
                                "slug" : "post-no-454"
                        },
                        "endKey" : {
                                "slug" : "post-no-454"
                        }
                }
        ]
}

Wow! That’s what I call an improvement!

What about posts with a specific tag? First I tried the following snippet, because I learned that the special $where field could be put in a query document to evaluate a predicate server-side:

var niftyPosts = db.posts.find({$where: function() { return this.tags != null && this.tags.indexOf("nifty") != -1; }})

- and this actually works. This syntax is sort of clunky though. Luckily, MongoDB provides a nifty mechanism for arrays that automagically checks if something is contained in it. So my query can be rewritten to this:

var niftyPosts = db.posts.find({tags: 'nifty'})

Nifty!

Now, to take advantage of indexes, the special query document fields should be used. I showed $where above, but there are more – to name a few: $gt (greater than), $gte (greater than or equal), $lt (less than), $lte (less than or equal), $ne (not equal), and many more. For example, to count the number of non-nifty posts in February 2010:

> db.posts.find({
    date: {$gte: 20100201, $lt: 20100301},
    tags: {$ne: 'nifty'}
}).count()
0

Check out the manual for some great documentation on available operators.

Conclusion

Querying with MongoDB actually seems pretty cool and flexible. I like the idea that it’s possible to execute ad-hoc queries, and for most usage I think the supplied operators are adequate. The ability to supply a predicate function via $where seems really cool, but it should probably only be used in conjunction with one or more of the other operators to avoid a full table scan.

Now – if you, like me, are into OO and normalized object models – the weirdness begins….. or maybe not?! (actually, I am not sure yet )

In an OO world (and in a normalized RDB world as well), you reference stuff, thus reducing the amount of redundant information as much as possible. E.g. the names of countries should not be put in a column in your address table, each country should have a row in the countries table, and then be referenced by a countryId in the address table.

In a document-oriented world, you generally embed objects instead of referencing them. This is done for performance reasons, and because there’s no way to join stuff – which means that a stored ID/foreign key merely remains free for the client to manually use in additional queries.

When you do need to actually reference another document, use DBRef to create a reference, supplying the collection and the ID as arguments. E.g. like so:

> var author = {name: 'joe'}
> db.authors.save(author)
> author
{ "name" : "joe", "_id" : ObjectId("4b8ed200384e0000000065d8") }
> var post = {headline: 'hello there', author: new DBRef('authors', author._id)}
> db.posts.save(post)
> post
{
        "headline" : "hello there",
        "author" : {
                "$ref" : "authors",
                "$id" : ObjectId("4b8ed200384e0000000065d8")
        },
        "_id" : ObjectId("4b8ed4c8384e0000000065d9")
}

This way, the reference is represented in a consistent manner which may – or may not – be picked up by the driver you are using. The C# driver can create DBRefs and follow them, but you don’t get to join stuff – you still need an extra query.

Embedding objects may seem a little clunky at first, but actually this plays nicely with some common OO concepts – take aggregation, for example: a blog post has an array of comments, each of which makes no sense without an aggregating post – i.e. comments live and die with their post. That’s an obvious sign that comments should be embedded in the post. So, instead of:

posts:
{'_id': ObjectId('11223344556677889900aabb'), headline: 'A post'}
 
comments:
{'_id': ObjectId('bbaa00112233554477669988'), text: 'hello there', post: { "$ref": "posts", "$id": ObjectId('11223344556677889900aabb') }}
{'_id': ObjectId('881122335544776699aa00bb'), text: 'hello again', post: { "$ref": "posts", "$id": ObjectId('11223344556677889900aabb') }}

- you should do this:

{
    '_id': ObjectId('11223344556677889900aabb'), 
    headline: 'A post',
    comments: [
        {text: 'hello there'},
        {text: 'hello again'}
    ]
}

Actually this makes me think about the concept of an aggregate root in DDD: an aggregate root “owns” the data beneath it, and is responsible for maintaining its own integrity. If you were to delete an aggregate root, all the data beneath it would dissappear.

This also fits kind of nicely with the fact that there’s no database transactions in MongoDB – i.e. there’s no way to issue multiple statements and have them rolled back in case of an error – there’s only documents, and either a document gets inserted/updated/deleted, or it doesn’t. So obviously, the document is the unit of atomicity, which fits (sort of nicely) with the aggregate root and its responsibility of keeping itself internally consistent.

Conclusion

The observations stated here pretty much make a document an aggregate root in the DDD sense – especially since only documents get an _id. There’s no obvious way to reference a particular comment inside the second post shown above.

If MongoDB’s performance is up to the task, data should probably be aggregated as much as possible into large documents. MongoDB’s limit is 4 MB per document, but I am unsure of how large documents should be before you should consider splitting them.

Maybe I am thinking too much about these things? Maybe I should just try and build something and see where my document modeling goes? Suggestions and comments are welcome

This post is a write-as-I-go summary of the information I have gathered from the following places:

• MongoDB Manual
• MongoDB C# Language Center
• K. Scott Allen: Experimenting with MongoDB from C#

More posts may follow….

Getting MongoDB

Piece of cake! Download MongoDB from the download center and shove the binaries away somewhere on your machine. Default is for MongoDB to store its data in /data/db which translates to c:\data\db if you are using Windows – go ahead and create this directory. The MongoDB daemon can be started by running mongod.exe, which will accept connections on localhost:27017.

It will probably look something like the screenshot shown below.

An alternative data path can be specified on the command line, e.g. like so: mongod --dbpath c:\somewhere\else.

Accessing it with JavaScript

Run mongo.exe to start the Mongo Shell. It will probably look something like this:

In the MongoDB prompt, you can use JavaScript to access the db. Here’s a sample session of some commands I have found useful:

// lists the dbs in this mongo
> show dbs   
admin
local
>
> // change database context to some db ("myblog" - will automagically be created)
> use myblog   
switched to db myblog
>
> show collections
system.indexes
>
> // save a couple of documents in a collection named "posts"
> // (collection will be automagically created as well...)
> db.posts.save({
    headline: 'Notes to self about MongoDB', 
    slug: 'notes-to-self-about-mongodb', 
    tags: ['mongodb', 'nosql', 'nifty', 'c#']
 })
> db.posts.save({
    headline: 'Someday I want to check out CouchDB as well', 
    slug: 'someday-i-want-to-check-out-couchdb-as-well', 
    tags: ['couchdb', 'nosql', 'nifty', 'c#']
 })
>
> // show documents in a collection (returns a cursor, which will be iterated for the first 10 or 20 results - next pages can be retrieved with the 'it' command)
> db.posts.find()
{ "_id" : ObjectId("4b8e4281781b000000005cfc"), "headline" : "Notes to self about MongoDB", "slug" : "notes-to-self-about-mongodb", "tags" : [ "mongodb", "nosql", "nifty", "c#" ] }
{ "_id" : ObjectId("4b8e42cc781b000000005cfd"), "headline" : "Someday I want to check out CouchDB as well", "slug" : "someday-i-want-to-check-out-couchdb-as-well", "tags" : [ "couchdb", "nosql", "nifty", "c#" ] }

Now, I have successfully added two documents representing blog posts in a collection named posts. As you can see, MongoDB assigns some funky IDs to the documents.

> // lets get the first post ('find' and 'findOne' accept a query document as their first parameter)
> db.posts.findOne({'_id': ObjectId('4b8e4281781b000000005cfc')})
{
        "_id" : ObjectId("4b8e4281781b000000005cfc"),
        "headline" : "Notes to self about MongoDB",
        "slug" : "notes-to-self-about-mongodb",
        "tags" : [
                "mongodb",
                "nosql",
                "nifty",
                "c#"
        ]
}
>
> // now let's find all IDs ('find' and 'findOne' accept as their second parameter a document
> // specifying which fields to return)
> db.posts.find({}, {'_id': true})
{ "_id" : ObjectId("4b8e4281781b000000005cfc") }
{ "_id" : ObjectId("4b8e42cc781b000000005cfd") }
{ "_id" : ObjectId("4b8e4595781b000000005cfe") }
>

That was a brief demonstration of the JavaScript API in the Mongo Shell. Now, let’s do this from C#.

Getting started with mongodb-csharp

Now, go to mongodb-csharp dowload section at GitHub and get a debug build of the driver. Create a C# project and reference the MongoDB.Driver assembly.

On my machine, punching in the following actually works:

[Test]
public void CanAddPost()
{
  using(var mongo = new Mongo())
  {
    mongo.Connect();
 
    var db = mongo["myblog"];
    var posts = db["posts"];
 
    posts.Insert(new Document
                   {
                     {"headline", "Post added from C#"},
                     {"slug", "post-added-from-csharp"},
                     {
                       "tags", new[]
                                 {
                                   "c#",
                                   "nifty"
                                 }
                       }
                   });
  }
}

Now, I can verify that the document is actually in there by going back to the console and doing this:

> db.posts.find()
{ "_id" : ObjectId("4b8e4281781b000000005cfc"), "headline" : "Notes to self about MongoDB", "slug" : "notes-to-self-about-mongodb", "tags" : [ "mongodb", "nosql", "nifty", "c#" ] }
{ "_id" : ObjectId("4b8e42cc781b000000005cfd"), "headline" : "Someday I want to check out CouchDB as well", "slug" : "someday-i-want-to-check-out-couchdb-as-well", "tags" : [ "couchdb", "nosql", "nifty", "c#" ] }
{ "_id" : ObjectId("4b8e4b72091abb14e4000001"), "headline" : "Post added from C#", "slug" : "post-added-from-csharp", "tags" : [ "c#", "nifty" ] }
>

Nifty! Now lets show the posts from C#. On my machine the following snippet displays the headlines of all posts:

[Test]
public void CanShowPosts()
{
  using(var mongo = new Mongo())
  {
    mongo.Connect();
 
    var db = mongo["myblog"];
    var posts = db["posts"];
 
    Console.WriteLine("Posts");
    foreach(var post in posts.FindAll().Documents)
    {
      Console.WriteLine("    {0}", post["headline"]);
    }
  }
}

- which is documented in the following screenshot:

Random nuggets of information

Document IDs

All MongoDB documents must have an ID in the _id field, either assigned by you (any object can be used), or automatically by MongoDB. IDs generated by MongoDB are virtually globally unique, as they consist of the following: 4 bytes of timestamp, 3 bytes of machine identification, 2 bytes of process identification, 3 bytes of something that gets incremented.

As a nifty consequence, the time of creation can be extracted from auto-generated IDs.

The ID type used by MongoDB can be created with ObjectId('00112233445566778899aabb') (where the input must be a string representing 12 bytes in HEX).

How are documents stored?

I you have not yet figured it out, documents are serialized to JSON – with the minor modification that it’s a BINARY version of JSON, hence it’s called BSON.

String encoding

UTF-8. No worries.

What about references?

I will research this and do a separate post on the subject. As MongoDB is non-relational, a “join” is – in principle – an unknown concept. There’s a mechanism, however, that allows for consistent representation of foreign keys that may/may not give you some extra functionality (depending on the driver you are using).

What about querying?

I will research this as well, posting as I go.

OR/M? (or OD/M?)

It is not yet clear to me how to handle Object-Document Mapping. Will require some research as well. As an OO dude, I am especially interested in finding out what a schema-less persistance mechanism will do to my design.

What else?

More topics include applying indices, deleting/updating, atomicity, and more. Implies additional blog posts.

Conclusion

My first impression of MongoDB is really good. It’s extremely easy to get going, and the few error messages I have received were easy to understand.

I am especially in awe with how little friction I encountered – mostly because of the schema-less nature, but also because everything just worked right away.

Most of these limitations, however, like the need for public/internal/protected members to be virtual, and the requirement for a default constructor to exist with at least protected accessibility, are not that hard to adhere to and usually don’t interfere with what you would do if there were no rules at all.

One of the limitations, however, can be pretty significant – Ayende describes the problem here, using the term “ghost objects”.

But, as I am about to show, this significance only arises if you follow a certain style of coding, which you should usually avoid!

Short explanation of the problem

When NHibernate lazy-loads an entity from the db (i.e. when you call session.Load<TEntity>(id) or when an entity in your session references something through a lazy-loaded association), it does so by providing an instance of a runtime-generated type, which acts as a proxy.

The first time you access something on the proxy, it gets “hydrated”, which is just a fancy way of saying that the data will be loaded from the database.

This would be fine and dandy, if it weren’t for the fact that the proxy is a runtime-generated subclass of your entity, which – in cases where inheritance is involved – will be a sibling to the other derived classes. Consider the simple inheritance hierarchy on the sketch to the right which in code could be something like so:

public abstract class LegalEntity
{
    public virtual Guid Id { get; set; }
}
 
public class Person : LegalEntity
{
    public virtual string FirstNames { get; set; }
    public virtual string LastName { get; set; }
}
 
public class Company : LegalEntity
{
    public virtual string CompanyName { get; set; }
}

- and then NHibernate will generate something along the lines of this (fake ) class signature:

public class LegalEntityProxy1234AndSomeMoreStuff : LegalEntity
{
   // ... secret stuff to access db in here
}

See the problem? Here’s the problem:

var legalEntity = session.Load<LegalEntity>(someKnownId);
Assert.IsTrue(legalEntity is Person
              || legalEntity is Company);  //< AssertionException! will never be Person or Company

This means that this kind of runtime type checking will fail in those circumstances where the entity is a lazy-loaded reference of the supertype, and the following will FAIL:

var legalEntity = session.Load<LegalEntity>(someKnownId);
if (legalEntity is Person)
{
    var person = (Person) legalEntity;
    return person.FirstNames + " " + person.LastName;
}
else
{
    // we know it's a company then, right? WRONG!
    var company = (Company) legalEntity;  //< InvalidCastException!
    return company.CompanyName;
}

One possible solution

The other day, Ayende blogged about a recent addition to NHibernate, namely lazy-loaded properties. This allows an entity to be partially hydrated, intercepting calls to certain properties to lazy-load the relevant fields on demand.

This feature is great when storing LOBs alongside the other fields on an entity, but it also laid the ground for his most recent addition, which is the ability to lazy-load an association by setting lazy="no-proxy" on it.

This way, NHibernate will not build a proxy, but instead it will intercept the property getter and load the entity at that point in time, thus being able to return the exact (sub)type of the loaded entity.

Now this seems to solve our problems, but let’s zoom out a bit … why did we have a problem in the first place? Our problem was actually that we failed to write object-oriented code, but instead we wrote a brittle piece of code that would fail at runtime whenever someone added a new subtype, thus violating the Liskov substitution principle. Moreover it just feels wrong to implement business logic that reflects on types!

What to do then?

Well, how about making your code polymorphic? The logic above could be easily rewritten as:

public abstract class LegalEntity
{
    public virtual Guid Id { get; set; }
 
    public abstract string Name { get; }
}
 
public class Person : LegalEntity
{
    public virtual string FirstNames { get; set; }
    public virtual string LastName { get; set; }
 
    public override string Name
    {
        get { return FirstNames + " " + LastName; }
    }
}
 
public class Company : LegalEntity
{
    public virtual string CompanyName { get; set; }
 
    public override string Name
    {
        get { return CompanyName; }
    }
}

- which moves the logic of yielding name as a oneliner into the class hierarchy, allowing us to always get a name from a LegalEntity.

What if I really really need a concrete instance?

Then you should use the nifty visitor pattern to extract what you need. In the example above, I would need to add the following additions:

public interface ILegalEntityVisitor
{
    void Visit(Person person);
    void Visit(Company company);
}
 
public abstract class LegalEntity
{
    // ...
 
    public abstract void Accept(ILegalEntityVisitor visitor);
}
 
public class Person : LegalEntity
{
    // ...
 
    public override void Accept(ILegalEntityVisitor visitor)
    {
        visitor.Visit(this);
    }
}
 
public class Company : LegalEntity
{
    // ...
 
    public override void Accept(ILegalEntityVisitor visitor)
    {
        visitor.Visit(this);
    }
}

This way, we’re taking advantage of the fact the each subclass knows its own concrete instance, thus allowing it to pass itself to the visitor we passed in.

This is the preferred solution when the logic you’re writing doesn’t belong inside the actual entitiy class, like e.g. when you want to convert the entity to an editable view object, because this will make your code break at compile time if someone adds a new specialization, thus requiring each piece of logic to handle that specialization as well.

Oh, and if you’re a Java guy, you might be missing the ability to create an inline anonymous visitor within the scope of the current method, but that can be easily emulated by a generic visitor, like so:

public class LegalEntityVisitor : ILegalEntityVisitor
{
    Action<Person> handlePerson;
    Action<Company> handleCompany;
 
    public LegalEntityVisitor(Action<Person> handlePerson, Action<Company> handleCompany)
    {
        this.handlePerson = handlePerson;
        this.handleCompany = handleCompany;
    }
 
    public void Visit(Person person)
    {
        handlePerson(person);
    }
 
    public void Visit(Company company)
    {
        handleCompany(company);
    }
}

- which would allow you to write inline typesafe code like this:

double risk = CalculateInitialRisk();
 
// multiply some number depending on type of legal entity
legalEntity.Visit(new LegalEntityVisitor(person => risk *= GetRiskExperienceForPeople(),
                                         company => risk *= GetRiskExperienceForCompany(company));

Only thing missing now is the ability to return a value in one line depending on the subclass. Well, the generic visitor can be used for that as well by adding the following:

 
public class LegalEntityVisitor : ILegalEntityVisitor
{
    // ...
 
    public static TResult Func<TResult>(LegalEntity legalEntity, 
                                        Func<Person, TResult> handlePerson, 
                                        Func<Company, TResult> handleCompany)
    {
        TResult result = default(TResult);
 
        legalEntity.Visit(new LegalEntityVisitor(p => result = handlePerson(p),
                                                 c => result = handleCompany(c)));
 
        return result;
    }
}

- allowing you to write code like this:

 
public string GetReportTypeCodeFor(LegalEntity legalEntity)
{
    return LegalEntityVisitor.Func(legalEntity, p => "P00000", c => "C" + GetReportingCode(c));
}

- and still have the benefit of compile-time safety that all specializations have been handled.

When to reflect on types?

IMO you should only reflect on types in business logic when it’s a shortcut that doesn’t break the semantics of your code. What do I mean by that? Well, e.g. the implementation of the extension method System.Linq.Enumerable.Count() looks something like this:

public int Count<T>(this IEnumerale<T> items)
{
    if (items is ICollection<T>)
    {
        return ((ICollection<T>)items).Count;
    }
 
    var count = 0;
    // iterate and count manually
 
    return count;
}

This way, providing the number of items is accelerated for certain implementations of IEnumerable<T> because the information is already there, and for other types there’s no way to avoid manually counting.

Conclusion

I don’t think I will be using the new lazy="no-proxy" feature, because if I need it, I think it is a sign that my design has a bad smell to it, and I should either go for polymorphism or using a visitor.

It’s amazing how the CSS-selector-based approach has proven its effectiveness and durability, and combined with the incredible wealth of extensions, choosing jQuery is simply a no-brainer.

By the way, jQuery 1.4 has been released. Check out this post for a walkthrough of some of the new features.

Inspired by the aforementioned problem, and an extended train ride home (thank you, Danish National Railways!!), I decided to see what a C# (v. 3) version could look like:

namespace NewsReader
{
  using System;
  using System.IO;
  using System.Linq;
  using System.Text.RegularExpressions;
 
  class Program
  {
    static void Main()
    {
      const string dir = @"c:\temp\20_newsgroups";
      var stopwatch = System.Diagnostics.Stopwatch.StartNew();
      var regex = new Regex(@"\w+", RegexOptions.Compiled);
 
      var list = (from filename in Directory.GetFiles(dir, "*.*", SearchOption.AllDirectories)
                  from match in regex.Matches(File.ReadAllText(filename).ToLower()).Cast<Match>()
                  let word = match.Value
                  group word by word
                  into aggregate
                    select new
                             {
                               Word = aggregate.Key,
                               Count = aggregate.Count()  ,
                               Text = string.Format("{0}\t{1}", aggregate.Key, aggregate.Count())
                             })
        .ToList();
 
      File.WriteAllLines(@"c:\temp\words-ordered-by-count.txt", list.OrderBy(c => c.Count).Select(c => c.Text).ToArray());
      File.WriteAllLines(@"c:\temp\words-ordered-by-word.txt", list.OrderBy(c => c.Word).Select(c => c.Text).ToArray());
 
      Console.WriteLine("Elapsed: {0:0.0} seconds", stopwatch.Elapsed.TotalSeconds);
    }
  }
}

Weighing in at 35 lines and executing in 10.2 seconds (on my Intel Core 2 laptop with 4 GB RAM), I think this is a pretty clear and performant alternative to the other languages mentioned.

It consists of an online activity inventory and a simple timer application that is pretty nifty when run out-of-browser. It will of course be even niftier when it gets fully ported to Silverlight 4 when the timer gets to run in chrome-less out-of-browser mode.

If you try out the application, and you have suggestions or ideas, we will be very grateful if you submit them to Tomatoday’s forum at Uservoice.

It goes like this (from www.pomodorotechnique.com):

1. Choose a task to be accomplished
2. Set the Pomodoro to 25 minutes (the Pomodoro is the timer)
3. Work on the task until the Pomodoro rings, then put a check on your sheet of paper
4. Take a short break (5 minutes is OK)
5. Every 4 Pomodoros take a longer break

- where tasks are chosen from your “todo today”, which you assemble in the morning by picking tasks from your “activity inventory”. There’s a few more tricks in it, e.g. a form of notation that fits the process well and how to track disturbances.

Having done this for little more than one month, I think I can safely say that it is almost guaranteed to either

• make you more productive
• make you very conscious about why you’re not that productive

Practitioners of the technique usually prefer to use a real egg timer (in the shape of a tomato of course), because of the tactile feedback you get from actually manipulating a physical object – but as I am sitting in an open office with 6 other developers, and some of them are doing pomodoros as well, we are using the next best thing: a Pomodoro timer app.

Right now I am using the Focus Booster app, because it’s pretty and doesn’t take up that much space on the screen. One of my colleagues, Troels Richter, is currently working on a more complete Pomodoro app (in Silverlight) that will help in all the aspects of the Pomodoro Technique.

One could image all kinds of cool integration scenarios where interested parties somehow receive events from our system regarding stuff that may be interesting to them. Let’s imagine that our user registration from the fifth post is an extremely interesting event to the sales department and the Über Boss of our enterprise.

To accomodate this, I create a separate assembly meant for holding messages that are not private to our application. This way, I can easily distinguish between messages that I can change whenever I feel like it, and messages that I must put more care and effort into crafting, because they will most likely live forever when a dozen applications in our enterprise are suddenly depending on them.

This means that my saga from the fifth post will handle an email confirmation like this:

    public void Handle(ConfirmRegistration message)
    {
      Console.WriteLine("Confirming email {0}", Data.Email);
 
      NewUserService.CreateNewUserAccount(Data.Email);
 
      MailSender.Send(Data.Email,
                      "Your registration request",
                      "Your email has been confirmed, and your user account has been created");
 
      Bus.Publish(new SomeoneActuallyRegistered {Email = Data.Email});
 
      MarkAsComplete();
    }

As you can see, I Bus.Publish(...) a message saying that someone with a particular email has registered.

To be able to publish stuff, some kind of subscription storage must be configure for the endpoint. Therefore, I change my backend’s EndpointConfig to be configured with this:

      Configure.With()
        .CastleWindsorBuilder(container)
        .Sagas()
        .NHibernateSagaPersisterWithSQLiteAndAutomaticSchemaGeneration()
        .MsmqSubscriptionStorage()
        .XmlSerializer();

- which means my backend will be storing subscriptions in a message queue.

If you are somehow unclear on the difference between sending and publishing, I can tell you that the conceptual difference is this: sending is like saying “do this stuff”, whereas publishing is like saying “this stuff happened” – see the difference in tense?

Functionally, the difference is that sending will put a message in the queue specified as recipient for that particular message whereafter one recipient will consume that message off its input queue (possibly competing with others), whereas publishing will put a message in the input queue of each one of whoever subscribed to that particular message (possibly many – possibly zero).

Now, to simulate the sales department and the über boss, I create two new projects containing an app.config that looks like this:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <configSections>
    <section name="MsmqTransportConfig" type="NServiceBus.Config.MsmqTransportConfig, NServiceBus.Core" />
    <section name="UnicastBusConfig" type="NServiceBus.Config.UnicastBusConfig, NServiceBus.Core" />
  </configSections>
 
  <MsmqTransportConfig InputQueue="salesDept"
                       ErrorQueue="error"
                       NumberOfWorkerThreads="1"
                       MaxRetries="5" />
 
  <UnicastBusConfig>
    <MessageEndpointMappings>
      <add Messages="PublicMessages" Endpoint="backend" />
    </MessageEndpointMappings>
  </UnicastBusConfig>
</configuration>

- and similarly for the über boss.

Note how a subscriber must have a message endpoint mapping telling where messages will be published – in the example above, my mapping specifies that all messages from my PublicMessages assembly can be subscribe to on the endpoint named “backend”. And obviously, a subscriber must have an input queue, through which subscribed messages will be received.

The EndpointConfig can look like this:

namespace SalesDept
{
  public class EndpointConfig : 
    IConfigureThisEndpoint, 
    AsA_Server, 
    IWantCustomLogging, 
    IWantToRunAtStartup
  {
    public IBus Bus { get; set; }
 
    public void Init()
    {
    }
 
    public void Run()
    {
      Bus.Subscribe<SomeoneActuallyRegistered>();
 
      Console.WriteLine("Sales department running...");
    }
 
    public void Stop()
    {
    }
  }
}

- and similarly for the über boss as well.

Note how the service subscribes to the messages that it wants to receive by calling Bus.Subscribe<...> in the Run() method. What happens, is that NServiceBus somehow communicates to whoever is configured as the publisher of messages of type SomeoneActuallyRegistered, that messages should be sent to this service’s input queue.

And then, I implement a message hander in each service… first, the sales department:

namespace SalesDept
{
  public class SendInformationAboutProducts : IMessageHandler<SomeoneActuallyRegistered>
  {
    public void Handle(SomeoneActuallyRegistered message)
    {
      Console.WriteLine("Sending stupid emails to {0}", message.Email);
    }
  }
}

and then, the über boss:

namespace ÜberBoss
{
  public class NoteEmailInLittleBlackBookOfEvilSchemes : IMessageHandler<SomeoneActuallyRegistered>
  {
    public void Handle(SomeoneActuallyRegistered message)
    {
      Console.WriteLine("Noting email {0}", message.Email);
    }
  }
}

To make this run, I go to “Set StartUp projects…” of my solution and make all my services start up when I press F5.

Now, we can check things out by pressing F5, thus running the backend, the sales department, and the über boss at once. On my machine, it looks like this:

Now it doesn’t take too much imagination to come up with all kinds of cool solutions to integration scenarios.

Conclusion

That concludes the sixth and last post in my “NServiceBus for dummies who want to be smarties” series. I hope the series can provide some information on how to get started using NServiceBus, and perhaps fill out some of the gaps that comes from documentation in the form of sparsely available blog posts from different sources.

One of the typical workflows in web applications is when someone signs up for an account, but the web site wants to check if the email address is valid. Let’s look at an example that goes like this:

1. User enters email address as a signup request
2. Web application sends email with a “secret” confirmation link
3. User visits the secret link, thereby activating the account

To keep things simple, I have made a simple page containing two forms: one for the email address, and one that simulates visiting a secret link by letting us enter a “ticket”. The view looks like this:

<p>
  Enter your email address to begin registration
</p>
 
<form method="post" action="!{Url.Action("BeginRegistration")}">
  Email: <input name="email" type="text" />
  <input type="submit" value="Begin registration" />
</form>
 
<p>
  - or enter your ticket to confirm your email
</p>
 
<form method="post" action="!{Url.Action("ConfirmRegistration")}">
  Ticket: <input name="ticket" type="text" />
  <input type="submit" value="Confirm registration" />
</form>

And then I have made this simple controller to handle the posts from the registration page:

  public class RegistrationController : TxBaseController
  {
    readonly IBus bus;
 
    public RegistrationController(IBus bus)
    {
      this.bus = bus;
    }
 
    public ViewResult Index()
    {
      return View();
    }
 
    public RedirectToRouteResult BeginRegistration(string email)
    {
      bus.Send(new RequestRegistration {Email = email});
 
      return RedirectToAction("Index");
    }
 
    public RedirectToRouteResult ConfirmRegistration(int ticket)
    {
      bus.Send(new ConfirmRegistration {Ticket = ticket});
 
      return RedirectToAction("Index");
    }
  }

Now, to model this workflow we need some kind of persistence on our backend, which is where sagas come into the picture. Sagas is the built-in mechanism in NServiceBus that helps in building stateful services. Stateful services are, as the word implies, services that preserve some kind of state between receiving messages.

The NServiceBus saga is a nifty way to declare what that state should contain (by letting a class implement ISagaEntity) and – given a message that the saga can handle – how to retrieve that saga (by overriding ConfigureHowToFindSaga and setting up which properties to compare).

Lets start out by specifying that NServiceBus should take care of persisting the saga entity (which will by done through Fluent NHibernate/NHibernate/SQLite under the hood)… that can easily be achieved by changing our backend’s endpoint configuration to this:

      Configure.With()
        .CastleWindsorBuilder(container)
        .Sagas()
        .NHibernateSagaPersisterWithSQLiteAndAutomaticSchemaGeneration()
        .XmlSerializer();

This will make NServiceBus persist ongoing sagas in an SQLite db file called “NServiceBus.Sagas.sqlite” inside the backend’s execution directory. If you omit the NHibernateSagaPersisterWithSQLiteAndAutomaticSchemaGeneration() thing, NServiceBus will store ongoing sagas by using its InMemorySagaPersister, which – surprise! – stores sagas in memory.

I had some problems with the in-memory persister however, as I could not make it correlate messages with my saga unless I correlated with interned strings only, by implementing getters on my messages like so:

public string Email
{ 
    get { return string.Intern(email); }
    set { email = value; }
}

I imagine this has to do with a deserializer somewhere, somehow generating strings that are not interned, although I have not verified this – I am only guessing! No biggie though, as long as the SQLite saga persister is so easy to use.

In my example the saga is initiated by the RequestRegistration message which contains only an email. When the saga receives that message, the email is stored, and a secret ticket is generated, emailed to the user, and stored in the saga data. The saga is completed when it receives a ConfirmRegistration message with the right ticket.

Let’s specify what will constitute the saga data:

  public class UserRegistrationSagaData : ISagaEntity
  {
    public virtual Guid Id { get; set; }
    public virtual string Originator { get; set; }
    public virtual string OriginalMessageId { get; set; }
 
    public virtual string Email { get; set; }
    public virtual int Ticket { get; set; }
  }

The first three properties come from the ISagaEntity interface – and then come my two properties for storing the email and the ticket.

Please do remember to mark all the properties of your saga data as virtual! Otherwise, Fluent NHibernate will throw some stupid exception, saying that “Database was not configured through Database method” (wtf?) (thanks to this post for sorting that one out!). The exception is fair though, as NHibernate would complain about not being able to create proxies if there were un-interceptable properties on the data class – the error message is just weird…

To create a saga, you let a class inherit from Saga<TSagaEntity> where TSagaEntity would be UserRegistrationSagaData in my example… and then we implement ISagaStartedBy and IMessageHandler where the two TMessage type parameters should be filled out with RequestRegistration and ConfirmRegistration respectively. Like so:

  public class UserRegistrationSaga : Saga<UserRegistrationSagaData>,
    ISagaStartedBy<RequestRegistration>,
    IMessageHandler<ConfirmRegistration>
  {
     //...
  }

- and then – given the two kind of messages my saga can handle – how to correlate the messages with my saga:

    public override void ConfigureHowToFindSaga()
    {
      ConfigureMapping<RequestRegistration>(saga => saga.Email, message => message.Email);
      ConfigureMapping<ConfirmRegistration>(saga => saga.Ticket, message => message.Ticket);
    }

Even though my saga is initiated by RequestRegistration, I set up a mapping that ensures that a new saga will not be created if someone requests registration twice with the same email.

Lastly, the actual logic carried out by the saga – the two message handlers (note that MailSender and NewUserService are application services that are automagically injected becuase they’re public properties of the saga):

    public void Handle(RequestRegistration message)
    {
      // generate new ticket if it has not been generated
      if (Data.Ticket == 0)
      {
        Data.Ticket = NewUserService.CreateTicket();
      }
 
      Data.Email = message.Email;
 
      MailSender.Send(message.Email,
                      "Your registration request",
                      "Please go to /registration/confirm and enter the following ticket: " + Data.Ticket);
 
      Console.WriteLine("New registration request for email {0} - ticket is {1}", Data.Email, Data.Ticket);
    }
 
    public void Handle(ConfirmRegistration message)
    {
      Console.WriteLine("Confirming email {0}", Data.Email);
 
      NewUserService.CreateNewUserAccount(Data.Email);
 
      MailSender.Send(Data.Email,
                      "Your registration request",
                      "Your email has been confirmed, and your user account has been created");
 
      // tell NServiceBus that this saga can be cleaned up afterwards
      MarkAsComplete();
    }

Note how I mark the saga as complete by calling MarkAsComplete(), thus allowing the saga data to be deleted. This could also be a nifty place to save the time of when the last registration email was sent to a particular address in order to disallow sending another registration email for the next hour or so, to avoid people spamming each other by using our web site.

Now, if I fire up my backend and request registration with the email omg@wtf.com, it looks like this:

Then, if I request registration with laterz@hax0rz.com followed by omg@wtf.com, I can verify that a new saga is only created for laterz@hax0rz.com:

And then, when I confirm a registration request, it looks like this:

Isn’t that great? I cannot imagine a framework with an API more elegant and terse than this.

Conclusion

That was the fifth post in my “NServiceBus for dummies who want to be smarties” series. Sixth and last post will be about our backend publishing messages whenever interesting stuff happens. This will provide a message based interface for interested parties to subscribe to.