Orthogonality (again)

There’s one thing, that almost always makes we want to assume the foetal position and cry: developers, who are ignorant to the fact that there is a difference between application logic and application framework.

I must admit that I have only recently started being (this) conscious about this difference, so I have written a buttload of code the last few years that violates almost everything that I stand for now, which really makes me sad inside. But it’s never too late to improve.

Once I realized this and started to try to adhere to it to keep the two things separated, I started seeing things so very clearly, and then other people’s ignorance of this fact just started to gnaw and irritate me. Hence this post – I need to get this off my chest – I need to write another post in the “rant” category…

An incredibly insightful (as always) post was made by Ayende a couple of months ago: Application structure: Concepts & features. Ayende’s post explains it so well, but basically he distinguished between concepts in a system, which is stuff that requires design, and features which merely build and/or augment existing concepts. I just want to add a personal experience to the rant in the context of Ayende’s post.

Here’s the setting

I am currently on a team that develops and maintains a mortgage deed trading and administration system. Part of this system is an extensive suite of automated nightly jobs and automatic reports.

Naturally, some of the reports are run every night, e.g. summing up the numbers recorded the previous day for automated export to accounting systems, other reports are run every week/month/year, some on bank days, some relative to bank days, etc.

Some jobs make changes to the state of the system (lige e.g. updating the particulars of people for whom we receive updates from the Central Office of Civil Registration, remembering that a particular batch of transactions have been exported to the accounting system, or remembering that information on interest fees for the previous year have been reported to the National Bank etc.), and some are just (idempotent) reports and exports.

Most jobs run automatically in the night. Some jobs can also be initiated by the user of our Windows Forms frontend through a web service call. And all reports should also be accessible through the built-in reporting frontend in the Winforms app.

Here’s our current solution

All reports are run through a web service call by instantiating a ReportCommand, which is capable of getting the names of all the reports accessible to the current user. And then, given a report name, the command can get all the parameters for that report. And then, given a report name and a set of parameters, it can run the report and output a file in SpreadSheetML format. This allows our frontend to dynamically build a GUI for all reports. No GUI work is needed whenever we need to create a new report, which is great.

The majority of our nightjobs are initiated by the Windows Scheduler, which stems from an old pragmatic solution to the very first automated job we needed almost three years ago. This has not changed, so jobs are still scheduled manually through the Windows Scheduler. Our job runner is an ordinary .NET Windows .exe, which gets executed with one or more arguments. Exporting transactions to accounting could look like this:

– which would use the string to look up a class that implements ITask, e.g. ExportTransactionsTask, which exports all non-exported transactions in a .csv file to some preconfigured location.

To be able to schedule reports to run, we have made a job named “report”, which is capable of invoking the ReportCommand directly, setting the parameters of the report from the given command line arguments. Running a report could look like this:

– which runs the ReportTask, which is fed a dictionary containing {{"name", "transactions"}, {"caseNo", "10000:20000"}, {"recordDate", "today-1:today"}, {"mail", "[email protected]"}}, which in turn runs the ReportCommand with the given report name and parameters, extrapolating macros like “today” into today’s date + some more simple stuff.

If this sounds complicated to you, then yes! It is actually pretty complicated! Not because it should be, but because it’s pretty legacy and implemented in a pretty messy way. And now the need has come for the users to be able to schedule jobs and reports from the Winforms frontend. Damn! This is a good time to reflect on how I wish we had done it (and how I plan to do stuff like this in the future).

How I wish we had done it

The problem above, as I see it, is that features and concepts are mixed together in a gray matter that noone can oversee. I wish we had thought more about separating the concepts (command, nightjob, report) from the features (implementation of the different jobs and reports). I wish implementing a new report was easy like this ( #region...#endregion added as explanation):

and then all possible implementations of the Report class would be picked up by the various concepts in the system, like e.g. our report command, the report scheduler, and an ad hoc reporting task runner… and then, in a similar fashion, I want to subclass an IdempotentTask class for all tasks, that do not change anything in the system, and a TaskWithSideEffects class for all tasks that change the state of the world.

This way, implementing the logic inside of reports and tasks will be orthogonal to implementing the capabilities of the reports and tasks and their scheduling.