Archive for Development category

Designing a great API

Several years ago I worked on a payroll package developing a core engine that required an API to let third parties write calculations, validations and security gates that would execute as part of it’s regular operation.

We were a small team and I had many conversations with another developer tasked with building a payroll using the API I would provide. Some methods here, classes there, the odd helper function and I had an API and then we had a mini payroll running.

Then he showed me the code he had written and that smug grin dropped off my face. It was awful.

Perhaps this other developer wasn’t as great as I’d thought? Looking at the code though made me realise he had done the best anyone could with a terrible API. I’d exposed parts of this core payroll engine with hooks when it needed a decision. Its job was to run the payroll – a very complex task that involved storage, translation, time periods, users and companies. That complexity and context had leaked out.

Unfortunately it’s not a unique story – many API’s are terrible to use. They’re concerned with their own terminology, limitations and quirks because they are exposed sections of an underlying system developed by those responsible for the underlying system.

If you want others to have a good experience with your product you have to put yourself in their shoes. Whether it’s a UI or an API makes no difference.

You are not the user

That’s the real difference between writing the classes that form your regular implementation and those that make up your public API.

We had time to fix our payroll API. Instead of refining and polishing here and there we took the 20 or so snippets developed for the mini payroll and pruned, cleaned and polished until they looked beautiful. They scanned well and made sense to payroll developers unfamiliar with our package. When a third developer familiar with payrolls but unfamiliar with out package developed the necessary code for a fully-functional jurisdiction in record time with minimal assistance we knew we had hit our goal.

Sure implementing that new API was hard work. Instead of simple methods sticking out of the engine we had a facade over our engine but it was justified. They were two different systems for two different types of user with distinct ideas about what the system was and how it was going to be used.

Code First

Many years later I found myself on a small team of 3 people tasked with putting a brand new API on top of Entity Framework for configuring models with code the .NET world would come to know as Code First. I was determined to use my experience and avoid another complex API surface littered with terminology and leaky abstractions. Parts of EF already suffered from that problem.

So for the first few weeks of that project we didn’t write any of the code that would in fact become Code First.

Instead we decided who our user was – in this case a C# developer who likes writing code, knows LINQ and some database concepts but doesn’t know Entity Framework as people who did were already using Model First or Database First.

Then we wrote tiny sample apps and tried to find simpler and simpler ways to describe them in code. We’d often start on a whiteboard with a scenario and write the complete mapping. We’d then try and find conventions that would remove the need for most of it and then try to write succinct code to configure the rest. As the newest guy to the team I’d fight to keep EF terms away from the main API surface in order to reduce that barrier to entry and help drive adoption.

Finally we’d hit the computer and develop stub classes and methods to make samples compile and let us try the IntelliSense. This isn’t always necessary but if you want to develop a fluent API or provide lots of type-safety such as Code First’s relationship mapping it’s highly recommended.

We’d then revisit the samples later and see if they could be read as easily as they were written and figure out what problems people were likely to run into and whether we could solve them without too much noise. Sometimes this meant having more than one way to do things such as chaining the fluent methods or allowing a bunch of properties to be set (solved with an extension method class providing the fluent API) and how users could manage larger models (solved by subclassing EntityConfiguration<T> – now EntityTypeConfiguration<T> sigh – and allowing redundant specification for things like relationships that span more than one class).

We finally ended up with succinct code like this with IntelliSense guiding you along the way and preventing you from even being able to specify invalid combinations. The HasMany prompts the properties on Customer and it won’t show you WithRequired unless it is valid. In the case of Required to Required it will ensure that the WithRequired specified which end is principle and dependent. In short it guides you through the process and results in highly readable code.

Entity<Customer>().HasMany(c => c.Orders).WithRequired(o => o.Customer).WillCascadeOnDelete();

This process took a little longer but given the amount of use the API will get that time will be saved by users countless times over.

Code First went down incredibly well with both the target audience and existing EF users and inspired the simpler DbContext interface that became the recommended way of accessing EF.

I think it’s one of the nicest API’s to come out of Microsoft and .NET.

[)amien

PS. Martin Fowler has some great guidance in his book Domain Specific Languages.

Behind the scenes at xbox.com – RSS enabling web marketplace

A number of people were requesting additional RSS feeds for the xbox.com web marketplace. (We had just one that included all new arrivals)

Looking across our site as the various lists of products we display today the significant views are:

  • Browse games by department
  • Search results
  • Promotions (e.g. Deal of the week)
  • Game detail (shows downloads available beneath it)
  • Avatar item browse

These views also have sorting options and a set of filters available for things like product type, game genre, content rating etc.

So we had a couple of options:

  1. Write controller actions that expose the results of specific queries as RSS
  2. Introduce a mechanism whereby any of our product result pages can render as RSS including any user-defined filtering

Our web marketplace is written in ASP.NET MVC (like most of xbox.com) so while option 1 sounds simpler MVC really helps us make option 2 more attractive by way of a useful feature called ActionFilters that let us jump in and reshape the way existing actions behave.

ActionFilters

ActionFilters can be applied to either to an individual action method on a controller or to the controller class itself which applies it to all the actions on that controller. They provide hooks into the processing pipeline where you can jump in and perform additional processing.

The most interesting events are:

  • OnActionExecuting
  • OnActionExecuted
  • OnResultExecuting
  • OnResultExecuted
We’re going to hook in to the OnActionExecuted step – this is because we always want to run after the code in the controller action has executed but before the ActionResult has done it’s work – i.e. before page or RSS rendering.

Writing our ActionFilter

The first thing we want to do is identify that a request wants the RSS version. One way is to read the accepts header and switch when it requests mime/type but this can be a little trickier to test,  another is to append a query parameter on the url which is very easy to test.

Once we’ve identified the incoming request should be for RSS we need to identify the data we want to turn into RSS and repurpose it.

All the views we identified at the start of this post share a common rendering mechanism and each view model subclasses from one of our base models. For simplicity though we’ll imagine an interface that just exposes an IEnumerable<Product> property.

public class RssEnabledAttribute : ActionFilterAttribute
{
    public override void OnActionExecuted(ActionExecutedContext filterContext) {
        var viewModel = filterContext.Controller.ViewData.Model as IProductResultViewModel;
        if (viewModel == null)
            return;

        var rssFeedTitle = FeedHelper.MakeTitle(viewModel.Results);
        filterContext.Controller.ViewData.Add("RssFeedTitle", rssFeedTitle);

        var format = filterContext.RequestContext.HttpContext.Request.QueryString["format"];
        if (format == "rss" && rssFeedTitle != null) {
            var urlHelper = new UrlHelper(filterContext.RequestContext);
            var url = QueryStringUtility.RemoveQueryStringParameter(filterContext.RequestContext.HttpContext.Request.Url.ToString(), "format");
            var feedItems = FeedHelper.GetSyndicationItems(viewModel.Results, urlHelper);
            filterContext.Result = FeedHelper.CreateProductFeed(rssFeedTitle, viewModel.Description, new Uri(url), feedItems);
        }

        base.OnActionExecuted(filterContext);
    }
}

This class relies on our FeedHelper class to achieve three things it needs:

  1. MakeTitle takes the request details – i.e. which page, type of products, filtering and sorting is selected and makes a title by re-using our breadcrumbs
  2. GetSyndicationItems takes the IEnumerable<Product> and turns it into IEnumerable<SyndicationItem> by way of a foreach projecting Product into SyndicationItem with some basic HTML formatting, combining the product image and setting the correct category (with a yield thrown in for good measure)
  3. CreateProductFeed then creates a Syndication feed with the appropriate Copyright and Language set and chooses the formatter – in our case RSS 2.0 but could easily be Atom 1.0, e.g.
public static SyndicationFeedResult CreateProductFeed(string title, string description, Uri link, IEnumerable<SyndicationItem> syndicationItems)
{
    var feed = new SyndicationFeed(title, description, link, syndicationItems) {
        Copyright = new TextSyndicationContent(String.Format(Resources.FeedCopyrightFormat, DateTime.Now.Year)),
        Language = CultureInfo.CurrentUICulture.Name
    };

    return new FeedResult(new Rss20FeedFormatter(feed, false));
}

The FeedResult class is a simple one that takes the built-in .NET SyndicationFeed class and wires it up to MVC by implementing an ActionResult that writes the XML of the SyndicationFeedFormatter into the response as well as setting the application/rss+xml content type and encoding.

Advertising the feed in the head

Now that we have the ability to serve up RSS we need to let browsers know it exists.

The ActionFilter we wrote above needs to know the title of the RSS feed regardless of whether it is rendering the RSS (which needs a title) or rendering the page (which will need to advertise the RSS title) so it always calculates it and then puts it into the ViewData dictionary with the key RssFeedTitle.

Now finally our site’s master page can check for the existence of that key/value pair and advertise it out with a simple link tag:

var rssFeedTitle = ViewData["RssFeedTitle"] as string;
if (!String.IsNullOrEmpty(rssFeedTitle)) { %>
<link rel="alternate" type="application/rss+xml" title="<%:rssFeedTitle%>" href="<%:Url.ForThisAsRssFeed%>" />
<% }

This code requires just one more thing – a very small UrlHelper which will append “format=rss” to the query string (taking into account whether there existing query parameters or not).

The result of this is we can now just add [RssEnabled] in front of any controller or action to turn on RSS feeds for that portion of our marketplace! :)

[)amien

Enums – Better syntax, improved performance and TryParse in NET 3.5

Recently I needed to map external data into in-memory objects. In such scenarios the TryParse methods of Int and String are useful but where is Enum.TryParse? TryParse exists in .NET 4.0 but like a lot of people I’m on .NET 3.5.

A quick look at Enum left me scratching my head.

  • Why didn’t enums receive the generic love that collections received in .NET 2.0?
  • Why do I have to pass in typeof(MyEnum) everywhere?
  • Why do I have to the cast results back to MyEnum all the time?
  • Can I write TryParse and still make quick – i.e. without try/catch?

I found myself with a small class, Enum<T> that solved all these. I was surprised when I put it through some benchmarks that also showed the various methods were significantly faster when processing a lot of documents. Even my TryParse was quicker than that in .NET 4.0.

While there is some small memory overhead with the initial class (about 5KB for the first, a few KB per enum after) the performance benefits came as an additional bonus on top of the nicer syntax.

Before (System.Enum)

var getValues = Enum.GetValues(typeof(MyEnumbers)).OfType();
var parse = (MyEnumbers)Enum.Parse(typeof(MyEnumbers), "Seven");
var isDefined = Enum.IsDefined(typeof(MyEnumbers), 3);
var getName = Enum.GetName(typeof(MyEnumbers), MyEnumbers.Eight);
MyEnumbers tryParse;
Enum.TryParse<MyEnumbers>("Zero", out tryParse);

After (Enum<T>)

var getValues = Enum<MyEnumbers>.GetValues();
var parse = Enum<MyEnumbers>.Parse("Seven");
var isDefined = Enum<MyEnumbers>.IsDefined(MyEnumbers.Eight);
var getName = Enum<MyEnumbers>.GetName(MyEnumbers.Eight);
MyEnumbers tryParse;
Enum<MyEnumbers>.TryParse("Zero", out tryParse);

I also added a useful ParseOrNull method that lets you either return null or default using the coalesce so you don’t have to mess around with out parameters, e.g.

MyEnumbers myValue = Enum<MyEnumbers>.ParseOrNull("Nine-teen") ?? MyEnumbers.Zero;

The class

GitHub has the latest version of EnumT.cs

Usage notes

  • This class as-is only works for Enum’s backed by an int (the default) although you could modify the class to use longs etc.
  • I doubt very much this class is of much use for flag enums
  • Casting from long can be done using the CastOrNull function instead of just putting (T)
  • GetName is actually much quicker than ToString on the Enum… (e.g. Enum<MyEnumbers>.GetName(a) over a.ToString())
  • IsDefined doesn’t take an object like Enum and instead has three overloads which map to the actual types Enum.IsDefined can deal with and saves runtime lookup
  • Some of the method may not behave exactly like their Enum counterparts in terms of exception messages, nulls etc.

[)amien

Anatomy of a good bug report

Working on the .NET Framework was an interesting but often difficult time especially when dealing with vague or incomprehensible bug reports.

Look before you file

Head to Bing, Google, official support sites and bug database if you have access to it (Microsoft Connect, Apple Radar, Bugzilla for Firefox etc.) to see if others have run into this issue. Searching for the error message can yield good results but remove any elements of the message specific to your project (e.g. class names, property names etc.)

Why accept a workaround?

Do not underestimate the value of a solid workaround.

A patch is significant effort for both sides and for re-distributable components like .NET Framework getting the components to the right place can be tough. You’ll need to deploy to your team, build, test and staging environments and, if you distribute software, to all your customers.

This can be especially painful when strictly controlled environments are shared with others and ops teams are hesitant about putting patches or hotfixes into production. Factor in processor architectures, operating systems and language support and weigh it up.

The reasons for this are:

  • you may have made a mistake (likely if the software has seen considerable use)
  • behave differently to your expectations (the dreaded ‘by design’)
  • already be fixed (in the next release and possibly as a patch)
  • have an acceptable workaround (why accept a workaround?)

If you are running into problems with developer tools also check out:

Consider how likely it is you’ve discovered a bug given the complexity of what you are doing, how unusual it is and how mature the software is. Attempting something simple on an established piece of software likely means you’ve made a mistake or misunderstood the documentation.

A good bug report

So you decided to go ahead and report the bug (we all want better software after all).

The essence of a good bug report is having just the right amount of information to identify the bug without any unnecessary detail.

Let’s break it down.

What happened

The most important element is describing what happened and there are four major possibilities.

Error message

You told the software to do something and it displayed an error message instead.

On a good day the error message lets you know why it can’t do what you asked and lets you know what to do to make it work. Given you’re filing a bug report it isn’t one of those days.

The error message is likely cryptic, doesn’t tell you how to get what you want or is just plain wrong. We’ll need that message in its entirety so:

  • Copy Windows message box contents as text by pressing control-c when it’s in focus
  • When there is a lot of text take a screen print (printscreen on Windows, command-shift-3 on Mac OS X)
  • Localized error messages may slow down support responses – switch the app back to their language first

Exceptions

When a piece of a program (called a method or function) can’t do what it claims it throws an ‘exception’ back up to the piece that asked (called) it. This exception is like an error message but with enough lot of technical detail that travels back up the program until something deals with it (known as a catch).

When you see an exception then nothing wanted to deal with it.

You will see an error message that contains the exception and possibly a list of program pieces that couldn’t deal with it. This is called a stack trace and is invaluable to the person investigating your report so include it.

Developers seeing exceptions in their own program need to determine if they should be catching that exception or whether it shouldn’t be occurring. Feel frim the stack trace so your own methods aren’t included but too much is better than too little.

Unexpected behaviour

The software should have done what you wanted but did something you didn’t expect instead.

Report what you thought should happen, what the software did instead, how this is different and the reason why you think it should be that way.

Other people may not agree with your change and in the case of shared programming libraries or frameworks a fix for you can become a break for others that rely on existing behaviour.

Terminated

The software just vanished from the screen without a trace. Crashed, terminated or unexpectedly quit and perhaps took some of your work with it :(

If you’re really unlucky the software that crashed is your operating system. The blue screen of death (BSOD) on Windows, the grey screen on Mac OS X.

Often there are logs left behind you can examine to identify what went wrong.

On Mac OS X fire up Console from Application > Utilities and see what you can find. iPhones, iPods and iPads do this quite often and iTunes will likely offer to send detailed information to Apple who will hopefully share it with the application developer if it’s not their own.

Windows users will want to head to the Event Viewer and find the error and any additional messages that appear to relate to it and included these too.

Steps to reproduce

Ideally the minimal number of steps to reproduce the error every time. You want this to be reliable as possible as companies have limited resources too and won’t spend days trying to reproduce a low-impact bug.

This can be a very important step in understanding the scope of the problem and it’s impact. If you can’t reproduce it in a minimal number of steps there’s always the possibility you’re overlooking some other aspect that could be causing the problem – bad data or rogue code elsewhere!

For an application this may be a data file or a number of manual steps a user must manually perform via the user interface.

For a framework a small self-contained project file with a minimal amount of code to reproduce the failing scenario.

Again here, if you can make the steps clear and ideally in the language of the company producing the software your bug report is not going to hit extra delays.

Environment

Bugs are often sensitive to their environment and you should always include the version number of the software you are using as well as pertinent platform details,  including:

  • What operating system, version & service pack
  • What processor architecture (x86, x64, IA-64)
  • What language & locale your machine is running in (e.g. US English (en-US), Brazil Portuguese (pr-BR))

In the specific case of Visual Studio and .NET bugs:

  • What version of Visual Studio you are using (including any service packs)
  • What processor architecture you are compiling for
  • What language and compiler version you are using (e.g. C# compiling for 4.0)

You may need to include details for your desktop or developer machine and details of the server it is connecting to if any are involved.

What you’ve tried

Chances are you tried several things before filing a bug report. Let us know if you tried:

  • Alternate routes through the user interface
  • Entering data in a different format or order
  • A different computer or environment

Letting them know this means they can avoid suggesting things you’ve already tried or waste time trying them too.

[)amien