WordPress to Jekyll part 1 - My history and reasoning

It’s hard to believe it was 13 years ago, back in a cold December on the little island of Guernsey, when I decided to start blogging. I’d had a static site with a few odd musings since 2000, but this was to be more regularly updated and with technical content. Blogspot seemed the easiest way to get started.

Briefly hosted at home

Within 18 months of regular blogging, I’d moved over to Subtext, which, being a .NET app, required Windows hosting, so I threw it on a small Shuttle PC on my home DSL. I started using it as an experiment for CSS and web techniques but, within a year, I’d had my 1MB DSL brought to its knees twice through articles featured on BoingBoing.

I did contribute a little to the project and started chatting with the maintainer Phil Haack. I’d later meet him when we both joined Microsoft years later then again at GitHub (small world).

Landing on WordPress

DamienG theme in 2008 In 2007, I migrated to a PHP-based CMS that was making a name for itself called WordPress. My blog would remain on WordPress for 10 years across shared hosting, VMs, and dedicated servers.

One server got caught in an explosion at the ISP. Another time my site got pwned through a WordPress vulnerability. I switched themes several times before creating my own home-grown super-light MootStrap theme based around the BootStrap 2 layout and navbar. I messed with wp-SuperCache, attempting to improve performance and scalability before switching out the PHP engine for HHVM. Then I switched Apache out for NGINX and MySQL for MariaDB in an attempt to eke out a bit of extra performance.

While my theme lives on today - for now at least - MootStrap and PHP are no more as I switched over to the Jekyll static site generator earlier this month after a long meandering journey to get there.

Why Jekyll?

I’ve had success with Jekyll on some other sites I run. Hosting it on GitHub pages or S3 with a CloudFront brings many benefits:

  1. Cost - S3 and CloudFront cost pennies rather than $40+ a month
  2. Security - there’s no code running to exploit, no WordPress plug-in back-doors
  3. Speed - CloudFront is a geo-distributed CDN, and S3 is no slouch either
  4. Editing - text files are simple to process, find, manipulate and markdown much more fluent to type

The price aspect is worth mentioning again. With the occasional bursts in traffic, my site hosting generally worked out around $40 a month for a decent VM. On AWS, I’m expecting it to max out at $3 despite these improvements and benefits.

Of course, another reason is that static-site generators are interesting, and I like to play.

Some challenges

Jekyll is a static site generator. You run the tool somewhere, and it produces plain HTML files with zero server-side code left in them. By its very nature is going to not have support for:

  • Comments - No way to accept or render them
  • Search - No site search facility
  • URL control - Difficult to match the paging/tags/categories with default plugins

Despite this, there are some blog-friendly plug-ins, specifically:

[)amien

Comma-separated parameter values in WebAPI

The model binding mechanism in ASP.NET is pretty slick - it’s highly extensible and built on TypeDescriptor for re-use that lets you avoid writing boilerplate code to map between CLR objects and their web representations.

One surprise, however, is that out of the box, neither WebAPI nor MVC support comma-separated parameter values when bound to an array, e.g.

public class MyController : Controller {
    public string Page([FromUri]int[] ids) {
        return String.Join(" ; ", ids);
    }
}

Will only return 1 ; 2 ; 3 when supplied with /my/page?ids=1&ids=2&ids=3 and if you instead give it /my/page?ids=1,2,3 it will fail.

The reason was likely because there is no standard for this at all and that the former - supported - scenario maps to what forms do when they post multiple value selections such as that in a select list box. The latter is much more readable and is expected by some client frameworks and supported by some other web frameworks such as the Java Spring MVC framework.

Of course, that extensible system lets us easily extend this behaviour. We can support both transparently - and interestingly enough - even mix-and-match on the same URL. So for example;

/my/page?ids=1,2&ids=3 will now return 1 ; 2 ; 3 in our example.

Although this supports both types if you are currently using commas in your number format this would break your app. e.g. ?ids=1,200&ids=3,500 would have been correctly received as 1200, 500 but now would be incorrectly received as 1, 200, 3, 500

CommaSeparatedArrayModelBinder class

My DamienGKit project contains the source but, I’ll also present it here.

Out of the box, this supports integer types and GUIDs and could be extended for floats and decimals – just be careful with that formatting!

public class CommaSeparatedArrayModelBinder : IModelBinder {
    private static readonly Type[] supportedElementTypes = {
        typeof(int), typeof(long), typeof(short), typeof(byte),
        typeof(uint), typeof(ulong), typeof(ushort), typeof(Guid)
    };

    public bool BindModel(HttpActionContext actionContext, ModelBindingContext bindingContext) {
        if (!IsSupportedModelType(bindingContext.ModelType)) return false;
        var valueProviderResult = bindingContext.ValueProvider.GetValue(bindingContext.ModelName);
        var stringArray = valueProviderResult?.AttemptedValue
            ?.Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries);
        if (stringArray == null) return false;
        var elementType = bindingContext.ModelType.GetElementType();
        if (elementType == null) return false;

        bindingContext.Model = CopyAndConvertArray(stringArray, elementType);
        return true;
    }

    private static Array CopyAndConvertArray(IReadOnlyList<string> sourceArray, Type elementType) {
        var targetArray = Array.CreateInstance(elementType, sourceArray.Count);
        if (sourceArray.Count > 0) {
            var converter = TypeDescriptor.GetConverter(elementType);
            for (var i = 0; i < sourceArray.Count; i++)
                targetArray.SetValue(converter.ConvertFromString(sourceArray[i]), i);
        }
        return targetArray;
    }

    internal static bool IsSupportedModelType(Type modelType) {
        return modelType.IsArray && modelType.GetArrayRank() == 1
                && modelType.HasElementType
                && supportedElementTypes.Contains(modelType.GetElementType());
    }

}

public class CommaSeparatedArrayModelBinderProvider : ModelBinderProvider {
    public override IModelBinder GetBinder(HttpConfiguration configuration, Type modelType) {
        return CommaSeparatedArrayModelBinder.IsSupportedModelType(modelType)
            ? new CommaSeparatedArrayModelBinder() : null;
    }
}

To register

It’s necessary to register ModelBinderProviders with your ASP.NET application at start-up - usually in the WebApiConfig.cs file.

public static class WebApiConfig {
    public static void Register(HttpConfiguration config) {
        // All your usual configuration up here
        config.Services.Insert(typeof(ModelBinderProvider), 0, new CommaSeparatedArrayModelBinderProvider());
    }
}

[)amien

Model binding form posts to immutable objects

I’ve been working on porting over my blog to a static site generator. I fired up an Azure Function to handle the form-comment to PR process to enable user comments to still be part of the site without using a 3rd party commenting system - more on that in the next post - and found the ASP.NET model binding for form posts distinctly lacking.

It’s been great getting back into .NET and brushing up some skills making the code clear, short and reusable. What I wanted was a super-clear action on my controller that tried to collect, validate and sanitize the data then, if all was well, create the pull request or report errors.

Ideally, it would look like this;

[FunctionName("PostComment")]
public static async Task<HttpResponseMessage> Run([HttpTrigger(AuthorizationLevel.Anonymous, "post")] HttpRequestMessage request) {
    var form = await request.Content.ReadAsFormDataAsync();
    if (TryCreateComment(form, out Comment comment, out var errors))
        await CreateCommentAsPullRequest(comment);
    return request.CreateResponse(errors.Any()
      ? HttpStatusCode.BadRequest : HttpStatusCode.OK, String.Join("\n", errors));
}

To do that, we need a function capable of creating the Comment class from the form post. You could manually do it field by field, but that’s not reusable, highly repetitive, and, of course, no fun. The Comment class is - like all well-behaved little objects - immutable.

Creating a function to do this is simple with a little bit of reflection;

private static object ConvertParameter(string parameter, Type targetType) {
    return String.IsNullOrWhiteSpace(parameter)
           ? null : TypeDescriptor.GetConverter(targetType).ConvertFrom(parameter);
}

private static bool TryCreateCommentFromForm(NameValueCollection form, out Comment comment, out List<string> errors) {
    var constructor = typeof(Comment).GetConstructors()[0];
    var values = constructor.GetParameters()
                            .ToDictionary(p => p.Name, p => ConvertParameter(form[p.Name], p.ParameterType)
                                      ?? (p.HasDefaultValue ? p.DefaultValue : new MissingRequiredValue()));
    errors = values.Where(p => p.Value is MissingRequiredValue)
                   .Select(p => $"Form value missing for '{p.Key}'").ToList();
    comment = errors.Any() ? null : (Comment)constructor.Invoke(values.Values.ToArray());
    return !errors.Any();
}

This method grabs the constructor for the Comment and tries to find keys in the form that match the parameter name. Any missing are reported as errors unless they have a default value, in which case that default is used. MissingRequiredValue is just an empty object to act as a sentinel. The use of TypeDescriptor.GetConverter means it should be quite happy handling integers, decimals, and URLs.

The Comment constructor specifies which fields are required, and the parameter names must match the form field names by convention. Any optional value has a default value that the constructor provides a sensible default for.

public Comment(string post_id, string message, string author, string email,
    DateTime? date = null, Uri url = null, int? id = null, string gravatar = null) {
    this.post_id = pathValidChars.Replace(post_id, "-");
    this.message = message;
    this.author = author;
    this.email = email;
    this.date = date ?? DateTime.UtcNow;
    this.url = url;
    this.id = id ?? new { this.post_id, this.author, this.message, this.date }.GetHashCode();
    this.gravatar = gravatar ?? EncodeGravatar(email);
}

I’ll post more of the form commenting system source soon once it’s a bit better tested and I’ve looked into anti-spam integration. The Jekyll rendering templates and WordPress exporter are available.

[)amien

Differences between Azure Functions v1 and v2 in C#

I’ve been messing around in the .NET ecosystem again, jumping back in with Azure Functions (similar to AWS Lambda) to get my blog onto 99% static hosting. I immediately ran into the API changes between v1 and v2 (currently in beta).

These changes are because v1 was based around .NET 4.6 using WebAPI 2 while v2 is based on ASP.NET Core which uses MVC 6. There are some guides around conversion, but none in the context of Azure Functions.

I’ll illustrate with a PageViewCount sample that uses Table Storage to retrieve and update a simple page count.

v1 (.NET 4.61 / WebAPI 2)

[FunctionName("PageView")]
public static async Task<HttpResponseMessage> Run(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get")]HttpRequestMessage req, TraceWriter log) {
    var page = req.MessageUri.ParseQueryString()["page"];
    if (String.IsNullOrEmpty(page))
        return req.CreateErrorResponse(HttpStatusCode.BadRequest, "'page' parameter missing.");

    var table = Helpers.GetTableReference("PageViewCounts");
    var pageView = await table.RetrieveAsync<PageViewCount>("damieng.com", page)
        ?? new PageViewCount(page) { ViewCount = 0 };
    var operation = pageView.ViewCount == 0
        ? TableOperation.Insert(pageView)
        : TableOperation.Replace(pageView);
    pageView.ViewCount++;
    await table.ExecuteAsync(operation);

    return req.CreateResponse(HttpStatusCode.OK, new { viewCount = pageView.ViewCount });
}

v2 (ASP.NET Core / MVC 6)

[FunctionName("PageView")]
public static async Task<IActionResult> Run(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get")]HttpRequest req, TraceWriter log) {
    var page = req.Query["page"];
    if (String.IsNullOrEmpty(page))
       return new BadRequestObjectResult("'page' parameter missing.");

    var table = Helpers.GetTableReference("PageViewCounts");
    var pageView = await table.RetrieveAsync<PageViewCount>("damieng.com", page)
        ?? new PageViewCount(page) { ViewCount = 0 };
    var operation = pageView.ViewCount == 0
        ? TableOperation.Insert(pageView)
        : TableOperation.Replace(pageView);
    pageView.ViewCount++;
    await table.ExecuteAsync(operation);

    return new OkObjectResult(new { viewCount = pageView.ViewCount });
}

Differences

The main differences are that:

  1. Return types are IActionResult/ObjectResult objects rather than extension methods against HttpRequestMessage (easier to mock/create custom ones)
  2. Input is the HttpRequest object rather than HttpResponseMessage (easier to get query parameters)

The error Can not create abstract class when executing your function means you are trying to use the wrong tech for that environment.

Helpers

Both classes above utilise a small helper class to take care of Table Storage which doesn’t have the nicest to use API. A data-context like wrapper that ensures the right types go to the right table might be an even better option.

static class Helpers {
    public static CloudStorageAccount GetCloudStorageAccount() {
        var connection = ConfigurationManager.AppSettings["DamienGTableStorage"];
        return connection == null ? CloudStorageAccount.DevelopmentStorageAccount : CloudStorageAccount.Parse(connection);
    }

    public static CloudTable GetTableReference(string name) {
        return GetCloudStorageAccount().CreateCloudTableClient().GetTableReference(name);
    }

    public static async Task<T> RetrieveAsync<T>(this CloudTable cloudTable, string partitionKey, string rowKey)
        where T:TableEntity {
        var tableResult = await cloudTable.ExecuteAsync(TableOperation.Retrieve<T>(partitionKey, rowKey));
        return (T)tableResult.Result;
    }
}

To compile

If you want to compile this, or Google led you here looking for code to do a simple page counter, here’s the missing TableEntity class;

public class PageViewCount : TableEntity
{
    public PageViewCount(string pageName)
    {
        PartitionKey = "damieng.com";
        RowKey = pageName;
    }

    public PageViewCount() { }
    public int ViewCount { get; set; }
}

[)amien