Skip to content

Comma-separated parameter values in WebAPI  

The model binding mechanism in ASP.NET is pretty slick – it’s clever and highly extensible and built on TypeDescriptor system for all sorts of re-use that lets you get out of having to write boilerplate code to map between CLR objects and their web representations.

One surprising thing however is that out of the box neither WebAPI or MVC support comma-separated parameter values when bound to an array, e.g.

public class MyController : Controller {
    public string Page([FromUri]int[] ids) {
        return String.Join(" ; ", ids);
    }
}

Will only return 1 ; 2 ; 3 when supplied with /my/page?ids=1&ids=2&ids=3 and if you instead give it /my/page?ids=1,2,3 it will fail.

The reason for this was likely because there isn’t a standard for this at all and that the former – supported – scenario maps to what forms do when they post multiple value selections such as that in a select list box. The latter however is much more readable and is expected by some client frameworks and supported by some other web frameworks such as the Java Spring MVC framework.

Of course that extensible system lets us easily extend this behavior so that we can support both transparently – and interestingly enough – even mix-and-match on the same URL. So for example;

/my/page?ids=1,2&ids=3 will now return 1 ; 2 ; 3 in our example.

Although this supports both types if you are currently passing invalid or blank values to your app they are treated by the default model binder as 0. e.g. ?ids=&ids=1&ids=a would have been received as 0, 1, 0 but now empty values are skipped and invalid values would throw an error – as they should!

CommaSeparatedArrayModelBinder class

The source is available in the DamienGKit project but also here.

Out of the box it supports integer types and Guid’s although you could extend it to floats and decimals – again just be careful with that formatting!

public class CommaSeparatedArrayModelBinder : IModelBinder {
  private static readonly Type[] supportedElementTypes = {
    typeof(int), typeof(long), typeof(short), typeof(byte),
    typeof(uint), typeof(ulong), typeof(ushort), typeof(Guid)
  };

  public bool BindModel(HttpActionContext actionContext, ModelBindingContext bindingContext) {
    if (!IsSupportedModelType(bindingContext.ModelType)) return false;
    var valueProviderResult = bindingContext.ValueProvider.GetValue(bindingContext.ModelName);
    var stringArray = valueProviderResult?.AttemptedValue?
                      .Split(new[] { ',' }, StringSplitOptions.RemoveEmptyEntries);
    if (stringArray == null) return false;
    var elementType = bindingContext.ModelType.GetElementType();
    if (elementType == null) return false;

    bindingContext.Model = CopyAndConvertArray(stringArray, elementType);
    return true;
  }

  private static Array CopyAndConvertArray(IReadOnlyList sourceArray, Type elementType) {
    var targetArray = Array.CreateInstance(elementType, sourceArray.Count);
    if (sourceArray.Count > 0) {
      var converter = TypeDescriptor.GetConverter(elementType);
      for (var i = 0; i < sourceArray.Count; i++)
        targetArray.SetValue(converter.ConvertFromString(sourceArray[i]), i);
    }
    return targetArray;
  }

  internal static bool IsSupportedModelType(Type modelType) {
    return modelType.IsArray && modelType.GetArrayRank() == 1
      && modelType.HasElementType
      && supportedElementTypes.Contains(modelType.GetElementType());
  }
}

public class CommaSeparatedArrayModelBinderProvider : ModelBinderProvider {
  public override IModelBinder GetBinder(HttpConfiguration configuration, Type modelType) {
    return CommaSeparatedArrayModelBinder.IsSupportedModelType(modelType)
      ? new CommaSeparatedArrayModelBinder() : null;
  }
}

To register

It’s necessary to register ModelBinderProviders with your ASP.NET application at start-up, usually in the WebApiConfig.cs file.

public static class WebApiConfig {
  public static void Register(HttpConfiguration config) {
    // All your usual configuration up here
    config.Services.Insert(typeof(ModelBinderProvider), 0, new CommaSeparatedArrayModelBinderProvider());
  }
}

[)amien

Model binding form posts to immutable objects  

I’ve been working on porting over my blog to a static site generator and fired up an Azure Function to handle the form-comment to PR process to enable user comments to still be part of the site without using a 3rd party commenting system – more on that in a future post – and found the ASP.NET model binding for form posts distinctly lacking.

It’s been great getting back into .NET and brushing up some skills making the code clear, short and reusable. What I wanted was a super-clear action on my controller that tried to collect, validate and sanitize the data then if all was well create the pull request or report errors.

Ideally it would look like this;

[FunctionName("PostComment")]
public static async Task Run([HttpTrigger(AuthorizationLevel.Anonymous, "post")] HttpRequestMessage request) {
    var form = await request.Content.ReadAsFormDataAsync();
    if (TryCreateComment(form, out Comment comment, out var errors))
        await CreateCommentAsPullRequest(comment);
    return request.CreateResponse(errors.Any()
      ? HttpStatusCode.BadRequest : HttpStatusCode.OK, String.Join("\n", errors));
}

To do that however we need a function capable of creating the Comment class from the form post. Sure you can manually do it field by field but that’s not very reusable, repetitive and of course no fun. The Comment class is also – like all good little objects – immutable.

Creating a function to do this is simple with a little bit of reflection;

private static object ConvertParameter(string parameter, Type targetType) {
    return String.IsNullOrWhiteSpace(parameter)
           ? null : TypeDescriptor.GetConverter(targetType).ConvertFrom(parameter);
}

private static bool TryCreateCommentFromForm(NameValueCollection form, out Comment comment, out List errors) {
    var constructor = typeof(Comment).GetConstructors()[0];
    var values = constructor.GetParameters()
                            .ToDictionary(p => p.Name, p => ConvertParameter(form[p.Name], p.ParameterType)
                                      ?? (p.HasDefaultValue ? p.DefaultValue : new MissingRequiredValue()));
    errors = values.Where(p => p.Value is MissingRequiredValue)
                   .Select(p => $"Form value missing for '{p.Key}'").ToList();
    comment = errors.Any() ? null : (Comment)constructor.Invoke(values.Values.ToArray());
    return !errors.Any();
}

What this does is grab the constructor for the Comment and try to find keys in the form that match the parameter name. Any that are missing are reported as errors unless they have a default value in which case that is used. MissingRequiredValue is just an empty object to act as a sentinel. The use of TypeDescriptor.GetConverter means it should be quite happy handling ints, decimals, urls etc.

The constructor for Comment specifies which fields are required and the parameter names must match the form field names by convention. Any value that is optional has a default value that the constructor will happily fill in a sensible default for.

public Comment(string post_id, string message, string author, string email, 
    DateTime? date = null, Uri url = null, int? id = null, string gravatar = null) {
    this.post_id = pathValidChars.Replace(post_id, "-");
    this.message = message;
    this.author = author;
    this.email = email;
    this.date = date ?? DateTime.UtcNow;
    this.url = url;
    this.id = id ?? new { this.post_id, this.author, this.message, this.date }.GetHashCode();
    this.gravatar = gravatar ?? EncodeGravatar(email);
}

I’ll post more of the form commenting system source soon once it’s a bit better tested and I look into anti-spam integration. Ideally I’ll also provide an AWS Lambda variant of the code so you can choose (or load balance) comment posting and almost certainly get what you need on the free tier. For now the Jekyll rendering templates and WordPress exporter are available.

[)amien

Differences between Azure Functions v1 and v2 in C#  

I’ve been messing around in the .NET ecosystem again and have jumped back in with Azure Functions (similar to AWS Lambda) to get my blog onto 99% static hosting. I immediately ran into the API changes between v1 and v2 (currently in beta).

These changes are because v1 was based around .NET 4.6 using WebAPI 2 while the v2 is based on ASP.NET Core which uses MVC 6. There are some guides around to converting but none in the pure context of Azure Functions.

I’ll illustrate with a PageViewCount sample that uses Table Storage to retrieve and update a simple page count.

v1 (.NET 4.61 / WebAPI 2)

[FunctionName("PageView")]
public static async Task<HttpResponseMessage> Run(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get")]HttpRequestMessage req, TraceWriter log)
{
    var page = req.MessageUri.ParseQueryString()["page"];
    if (String.IsNullOrEmpty(page))
        return req.CreateErrorResponse(HttpStatusCode.BadRequest, "'page' parameter missing.");

    var table = Helpers.GetTableReference("PageViewCounts");
    var pageView = await table.RetrieveAsync<PageViewCount>("damieng.com", page) ?? new PageViewCount(page) { ViewCount = 0 };
    var operation = pageView.ViewCount == 0
        ? TableOperation.Insert(pageView)
        : TableOperation.Replace(pageView);

    pageView.ViewCount++;
    await table.ExecuteAsync(operation);

    return req.CreateResponse(HttpStatusCode.OK, new { viewCount = pageView.ViewCount });
}

v2 (ASP.NET Core / MVC 6)

[FunctionName("PageView")]
public static async Task<IActionResult> Run(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get")]HttpRequest req, TraceWriter log)
{
    var page = req.Query["page"];
    if (String.IsNullOrEmpty(page))
       return new BadRequestObjectResult("'page' parameter missing.");

    var table = Helpers.GetTableReference("PageViewCounts");
    var pageView = await table.RetrieveAsync<PageViewCount>("damieng.com", page) ?? new PageViewCount(page) { ViewCount = 0 };
    var operation = pageView.ViewCount == 0
        ? TableOperation.Insert(pageView)
        : TableOperation.Replace(pageView);

    pageView.ViewCount++;
    await table.ExecuteAsync(operation);

    return new OkObjectResult(new { viewCount = pageView.ViewCount });
}

Differences

The main differences are that:

  1. The return types are now the simpler IActionResult/ObjectResult objects rather than extension methods off of HttpRequestMessage (so, easier to mock/create custom ones)
  2. The input is the HttpRequest object rather than HttpResponseMessage (easier to get query parameters)

If you get an error about ‘Can not create abstract class’ when executing your function then you are trying to use the wrong one in the wrong environment!

Helpers

Both classes above use a small helper class to take care of Table Storage which doesn’t have the nicest to use API. A wrapper much like a data context that ensures the right types go to the right table names might be an even better options.

static class Helpers
{
    public static CloudStorageAccount GetCloudStorageAccount()
    {
        var connection = ConfigurationManager.AppSettings["DamienGTableStorage"];
        return connection == null 
            ? CloudStorageAccount.DevelopmentStorageAccount
            : CloudStorageAccount.Parse(connection);
    }

    public static CloudTable GetTableReference(string name)
    {
        return GetCloudStorageAccount().CreateCloudTableClient().GetTableReference(name);
    }

    public static async Task<T> RetrieveAsync<T>(this CloudTable cloudTable, string partitionKey, string rowKey) 
       where T:TableEntity
    {
        var tableResult = await cloudTable.ExecuteAsync(TableOperation.Retrieve(partitionKey, rowKey));
        return (T)tableResult.Result;
    }
}

To compile

If you want to compile this or maybe you were just looking for code to do a simple page counter here’s the missing TableEntity class;

public class PageViewCount : TableEntity
{
    public PageViewCount(string pageName)
    {
        PartitionKey = "damieng.com";
        RowKey = pageName;
    }

    public PageViewCount() { }
    public int ViewCount { get; set; }
}

[)amien

Download files with progress in Electron via window.fetch  

Working on Atom lately I need to be able to download files to disk. We have a couple of ways to do this today but they do not show download progress which leads to confusion and sometimes frustration on larger downloads such as updates or big packages.

There are many npm libraries out there but they either don’t expose a progress indicator or they bypass Chrome (thus not using proxy settings, caching and network inspector) by using node directly.

I’m also not a fan of sprawling dependencies to achieve what can be done simply in a function or two.

Hello window.fetch

window.fetch is a replacement for XMLHttpRequest currently shipping in Chrome (and therefore Electron) as well as a whatWG living standard. While there is some documentation around most of it relies on grabbing the entire content as JSON, a blob or text. This is not advised for streaming where the files might be large and you want to not only minimize memory impact but also display a progress indicator to your users.

Thankfully window.fetch has a getReader() function that will give you a ReadableStreamReader although this reads in chunks (32KB on my machine) and isn’t compatible with Node’s streams, pipes and data events.

Download function

With a little work though we can wire these two things up to get us a file downloader that has no extra dependencies outside of Electron, honors the Chrome cache, proxy and network inspector and best of all is incredibly easy to use;

import fs from 'fs';

export default async function download(sourceUrl, targetFile, progressCallback, length) {
  const request = new Request(sourceUrl, {
    headers: new Headers({'Content-Type': 'application/octet-stream'})
  });

  const response = await fetch(request);
  if (!response.ok) throw Error(`Unable to download, server returned ${response.status} ${response.statusText}`);

  const body = response.body;
  if (body == null) throw Error('No response body');

  const finalLength = length || parseInt(response.headers.get('Content-Length') || '0', 10);
  const reader = body.getReader();
  const writer = fs.createWriteStream(targetFile);

  await streamWithProgress(finalLength, reader, writer, progressCallback);
  writer.end();
}

async function streamWithProgress(length, reader, writer, progressCallback) {
  let bytesDone = 0;

  while (true) {
    const result = await reader.read();
    if (result.done) {
      if (progressCallback != null) {
        progressCallback(length, 100);
      }
      return;
    }

    const chunk = result.value;
    if (chunk == null) {
      throw Error('Empty chunk received during download.');
    } else {
      writer.write(Buffer.from(chunk));
      if (progressCallback != null) {
        bytesDone += chunk.byteLength;
        const percent = length === 0 ? null : Math.floor(bytesDone / length * 100);
        progressCallback(bytesDone, percent);
      }
    }
  }
}

FlowType annotated and TypeScript versions are also available.

Using it

Using it is simplicity – call it with a URL to download and a local file name to save it as along with an optional callback that will receive download progress.

Downloader.download('https://download.damieng.com/fonts/original/EnvyCodeR-PR7.zip', 'envy-code-r.zip', 
   (bytes, percent) => console.log(`Downloaded ${bytes} (${percent})`));

Caveats

Some servers do not send the Content-Length header. You have two options if this applies to you;

  1. Don’t display a percentage just the KB downloaded count (percentage will be null in the callback)
  2. Bake-in the file size if it’s a static URL – just pass in as final parameter to the download function

Enjoy!

[)amien