Posts in category .net

Creating OR expressions in LINQ

As everybody who has read my blog before knows, I love LINQ and miss it when coding in other languages, so it’s nice when I get a chance to use it again. When I come back to it with fresh eyes, I notice some things aren’t as easy as they should be - and this time is no exception.

Background

People often need to build up LINQ expressions at runtime based on filters or criteria a user has selected. Adding criteria is incredibly easy, as you can chain operations together on the IQueryable interface, e.g.

if (customerActive)
  query = query.Where(c => c.IsActive);
if (customerCountry != null)
  query = query.Where(c => c.Country == customerCountry);

This is great if you want to AND things together, which is often the case in building up filters. Other operations, like Contains, are useful in allowing many options against a specific field.

Problem

But what about offering a choice involving either of two properties? One way to write it would be:

if (customerActive) {
  if (customerEnterprise) {
    query = query.Where(c => c.IsActive || c.IsEnterprise);
  } else {
    query = query.Where(c => c.IsActive);
  } else {
    if (customerEnterprise) {
      query = query.Where(c => c.IsEnterprise);
    }
  }
}

This code is already hard to read and is going to increase in complexity for each new option.

Solution

With some helper code, we can combine two expression predicates into a single OrElse for use in a Where condition.

class Or {
  public static Expression<Func<T, bool>> Combine<T>(Expression<Func<T, bool>> left, Expression<Func<T, bool>> right) {
    if (left == null && right == null) throw new ArgumentException("At least one argument must not be null");
    if (left == null) return right;
    if (right == null) return left;
    
    var parameter = Expression.Parameter(typeof(T), "p");
    var combined = new ParameterReplacer(parameter).Visit(Expression.OrElse(left.Body, right.Body));
    return Expression.Lambda<Func<T, bool>>(combined, parameter);
  }

  class ParameterReplacer : ExpressionVisitor {
    readonly ParameterExpression parameter;

    internal ParameterReplacer(ParameterExpression parameter) {
      this.parameter = parameter;
    }

    protected override Expression VisitParameter(ParameterExpression node) {
      return parameter;
    }
  }
}

Now we can:

Expression<Func<Customer, bool>> criteria = null;
if (customerActive)
    criteria = Or.Combine(criteria, c => c.IsActive == customerActive);
if (customerEnterprise)
    criteria = Or.Combine(criteria, c => c.IsEnterprise == customerEnterprise);

How it works

There are two parts to this working. The first combines the two separate predicates into a single OR lambda, taken care of by the Expression.OrElse method.

That alone can’t quite do what we need because the two expressions each have a separate parameter variable. The visitor replaces both of these with a new parameter we’ll pass into the new Lambda expression we’re creating that combines both conditions.

Caveats

The code, as it stands, creates an unbalanced tree. Ideally, the tree should be balanced to limit the depth LINQ providers must traverse to translate the query. I doubt you’ll run into the dreaded Stack Overflow (the message, not the site), but if you do, that’s why.

An overload that takes an Enumerable of Expression<Func<T, bool>> and produces a balanced tree is left as an exercise to the reader :p

[)amien

Azure Pipeline Build Variables

Azure Pipelines has been interesting to me especially given the generous free open source tier and seemingly instant availability of build agents. The setup is easy if you’re building .NET targets with lots of useful starters available.

Some areas have been frustrating, specifically build variables, the difficulty in getting the app installed, and the limitations on triggers if you can’t (non-GitHub apps can’t be installed on Enterprise accounts even if FREE).

We’ll concentrate on the build variable oddities for this post.

The Pipeline configuration helpfully has a tab named Variables that looks much like you’d expect with a big list of names, values, a lock icon and a checkbox for ‘Settable at queue time’.

There are, however, some oddities when compared to existing CI servers with these - especially if you expect to be able to use them as environment variables.

Naming

The documentation for these - especially for Predefined build variables shows the expected variables but, they use a dot naming convention.

This dot naming convention does not work if you are using them as environment variables. They are upper-cased and . is replaced with _ so for example:

Build.BuildNumber is actually BUILD_BUILDNUMBER.

Secrets

One even more confusing element is that when you mark your own (sensibly named) value as secret it suddenly stops being available as an environment variable.

Note: You should never build PRs with secret variables. Doing so allows anyone to submit a PR that simply echos them to the build log for harvesting. By default secrets are not available to pull requests in Azure Pipelines - leave it that way!

So how do you get these secret variables re-exposed as environment variables? While Stack Overflow has a bunch of complicated answers there is a much simpler way:

  1. Select the step from your job
  2. Expand the environment variables section
  3. Repeat the following steps for each environment variable
  4. Enter your variable name in the Name box
  5. Enter $(variable_name) in the Value box

Now when you run the job, the build step will use the secure variable. This screenshot shows you an example:

Screenshot of the Azure Pipeline Variable being mapped

If I find workarounds for the app installations, I’ll blog them!

[)amien

Model binding form posts to immutable objects

I’ve been working on porting over my blog to a static site generator. I fired up an Azure Function to handle the form-comment to PR process to enable user comments to still be part of the site without using a 3rd party commenting system - more on that in the next post - and found the ASP.NET model binding for form posts distinctly lacking.

It’s been great getting back into .NET and brushing up some skills making the code clear, short and reusable. What I wanted was a super-clear action on my controller that tried to collect, validate and sanitize the data then, if all was well, create the pull request or report errors.

Ideally, it would look like this;

[FunctionName("PostComment")]
public static async Task<HttpResponseMessage> Run([HttpTrigger(AuthorizationLevel.Anonymous, "post")] HttpRequestMessage request) {
    var form = await request.Content.ReadAsFormDataAsync();
    if (TryCreateComment(form, out Comment comment, out var errors))
        await CreateCommentAsPullRequest(comment);
    return request.CreateResponse(errors.Any()
      ? HttpStatusCode.BadRequest : HttpStatusCode.OK, String.Join("\n", errors));
}

To do that, we need a function capable of creating the Comment class from the form post. You could manually do it field by field, but that’s not reusable, highly repetitive, and, of course, no fun. The Comment class is - like all well-behaved little objects - immutable.

Creating a function to do this is simple with a little bit of reflection;

private static object ConvertParameter(string parameter, Type targetType) {
    return String.IsNullOrWhiteSpace(parameter)
           ? null : TypeDescriptor.GetConverter(targetType).ConvertFrom(parameter);
}

private static bool TryCreateCommentFromForm(NameValueCollection form, out Comment comment, out List<string> errors) {
    var constructor = typeof(Comment).GetConstructors()[0];
    var values = constructor.GetParameters()
                            .ToDictionary(p => p.Name, p => ConvertParameter(form[p.Name], p.ParameterType)
                                      ?? (p.HasDefaultValue ? p.DefaultValue : new MissingRequiredValue()));
    errors = values.Where(p => p.Value is MissingRequiredValue)
                   .Select(p => $"Form value missing for '{p.Key}'").ToList();
    comment = errors.Any() ? null : (Comment)constructor.Invoke(values.Values.ToArray());
    return !errors.Any();
}

This method grabs the constructor for the Comment and tries to find keys in the form that match the parameter name. Any missing are reported as errors unless they have a default value, in which case that default is used. MissingRequiredValue is just an empty object to act as a sentinel. The use of TypeDescriptor.GetConverter means it should be quite happy handling integers, decimals, and URLs.

The Comment constructor specifies which fields are required, and the parameter names must match the form field names by convention. Any optional value has a default value that the constructor provides a sensible default for.

public Comment(string post_id, string message, string author, string email,
    DateTime? date = null, Uri url = null, int? id = null, string gravatar = null) {
    this.post_id = pathValidChars.Replace(post_id, "-");
    this.message = message;
    this.author = author;
    this.email = email;
    this.date = date ?? DateTime.UtcNow;
    this.url = url;
    this.id = id ?? new { this.post_id, this.author, this.message, this.date }.GetHashCode();
    this.gravatar = gravatar ?? EncodeGravatar(email);
}

I’ll post more of the form commenting system source soon once it’s a bit better tested and I’ve looked into anti-spam integration. The Jekyll rendering templates and WordPress exporter are available.

[)amien

Differences between Azure Functions v1 and v2 in C#

I’ve been messing around in the .NET ecosystem again, jumping back in with Azure Functions (similar to AWS Lambda) to get my blog onto 99% static hosting. I immediately ran into the API changes between v1 and v2 (currently in beta).

These changes are because v1 was based around .NET 4.6 using WebAPI 2 while v2 is based on ASP.NET Core which uses MVC 6. There are some guides around conversion, but none in the context of Azure Functions.

I’ll illustrate with a PageViewCount sample that uses Table Storage to retrieve and update a simple page count.

v1 (.NET 4.61 / WebAPI 2)

[FunctionName("PageView")]
public static async Task<HttpResponseMessage> Run(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get")]HttpRequestMessage req, TraceWriter log) {
    var page = req.MessageUri.ParseQueryString()["page"];
    if (String.IsNullOrEmpty(page))
        return req.CreateErrorResponse(HttpStatusCode.BadRequest, "'page' parameter missing.");

    var table = Helpers.GetTableReference("PageViewCounts");
    var pageView = await table.RetrieveAsync<PageViewCount>("damieng.com", page)
        ?? new PageViewCount(page) { ViewCount = 0 };
    var operation = pageView.ViewCount == 0
        ? TableOperation.Insert(pageView)
        : TableOperation.Replace(pageView);
    pageView.ViewCount++;
    await table.ExecuteAsync(operation);

    return req.CreateResponse(HttpStatusCode.OK, new { viewCount = pageView.ViewCount });
}

v2 (ASP.NET Core / MVC 6)

[FunctionName("PageView")]
public static async Task<IActionResult> Run(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get")]HttpRequest req, TraceWriter log) {
    var page = req.Query["page"];
    if (String.IsNullOrEmpty(page))
       return new BadRequestObjectResult("'page' parameter missing.");

    var table = Helpers.GetTableReference("PageViewCounts");
    var pageView = await table.RetrieveAsync<PageViewCount>("damieng.com", page)
        ?? new PageViewCount(page) { ViewCount = 0 };
    var operation = pageView.ViewCount == 0
        ? TableOperation.Insert(pageView)
        : TableOperation.Replace(pageView);
    pageView.ViewCount++;
    await table.ExecuteAsync(operation);

    return new OkObjectResult(new { viewCount = pageView.ViewCount });
}

Differences

The main differences are that:

  1. Return types are IActionResult/ObjectResult objects rather than extension methods against HttpRequestMessage (easier to mock/create custom ones)
  2. Input is the HttpRequest object rather than HttpResponseMessage (easier to get query parameters)

The error Can not create abstract class when executing your function means you are trying to use the wrong tech for that environment.

Helpers

Both classes above utilise a small helper class to take care of Table Storage which doesn’t have the nicest to use API. A data-context like wrapper that ensures the right types go to the right table might be an even better option.

static class Helpers {
    public static CloudStorageAccount GetCloudStorageAccount() {
        var connection = ConfigurationManager.AppSettings["DamienGTableStorage"];
        return connection == null ? CloudStorageAccount.DevelopmentStorageAccount : CloudStorageAccount.Parse(connection);
    }

    public static CloudTable GetTableReference(string name) {
        return GetCloudStorageAccount().CreateCloudTableClient().GetTableReference(name);
    }

    public static async Task<T> RetrieveAsync<T>(this CloudTable cloudTable, string partitionKey, string rowKey)
        where T:TableEntity {
        var tableResult = await cloudTable.ExecuteAsync(TableOperation.Retrieve<T>(partitionKey, rowKey));
        return (T)tableResult.Result;
    }
}

To compile

If you want to compile this, or Google led you here looking for code to do a simple page counter, here’s the missing TableEntity class;

public class PageViewCount : TableEntity
{
    public PageViewCount(string pageName)
    {
        PartitionKey = "damieng.com";
        RowKey = pageName;
    }

    public PageViewCount() { }
    public int ViewCount { get; set; }
}

[)amien