' %} Software development, technology and typography » DamienG

SemVer is an intent - not a promise

What is semantic versioning (semver) ?

Semantic versioning is simple agreement on how packages should be versioned. It gives developers creating packages a framework to version their software with and consumers of those packages an expectation of how they can use them and upgrade as required.

The versioning format is major.minor.patch, described as:

  • major - Large improvements and breaking-changes.
  • minor - New features that are backward compatible.
  • patch - Bug fixes that are backward compatible.

The important thing to note here is that any breaking change is only supposed to be published in a major release.

What’s wrong with that?

Well, the reality is any change could theoretically break somebody already using your package. XKCD has a famous comic about it but even not taken to that extreme there are many ways you can unintentionally break existing users.

How can you unintentionally break existing users?

Here are a few examples of how you can break existing users with a seemingly safe bug fix or innocuous new feature:

Create a new class

Seemingly innocent thing to do - just create a new class. The problem is many languages allow you to import entire namespaces - C# for example - and class names must be unique or fully qualified. So if your class name collides with any others in-scope… boom, compiler error.

Add a method to an existing class

Okay, this is our class we can add a method to it. Wait… did we let the customer inherit from this class? If that well-chosen method name collides with one they implemented in the sub-class, boom. Compiler error.

Change a default value

Convention over configuration is a well-established mantra with a liberal use of defaults. What if you need to change the default? The industry guidance before was X and now it’s Y so you want to change the default to match. The problem is you have no idea of knowing if a customer accepted X because they always want your best-intentioned default or if they specifically wanted that value and knew it was the default when they first observed it.

Speed things up

Performance improvements are always good right?! Well, any speed up… or slow down… can cause previously rare deadlocks to increase or decrease. They can also affect back-pressure on other systems and exceed rate limits on API calls.

Upgrade a dependency

You upgraded a dependency… and now the package manager upstream is having difficulty reconciling it with the customers project as they also have this dependency.

Is semver a dream?

Semantic versioning isn’t a dream. It’s an ideal, an intent, something to strive for.

We can never be absolutely sure an update won’t break somebody. There’s no guarantees.

When we intentionally break things though we can make sure to align it with a major release and ideally bundle it up with some other breaks and a big enough set of features and enhancement to make it worth the effort.

ZX-Origins - free 8-bit fonts for games

I started designing fonts around 1987 on an 8-bit Sinclair ZX Spectrum. Many years later because of my active involvement in the Spectrum emulation scene Paul Dunn asked me to provide some fonts for his excellent BASIN Sinclair BASIC for Windows. My interest in 8x8 monospaced fonts was suitably rekindled and I ended up delivering about 60 - some even extracted from my original +3 disk images.

I’ve wanted to get these fonts online earlier but the raw files are only of use to BASIN users or those suitably familiar with the convoluted process and tools to get them into a Spectrum or emulator. Even trying to use them on Windows was a pain given that TrueType is all about scalable fonts and getting them pixel-perfect is quite the endeavor.

Fellow typographer and Spectrum fan Paul van der Laan gave me some suitable tips for pixel-perfect fonts at specific multiples (8px on Windows) and I put together an automated pipeline using a modified version of John Elliot’s psftools which converts the original ZX font to PSF and then to BDF. Getting to TrueType TTF involved FontLab 5 Studio and some Python scripting and the various conversion process and uploads were taken care of with some bash scripting.

It’s always hard to get a true feel for a font from a simple run of letters so I created a simulated preview mechanism that re-rendered some hand-picked screens from Sinclair, Commodore, Amiga, Atari and BBC computers to showcase each one. If your browser supports HTML5 canvas you’ll see these! You can also drag any raw Spectrum font file such as those created by BASIN to any of my ZX-Origins pages to see your own font rendered with these samples.

The uploads are now mostly complete - there are 78 typefaces and well over 120 actual fonts once you include the variations, styles and weights that some typefaces have. This number may increase from time to time if and when I finish off styles in my _incomplete folder, resurrect some abandoned ones (sorry Unwanted Attention), or get inspired for something entirely new.

So without further ado head over to ZX Origins and grab a font or two for your latest 8-bit retro game!

WordPress to Jekyll part 6 - A faster build

My site goes back to 2004 and is reasonably sized but not massive even with the comments so waiting 30 seconds for every change to be reflected is not only a pain but quite disappointing.

I investigated several options and ended up using a few in combination, skipping others and holding out for one.

Incremental build

Launching with the --incremental switch e.g. bundle exec jekyll serve --incremental can speed things up a bit but there are issues here. One is that it still builds a lot of pages, tags and categories as it updates and the second is that if you’re using collections it misses rebuilding pages that reference that collection.

Verdict: Not great.

WSL2 or a Mac?

My primary dev box these days is a beefy Windows box. Jekyll doesn’t seem to want to run at all under native Windows and Windows Subsystem for Linux (WSL) can be pretty slow compared to a Mac. Microsoft have announced Windows Subsystem for Linux 2 (WSL2) which will be much faster but it isn’t available until June 2019 and even then that’s to insiders.

Verdict: Hold tight.

Caching

Hubber extrodinaire Ben Balter developed the Jekyll Include Cache which can help speed up building.

The approach it uses is that you can switch out a regular include x for include_cached x providing that x doesn’t depend on any state/variables. For me that meant my footer, sidebar and navbar could all easily be parsed and stored just once by changing things like: {% include footer.html %} to {% include_cached footer.html %}

There are however places where state is important and this plugin can still help there. For example my header was called using {% include head.html %} and contained code like this:

<meta name="description" content="{{ page.excerpt | markdownify | strip_html | truncatewords:40 }}" />
<title>{{ page.title }} &raquo; DamienG</title>
<meta name="twitter:url" content="{{ page.url | absolute_url }}" />

What you can do here is pass parameters to include_cached and read them the other side. Replacing the include with:

{% include_cached head.html title=page.title url='page.url | absolute_url' excerpt='page.excerpt | markdownify | strip_html | truncatewords:40' %}

And changing the include itself to:

<meta name="description" content="{{ include.excerpt }}" />
<title>{{ include.title }} &raquo; DamienG</title>
<meta name="twitter:url" content="{{ include.url }}" />

Which actually also made the header easy to read as the title, description and url are used multiple times for the various types of metadata (Twitter, OpenGraph etc).

Verdict: Some good improvement.

No paging, tags and collections for local dev

I never actually browse the pages, tags or collections locally but they’re very important for the CI and publishing. I’ve manually turned them off in the past but it’s error-prone and a pain but there is a way to avoid that!

Simply move the parts of your configuration you don’t want to be part of your local build into a new configuration file, for example I moved the following parts from my _config.yml into a new _config-publish.yml:

autopages:
  enabled: true
  categories:
    enabled: true
  tags:
    enabled: true

pagination:
  enabled: true

I also moved my Aloglia configuration block here too so that local builds with content not yet published does not turn up in the search index!

It’s important to remember to add --config _config.yml,_config-publish.yml to any Jekyll build steps on your CI server, for example mine has:

 - run:
    name:    Build site
    command: bundle exec jekyll build --config _config.yml,_config-publish.yml 2>&1 | tee $JOB_RESULTS_PATH/build-results.txt

and:

 - run:
    name:    Index with Algolia
    command: bundle exec jekyll algolia --config _config.yml,_config-publish.yml

Verdict: Awesome, drops from 30s to 17s for full build.

Turn off comments for local

I thought this would have more of an impact but it only shaves another couple of seconds off the site build times.

To do this add the following first and last lines to your comments.html include (I’ll probably add this to the Jekyll Blog Comments system)

{% if site.comments.enabled %}
... (existing comments.html contents here)
{% endif %}

and remember to add this to your _config-publish.yml:

comments:
  enabled: true

Verdict: Okay, drops from 17s to 15s for full build.

Conclusion

Between these steps my site has gone from almost 50 seconds to about 17 for a full build without incremental and using that now means about 1-2 seconds!

I can’t wait for WSL2 to push this over the edge!

[)amien

Azure Pipeline Build Variables

Azure Pipelines has been interesting to me especially given the generous free open source tier and seemingly instant availability of build agents. The setup is especially easy if you’re building .NET targets with lots of useful starters available.

A couple of areas have, however, been frustrating however - build variables and the difficulty in getting the app installed and the limitations on triggers if you can’t (non-GitHub apps can’t be installed on Enterprise accounts even if FREE).

We’ll concentrating on the build variable oddities for this post.

The Pipeline configuration helpfully has a tab named Variables that looks much like you’d expect with a big list of names, values, a lock icon and a checkbox for ‘Settable at queue time’.

There are however some oddities when compared to existing CI servers with these especially if you expect to be able to use them as environment variables.

Naming

The documentation for these - especially for Predefined build variables shows the expected variables but they use a dot naming convention.

This dot naming convention does not work if you are using them as environment variables. They are upper-cased and . is replaced with _ so for example:

Build.BuildNumber is actually BUILD_BUILDNUMBER.

Secrets

One even more confusing element is that when you mark you own (sensibly named) value as secret it suddenly stops being available as a environment variable.

Note: You should never build PRs with secret variables. Doing so allows anyone to submit a PR that simply echos them to the build log for harvesting. By default secrets are not available to pull requests in Azure Pipelines - leave it that way!

So how do you get these secret variables re-exposed as environment variables? While Stack Overflow has a bunch of complicated answers there is a much simpler way:

  1. Select the step from your job
  2. Expand the environment variables section
  3. Repeat the following steps for each environment variable
  4. Enter your variable name in the Name box
  5. Enter $(variable_name) in the Value box

Now when you run the job the build step will use the secure variable. This screenshot shows you an example:

Screenshot of the Azure Pipeline Variable being mapped

If I figure out workarounds for the app installations I’ll blog them!

[)amien