Posts in category development

Shipping breaking changes

Breaking changes are always work for your users. Work you are forcing them to do when they upgrade to your new version. They took a dependency on your library or software because it saved them time but now it’s costing them time.

Every breaking change is a reason for them to stop and reconsider their options. If your library is paid-for or the preferred way for paying users to access your services then lost users can come with a real financial cost.

Use breaking changes sparingly.

Good reasons for breaking changes

  • A feature is being removed for business reasons
  • Standards are changing how something works
  • The feature did not work as intended and it’s impossible to fix without a break
  • Service or library dependencies you rely upon are forcing a change
  • A small breaking change now prevents actual significant breaking changes later

Even when presented with these you should think not only about whether you can avoid a break but also take the opportunity to think about what you can do now to avoid similar breaks in the future.

Poor reasons for breaking changes

  • It makes the internals of the library tidier
  • Parameter order, method naming or property naming would be ”clearer”
  • Consistency with other platforms or products
  • Personal subjective interpretations of “better”
  • Compatibility with a different library to attract new users

While many of these are admirable goals in of themselves they are not a reason to break your existing customers.

Managing breaking changes

It goes without saying that intentional breaking changes should only occur in major versions with the exception of security fixes that require users of your library take some action.

Here are some thoughts to ease the pain of breaking changes:

  • List each breaking change in a migration guide with a before and after code fragment
  • Summarize breaking changes in the README with a link to the migration guide for more information
  • Keep breaking change count low even on major releases

Users should ideally also be able to find/replace or follow compiler errors. Consider:

  • Platform-specific mechanisms for dealing with breaking changes. e.g. in C# you can use the [Obsolete] attribute to help guide to the replacement API, Java has the @deprecated annotation.
  • Leaving a stub for the old method in place for one whole major release that calls the new method with the right arguments and produces a log warning pointing to the migration guide.

Rewrites

If a package is drastically different users will need to rewrite code. This is always a bigger deal for them than you expect, because:

  1. It is unscheduled and was not on their radar (no they are not monitoring your GitHub issue discussions)
  2. They use your library in ways you likely don’t expect or anticipate
  3. The more they depend on your product then the more work your rewrite involves

Really consider whether your new API is actually better (ideally before you ship it). One way to do this is to produce a set of example usage code for using the old library vs the new library. Put them side-by-side and open them up to feedback. Is the new API genuinely better? Some indicators it might be are:

  • Less code for users to write
  • Plenty of sensible and safe defaults
  • Less specialized terminology and concepts
  • Easier to test and abstract
  • Existing customers prefer the new syntax and think it’s worth changing

Some indicators that it isn’t really any better is: internal staff prefer it, it aligns better with some other platform or just being different.

Sometimes it’s worth doing because it targets a different crowd or comes at the problem from a simpler direction or abstraction. If so, then seriously consider giving it a new package name and put it out for early access.

Make sure users are led to the right library of the two and if there is a lot of code duplication and users on the “old” library then consider making the next version of the old library a compatibility wrapper around the new library.

[)amien

From CircleCI to GitHub Actions for Jekyll publishing

I’ve been a big fan of static site generation since I switched from WordPress to Jekyll back in 2018. I’m also a big fan of learning new technologies as they come along, and now GitHub Actions are out in the wild; I thought this would be an opportunity to see how I can port my existing custom CircleCI build to Jekyll.

The CircleCI job

A quick recap from part 5 - Hosting & Building, my CircleCI configuration was basically two jobs that have subsequently been tweaked since then. They are:

Build

The build job’s responsibility was to configure a Ruby environment capable of executing Jekyll to build the site, removing the .html extension from the output filenames, and then indexing the content using Algolia. Here is the start of the .circleci/config.yml I was using for that:

version: 2
jobs:
  build:
    docker:
      - image: circleci/ruby:2.6.1
    working_directory: ~/jekyll
    environment:
      - JEKYLL_ENV=production
      - NOKOGIRI_USE_SYSTEM_LIBRARIES=true
      - JOB_RESULTS_PATH=run-results
    steps:
      - checkout
      - restore_cache:
          key: jekyll-{{ .Branch }}-{{ checksum "Gemfile" }}
      - run:
          name: Update gems
          command: gem update --system
      - run:
          name: Install dependencies
          command: bundle check --path=vendor/bundle || bundle install --path=vendor/bundle --jobs=4 --retry=3
      - save_cache:
          key: jekyll-{{ .Branch }}-{{ checksum "Gemfile" }}
          paths:
            - "vendor/bundle"
      - run:
          name: Create results directory
          command: mkdir -p $JOB_RESULTS_PATH
      - run:
          name: Build site
          command: bundle exec jekyll build --config _config.yml,_config-publish.yml 2>&1 | tee $JOB_RESULTS_PATH/build-results.txt
      - run:
          name: Remove .html suffixes
          command: find _site -name "*.html" -not -name "index.html" -exec rename -v 's/\.html$//' {} \;
      - run:
          name: Index with Algolia
          command: bundle exec jekyll algolia --config _config.yml,_config-publish.yml
      - store_artifacts:
          path: run-results/
          destination: run-results
      - persist_to_workspace:
          root: ~/jekyll
          paths:
            - _site

Deploy

Deploy takes the output from the build job and syncs it with the S3 bucket I use to publish the site. It then applies AWS S3-specific commands using the AWS CLI tool to ensure metadata, redirects, and caching are correctly set using a Python environment.

  deploy:
    docker:
      - image: circleci/python:2.7
    working_directory: ~/jekyll
    steps:
      - attach_workspace:
          at: ~/jekyll
      - run:
          name: Install AWS CLI
          command: sudo pip install awscli
      - run:
          name: Deploy to S3
          command: aws s3 sync _site s3://damieng-static/ --delete --content-type=text/html
      - run:
          name: Correct MIME for robots.txt automatically
          command: aws s3 cp s3://damieng-static/robots.txt s3://damieng-static/robots.txt --metadata-directive="REPLACE"
      - run:
          name: Correct MIME for sitemap.xml automatically
          command: aws s3 cp s3://damieng-static/sitemap.xml s3://damieng-static/sitemap.xml --metadata-directive="REPLACE"
      - run:
          name: Correct MIME for Atom feed manually
          command: aws s3 cp s3://damieng-static/feed.xml s3://damieng-static/feed.xml --no-guess-mime-type --content-type="application/atom+xml" --metadata-directive="REPLACE"
      - run:
          name: Redirect /damieng for existing RSS subscribers
          command: aws s3api put-object --bucket damieng-static --key "damieng" --website-redirect-location "https://damieng.com/feed.xml"
      - run:
          name: Latest Envy Code R redirect
          command: aws s3api put-object --bucket damieng-static --key "envy-code-r" --website-redirect-location "https://damieng.com/blog/2008/05/26/envy-code-r-preview-7-coding-font-released"
      - run:
          name: Latest Envy Code R redirect #2
          command: aws s3api put-object --bucket damieng-static --key "fonts/envy-code-r" --website-redirect-location "https://damieng.com/blog/2008/05/26/envy-code-r-preview-7-coding-font-released"
      - run:
          name: Latest Envy Code R download
          command: aws s3api put-object --bucket damieng-static --key "downloads/latest/EnvyCodeR" --website-redirect-location "https://download.damieng.com/fonts/original/EnvyCodeR-PR7.zip"
      - run:
          name: Correct MIME for CSS files
          command: aws s3 cp s3://damieng-static/css/ s3://damieng-static/css/ --metadata-directive="REPLACE" --recursive

GitHub Actions

So how could I go about this in GitHub Actions? I have to admit I spent far too long poking around and examining existing Jekyll actions. I have a bunch of steps here I need fine control of, especially around Algolia and S3. I finally ended up on what was quite a simple port.

Unlike the CircleCI configuration, I did not split these into two separate jobs because:

  1. There is no exact equivalent to persist_to_workspace and attach_workspace
  2. The alternative of storing and restoring the artifacts leaves large useless artifacts around
  3. I never ended up running the jobs separately
  4. GitHub Actions provides an environment with both Ruby and AWS CLI installed

So, on to the configuration which lives in .github/workflows/jekyll.yml in my case:

name: Build site and deploy

on:
  push:
    branches: [ master ]

jobs:
  build:
    name: Build + Deploy
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Setup Ruby
        uses: ruby/setup-ruby@v1
        with:
          ruby-version: 2.6.1

      - name: Ruby gem cache
        uses: actions/cache@v1
        with:
          path: vendor/bundle
          key: ${{ runner.os }}-gems-${{ hashFiles('**/Gemfile.lock') }}
          restore-keys: |
            ${{ runner.os }}-gems-
            
      - name: Install dependencies
        run: |
          bundle config path vendor/bundle
          bundle install --jobs 4 --retry 3

      - name: Build Jekyll site
        run:  bundle exec jekyll build --config _config.yml,_config-publish.yml

      - name: Remove .html suffixes except for index.html
        run: find _site -name "*.html" -not -name "index.html" | while read f; do mv "$f" "${f%.html}"; done                      

      - name: Index with Algolia
        env:
          ALGOLIA_API_KEY: ${{ secrets.ALGOLIA_API_KEY }}
        run:  bundle exec jekyll algolia --config _config.yml,_config-publish.yml
    
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1
    
      - name: Sync site with S3
        run:  aws s3 sync _site s3://damieng-static/ --delete --content-type=text/html

      - name: Correct MIME types
        run: |
          aws s3 cp s3://damieng-static/ s3://damieng-static/ --exclude "*" --include "*.txt" --metadata-directive="REPLACE"
          aws s3 cp s3://damieng-static/sitemap.xml s3://damieng-static/sitemap.xml --metadata-directive="REPLACE"
          aws s3 cp s3://damieng-static/feed.xml s3://damieng-static/feed.xml --no-guess-mime-type --content-type "application/atom+xml" --metadata-directive "REPLACE"
          aws s3 cp s3://damieng-static/css/ s3://damieng-static/css/ --metadata-directive "REPLACE" --recursive
          aws s3 cp s3://damieng-static/js/ s3://damieng-static/js/ --metadata-directive "REPLACE" --recursive
      
      - name: Redirect /damieng for existing RSS subscribers
        run:  aws s3api put-object --bucket damieng-static --key "damieng" --website-redirect-location "https://damieng.com/feed.xml"

      - name: Latest Envy Code R redirects
        run:  |
          aws s3api put-object --bucket damieng-static --key "fonts/envy-code-r" --website-redirect-location "https://damieng.com/blog/2008/05/26/envy-code-r-preview-7-coding-font-released"
          aws s3api put-object --bucket damieng-static --key "downloads/latest/EnvyCodeR" --website-redirect-location "https://download.damieng.com/fonts/original/EnvyCodeR-PR7.zip"
        
      - name: Set caching for images at 30 days
        run:  aws s3 cp s3://damieng-static/ s3://damieng-static/ --exclude "*" --include "*.svg" --include "*.ico" --include "*.jpg" --include "*.png" --include "*.webp" --include "*.gif" --recursive --metadata-directive REPLACE --expires 2034-01-01T00:00:00Z --acl public-read --cache-control max-age=2592000,public

      - name: Set caching for CSS and JS at 1 hour
        run:  aws s3 cp s3://damieng-static/ s3://damieng-static/ --exclude "*" --include "*.css" --include "*.js" --recursive --metadata-directive REPLACE --expires 2034-01-01T00:00:00Z --acl public-read --cache-control max-age=3600,public

I also took the opportunity to fix a long-running issue in that my S3 objects would lose my manually-applied cache settings (and new posts and files would not have any).

Gotchas

There were only a couple of bumps in the road once I decided on a straight-port rather than trying to leverage higher-level existing actions:

  1. The secrets configured in the repo settings were not automatically exposed to the commands running in the action. Instead, you have to expose them using the ${{ secrets.KEY_NAME }} syntax.
  2. I was using rename instead of mv. I don’t recall why. Perhaps it was my Windows-ness creeping in. Rename has been dropped in newer distros.

Summary

The syntax is surprisingly similar with build-times about the same as CircleCI. It’s just nice to have it in one place. Time to port some other repos over!

[)amien

Notes from my Spectrum +3 manual

I’ve recently been working on a full HTML5 conversion of the Sinclair Spectrum +3 manual with full canvas-drawn screenshots and diagrams for smooth scaling/high res displays as well as some close font matching and layout as well as cross-reference links all over the place. More on that in the future.

What I wanted to show today was the odd commands, pokes and outs that were hastily scribbled down in the back of my manual over the years.

Spectrum +3/+2A specific

  • You know you can type SPECTRUM to go from +3 BASIC to 48K BASIC but, did you know you can get back? RANDOMIZE USR 23354. It leaves CHR$ 163/164 as UDGs.
  • Switch off the +3 drive with POKE 23399,4. In 48K mode, you can use POKE 23611,221 if it somehow stayed on.

128K specific

  • 128K machines are very compatible but the last two user-defined graphics (UDGs) T and U have become the SPECTRUM and PLAY command (for switching to 48K mode and using the AY-3-8912 sound chip). You can, however, switch them back to UDGs using POKE 23611,205.
  • POKE 23611,205; RANDOMIZE USR 4867; alternative way to switch from 128K to 48K.

A couple of VTX-5000 modem specific hacks:

  • OUT 8189,0 to switch the 128K ROM back in.
  • POKE 23611,29 also to switch from 48K to 128K mode.

All machines

  • POKE 23617,n changes the input mode for INPUT prompts. 0=Alpha (C/L), 1=Extended (E), 2=Graphics (G), 4=Keyword (K).
  • POKE 23658,0 turns off CAPS LOCK while POKE 23658,1 turns it on.
  • POKE 23692,255 will suppress the Scroll? prompt 255 times so you can auto-scroll.
  • RANDOMIZE USR 3582 to scroll the screen one line.
  • RANDOMIZE USR 3330 to scroll the entire screen.
  • LET A=32768: POKE 23607,A/256: POKE 23606,A-(PEEK 23607 * 256)

A few pokes to stop people messing with your BASIC programs!

  • POKE 23756,n to change the first line number to n. If you set this to 0 then, it is not editable without reversing the poke! In 128K mode it lists line 0 over and over and can cause the editor to crash while adding new lines!
  • POKE 23613,82 will disable the BREAK key in your BASIC program
  • POKE 23613,0 causes BREAK to crash.
  • POKE 23755,100 causes the program listing to be invisible on BREAK while running (use POKE 23755,0 to get it back)

Header-less tape loading and saving…

LD A,255
LD IX,@START
LD DE,@LENGTH
SCF
JP 1218 ; Save or
JP 1366 ; Load

Enjoy,

[)amien

My own Delphi story - celebrating 25 years

Setting the scene

It’s 1995, and a wiry-looking engineer in need of a haircut is working at a tech services company in the Channel Islands. The island is Jersey (you can see the French coastline on a clear day), and he’s over here for a week or two for training from his nearby Guernsey home.

This company does everything from IBM AS/400 maintenance to custom PC software development - the developer involved was brought in and trained to work on a banking package on those AS/400s but has ended up on the PC development side through a series of improbable events.

Visual Basic 3 has been the company tool of choice for these tasks - supplemented with Microsoft Access for the data-driven projects (usually backed by Microsoft’s brand-new SQL Server, which is taking over from Sybase installs). He’s been helping work on the in-house “In/Out” software the company uses to track who is in the office vs. who is out on calls (this is 18 months before the first mainstream instant-messenger product - ICQ - will appear and offer similar functionality).

He’s also been working on a document imaging and indexing solution (iStation) for two local government organizations (they love to make a solution for one company then tweak it for others like many ISVs), and that is the limit of his VB experience. He’s faced the insurmountable wall of Borland C++ and OWL enough times to feel that when it comes to VB… “This is the way.”

Arrival

I’m kicking around in the narrow row of desks - unlike the sales team; we’re not important enough to get dividers between our desks to turn them into semi-cubes.

Screenshot of Delphi 1 IDE in action A colleague named Mike strolls in, drops his bag and announces ambiguously, “I got it,” and proceeds to power on his PC. “It” is a copy of Delphi - and a bootleg pre-release version at that (if I recall correctly).

I watch and ask questions as he and a few other developers crowd around his giant 17” CRT to check it out. At first glance, it looks like Visual Basic with a few more included controls, a slicker IDE, but a more confusing language. Over the week that follows, he keeps showing us discoveries, and interest increases.

Before the week is up, I have a ZIPped copy of my own to carry home.

Back home, I get past the confusion of unit dependencies and Pascal idiosyncrasies like := for assignment and dig deeper, discovering the beauty of the VCL - Visual Component Library.

My journey is aided by a curious programming book written in a style I have never come across before or since. It is a humorous tale of a private detective also writing Delphi apps on the side. It is called the Delphi Programming Explorer and helps me get over the hump.

The VCL is hard to describe if you’ve never used it, but I’ll give it a shot. It was basically like the VB controls, but they gave you the source code, and there was a proper hierarchy to it. Up to this point in my life, I thought Object-Oriented meant you had classes and instances of classes. I had no idea about interfaces, inheritance, or good OO design, but Delphi and the VCL introduced it.

I was hooked. When Delphi 2 came out, it was time to dig very deep and buy my copy.

Using it

 Icons for controls in the Envy Delphi Development pack

Over time I pumped out an About95 control, then a Delphi Control Pack that included a font combo box, a gradient control, an improved list view, a button that popped out a menu control, a splitter control, a color combo box. Useful functionality that was otherwise missing but often seen in other non-Delphi apps.

I was shocked years later when I picked up the book Developing Custom Delphi 3 Components that my controls, and source, were included on the CD.

It received so many downloads that my local ISP got in contact and asked me very nicely, in the absence of any formal limits on bandwidth back then, to host the ZIP elsewhere. That and a few other tools I’d uploaded to my free personal site were now responsible for a significant percentage of their bandwidth. Double digits if my memory serves me correctly, and so relocated to Demon Internet.

Hot off the heels of that “success,” I tried my hand at shareware again with the Envy Delphi Development pack. Polished and extended versions of the previous release now accompanied by a host more. An image view control, a database-bound label, a progress bar, shading control (like Borland C++), a tip-of-the-day control that needed nothing more than a text file of tips, a frame control to allow inset/outset and a bitmap tiling control for more texture. At this point, icons are sufficiently basic in the world that I can mostly get away with designing my own. I was a team of one.

I also put out my first proper shareware tool that provided a trickle of income for a while in the form of my Windows monitoring tool for “Remote Access BBS” called Monitor/RA. At one point, Diamond Multimedia purchased a commercial license to monitor their European support BBS.

Later on, I’d even make some controls that drew themselves looking like MacOS Platinum and NextStep.

My site back then had a heavy Delphi focus and is in love with small fonts, tiny icons, pastel shades, and almost-invisible watermark textures.

Alternatives

I never managed to get into Java desktop development despite many attempts. The screenshots looked good, but it never felt properly native-like Delphi did. Having to install a non-trivial runtime was also off-putting, especially for these small side-projects.

Borland C++ still felt like a large mountain to climb, and I had no reason, and so those books stayed on my shelf for many years in the “I should get to this.” Of course, OWL died as Microsoft Visual C++ came up - a product that has never really delivered on the Visual unless you were prepared to commit to another mountain in the form of the MFC.. the mountain might have been climbable with a visual designer but, one never materialized.

Visual Basic was hard to go back to. It immediately felt limiting, and having to learn COM to develop controls was a show-stopper. After I had picked up Delphi 2, in fact, I never touched VB again as far as I can recall. I instead spent a lot of time in Microsoft Access connected to SQL Server, then onto ASP, and then finally ASP.NET.

I remember being excited when WinForms was announced. C# is a language I came to love more than Pascal, and WinForms was a definite improvement over VB - you could now easily write your own controls, but .NET Framework eschewed object-orientated design and patterns and instead just went with some inheritance and helper methods - ironically something Java did better. Comparing IO in .NET vs. IO in Java also highlights this - the latter more than happy to provide wrappers and layers for great composition vs. .NET lumping tons of stuff into static methods on File and providing a bunch of Readers you can never remember.

During my degree, we used Borland C++ builder, which took the VCL and provided you with the ability to use C++ instead of Pascal. I enjoyed using that, and just this year, I’ve found myself wanting to create a single-executable solution, but C++ was again something nobody in my circles was needing.

Legacy

Screenshot of Disk Image Managers sector overview

I kept buying copies of Delphi (at every alternate release, it’s my own money) and found myself creating a tool for working with disk images of 3” floppy disks (yes, not 3.5”) used by Sinclair, Amstrad, and Tatung machines. It was aimed at the archiving community, and Delphi was the tool I fell back to - I had zero interest in telling people to install large runtime packages. It’s still maintained today, although it was ported to the free-software Lazarus alternative.

At one point, I also worked with a team on a ZX Spectrum emulator developed in Delphi. Their diving into x86 assembler at the tip of a hat irked me. I wanted to work on something maintainable with cool features rather than cater to a small vocal crowd of people on lower-powered PCs. I didn’t know x86, and I had no desire to learn it - my brief encounter with 8051 had been enough.

Pascal always felt a little awkward to me, but the VCL, IDE, and those self-contained small executables (200KB in initial versions, IIRC) were worth staying for. Living on a small island meant that programming jobs were scarce and almost exclusively limited to COBOL on AS/400’s or Microsoft tools on Windows.

The draw for me was always this idea of small standalone exes, which is hard to achieve on Windows with a decent GUI builder. There are third-party options, but every one of them seems to fail to provide the seamlessly enjoyable experience Delphi and the VCL did, and many of them are either poorly maintained or can cost a small fortune.

Interestingly Delphi and C++ Builder and now available in free community editions but confusing bundled together as “RAD Studio.” I’ve wanted to write a specific Windows native app of late that needs to call a bunch of Windows APIs, and so C++ Builder is back on my machine. (I wish the small binaries of old hadn’t become megabyte monsters… I’m sure I don’t need most of what’s in there)

[)amien

I know some of this is a bit gushing; that’s nostalgia for you, I guess. I can at least assure you that I receive no compensation/kickback/affiliate from Embarcadero, who currently own Delphi/C++ Builder and RAD Studio (I had to Google their name, seriously).