I'm often digging into old bitmap font and UX design out of curiosity - and someday hope to revive a lot of these fonts in more modern formats using a pipeline similar to that for ZX Origins so we can get all the usable fonts, screenshots etc. out of them.

One limitation I've run into is digging into old Macintosh fonts. While James Friend's PCE.js puts System 6 and System 7 at your fingertips when it comes to later 7.5, 8 or 9 the site doesn't have you covered as PCE doesn't support PowerPC emulation (it handles Motorola 68000 and Intel 8086 processors).

This is a shame for me as that's where the interface started diverging by adding color and some more interesting fonts. Additionally some third-party fonts are distributed in .sit (StuffIt Expander) or only work with later Mac OS versions.

Enter QEMU

Thankfully QEMU has us covered. It's an open-source emulator that unlike regular virtualization tools is quite capable of emulating completely different CPU architectures from ARM through to MIPS, PowerPC, RISC-V, Sparc and even IBM's big s390x z/Architecture.

With such a wide variety of options and settings available you can imagine it will require some digging through the user interface and you'd be wrong. There is no GUI however and the third party ones that exist mostly seem to be from 10 to 2 years out of date and many don't support Windows at all. The only "up to date" one I found - QtEmu - only supports configuring x86 virtual machines.

This is a shame as although I love the command-line for its scriptability when it comes to exploring valid combinations of options the command-line is mostly awful (the IBM AS/400 command-line and prompting system excluded).

You could try and build this yourself but Stefan Weil has you covered for pre-built QEMU Windows binaries

Please note that Mac sound support is missing here. There are "screamer" forks but the only binaries available are for Mac OS X so you'd have to build it yourself and there will probably be a whole lot of hoops to jump through.

Obtaining an OS install image

To install Mac OS 9 we're going to need a disk image/ISO to install from.

The nice people over at Mac OS 9 Lives have a Mac OS 9.2.2 Universal Installer ISO which is pre-configured and easy to use - it also conveniently includes a few extra tools and apps you'll need.

While Mac OS is copyrighted this image has been up for over 6 years so I like to think Apple are turning a blind eye in that people who want to use their legacy stuff can do so without expecting support from Apple - win-win. They also don't charge for their Operating Systems instead it's "free" with the hardware and I still have a MacBook Pro 15" so I won't feel bad about using it. Your mental mileage may vary.

Creating a machine & installing Mac OS 9

First off create a new folder to put your machine config into. (Windows won't let you stuff it into Program Files). I've chosen c:\retro\mac.

Now let's create an empty hard-drive image file:

cd "c:\program files\qemu"
qemu-img create -f qcow2 c:\retro\mac\MacOS9HD.img 5G

This creates a virtual hard-drive that can grow up to 5 GB in size and will allocate disk as it needs (copy-on-write). After installation this file will grow to about 660 MB.

Now before we go further remember the keyboard-shortcuts:

• CtrlAltG as you'll need it to get the mouse back
• CtrlAltF to get you in and out of full-screen mode

Now lets start the installer:

qemu-system-ppc -cpu "g4" -M mac99,via=pmu -m 512 -hda c:/retro/Mac/MacOS9HD.img -cdrom "c:/retro/Mac/Mac OS 9.2.2 Universal Install.iso" -boot d -g 1024x768x32 -device usb-kbd -device usb-mouse -sdl

This specifies that we want to use:

• a PowerPC G4 900MHz CPU -cpu g4
• a PowerMAC based Mac with USB support -M mac99,via=pmu
• 512 MB of RAM -m 512
• Our hard-drive image -hda xxx
• Our Installation CD mounted -cdrom xxx
• Boot from CD -boot d
• 1024x768 32-bit display -g 1024x768x32
• a USB keyboard -device usb-kbd
• a USB mouse -device usb-mouse
• SDL display buffer -sdl

There are many other useful config switches available for the PowerPC emulation if you need to troubleshoot or tweak. The final item, SDL, is required because the default GTK emulation, while faster, has major problems on Windows trying to keep the mouse captured.

You should now be presented after a few seconds with a ReadMe. Just close that with the top-left window control then click into the Drive Setup window, select <not initialized>, press the Initialize button then confirm it with the default on the subsequent Initialize window.

You might now want to click the "untitled" hard drive icon that's appeared on the desktop, wait a few seconds and you should be able to rename it. Typically Macintosh HD is a popular choice.

Head up to the MacOS9Live CD icon, double click it then double-click on Apple Software Restore. The following Window will appear. You can accept all the defaults or just change Volume Format to Extended - I did this just in case I want to try and mount the image on my MacBook at a later date.

Clicking Restore, then confirming the dialog will give you a progress bar that is comically fast for installing an operating system (via software emulation no less).

Now you'll need to head to the Special menu and choose Shut Down.

Using our virtual Mac OS 9

Finally, we want to start our freshly created machine without booting from the ISO. The command line is mostly the same just omitting the ISO and boot-from-CD options:

qemu-system-ppc -cpu g4 -M mac99,via=pmu -m 512 -hda c:/retro/Mac/MacOS9HD.img -device usb-kbd -device usb-mouse -sdl

You'll probably want to put that in a shortcut icon.

Here's a PowerMac G4 "Quicksilver" Icon (128 KB) I put together.

When it boots for the first time you'll get a Register With Apple "wizard". Just press WindowsQ to quit this and get to that Platinum desktop!

Head to the Control Panel's Monitors applet to set the screen size/resolution you want. You might also want to head into Appearance applet's Fonts tab to turn off anti-aliasing so you can enjoy the fonts in their pixel-glory. (You can also switch from the revised Charcoal front back to the classic Chicago font here). You may also have to switch screen-resolution again if you see some odd artifacts/missing/doubled pixels when turning it off. (There's a quick resolution changer on the control-strip in the lower left, it's the one with the checkerboard effect)

Remember to always shut-down correctly! Use the switcher at the top-right to "switch" to Finder then go through that Special, Shutdown process each time. QEMU will close several seconds after it's complete.

On your hard-drive you'll find an Applications folder, dig into Internet Utilities, Classilla folder and you can launch Classilla which is a port of the Netscape browser made in 2014 (based on Netscape Navigator 1.3.1 Nokia N90 port). It was a valiant effort given how diferent Mac OS development was prior to Mac OS X - there were no Unix libraries/support so ports were difficult and most applications were written in MetroWerks CodeWarrior - the "classic" Mac OS version was discontinued in 2002.

Still Classilla is much better than IE 5.5 which fails to do anything at all. Google works, for example, but many sites don't render at all because of the push to later versions of SSL the browser does not support.

You can find a ton of old Mac software at The Macintosh Repository but there are no more capable browsers.

Still, it's a fun environment to play with and it's nice to have 100% accurate references to Geneva, Chicago, Monaco, Espy Sans etc. as most "conversions" tend to be hand-converted and mistakes are a-plenty. I've done a few conversions myself this way on FontStruct and know how easy it is to make mistakes when working from screenshots especially when it comes to spacing between letters.

It's also nice to see an old friend again. Despite regularly finding myself on retro machines and emulators spanning 8 and 16 bit machines I don't have (or have the space for) a classic Mac and emulation has been difficult. I think I last used Mac OS 9 in 2000 on an iMac at work before we put the Mac OS X Public Beta on it (I was a bit NeXT/OpenStep fan and wanted to see what they had done to it!)

My thanks to James Badger for his general article on Mac OS 9 on QEMU.

[)amien

As everybody who has read my blog before knows, I love LINQ and miss it when coding in other languages, so it's nice when I get a chance to use it again. When I come back to it with fresh eyes, I notice some things aren't as easy as they should be - and this time is no exception.

Background

People often need to build up LINQ expressions at runtime based on filters or criteria a user has selected. Adding criteria is incredibly easy, as you can chain operations together on the IQueryable interface, e.g.

if (customerActive)
  query = query.Where(c => c.IsActive);
if (customerCountry != null)
  query = query.Where(c => c.Country == customerCountry);

This is great if you want to AND things together, which is often the case in building up filters. Other operations, like Contains, are useful in allowing many options against a specific field.

Problem

But what about offering a choice involving either of two properties? One way to write it would be:

if (customerActive) {
  if (customerEnterprise) {
    query = query.Where(c => c.IsActive || c.IsEnterprise);
  } else {
    query = query.Where(c => c.IsActive);
  } else {
    if (customerEnterprise) {
      query = query.Where(c => c.IsEnterprise);
    }
  }
}

This code is already hard to read and is going to increase in complexity for each new option.

Solution

With some helper code, we can combine two expression predicates into a single OrElse for use in a Where condition.

class Or {
  public static Expression<Func<T, bool>> Combine<T>(Expression<Func<T, bool>> left, Expression<Func<T, bool>> right) {
    if (left == null && right == null) throw new ArgumentException("At least one argument must not be null");
    if (left == null) return right;
    if (right == null) return left;

    var parameter = Expression.Parameter(typeof(T), "p");
    var combined = new ParameterReplacer(parameter).Visit(Expression.OrElse(left.Body, right.Body));
    return Expression.Lambda<Func<T, bool>>(combined, parameter);
  }

  class ParameterReplacer : ExpressionVisitor {
    readonly ParameterExpression parameter;

    internal ParameterReplacer(ParameterExpression parameter) {
      this.parameter = parameter;
    }

    protected override Expression VisitParameter(ParameterExpression node) {
      return parameter;
    }
  }
}

Now we can:

Expression<Func<Customer, bool>> criteria = null;
if (customerActive)
    criteria = Or.Combine(criteria, c => c.IsActive == customerActive);
if (customerEnterprise)
    criteria = Or.Combine(criteria, c => c.IsEnterprise == customerEnterprise);

How it works

There are two parts to this working. The first combines the two separate predicates into a single OR lambda, taken care of by the Expression.OrElse method.

That alone can't quite do what we need because the two expressions each have a separate parameter variable. The visitor replaces both of these with a new parameter we'll pass into the new Lambda expression we're creating that combines both conditions.

Caveats

The code, as it stands, creates an unbalanced tree. Ideally, the tree should be balanced to limit the depth LINQ providers must traverse to translate the query. I doubt you'll run into the dreaded Stack Overflow (the message, not the site), but if you do, that's why.

An overload that takes an Enumerable of Expression<Func<T, bool>> and produces a balanced tree is left as an exercise to the reader :p

[)amien

I've been a big fan of static site generation since I switched from WordPress to Jekyll back in 2018. I'm also a big fan of learning new technologies as they come along, and now GitHub Actions are out in the wild; I thought this would be an opportunity to see how I can port my existing custom CircleCI build to Jekyll.

The CircleCI job

A quick recap from part 5 - Hosting & Building, my CircleCI configuration was basically two jobs that have subsequently been tweaked since then. They are:

Build

The build job's responsibility was to configure a Ruby environment capable of executing Jekyll to build the site, removing the .html extension from the output filenames, and then indexing the content using Algolia. Here is the start of the .circleci/config.yml I was using for that:

{%raw%}version: 2
jobs:
  build:
    docker:
      - image: circleci/ruby:2.6.1
    working_directory: ~/jekyll
    environment:
      - JEKYLL_ENV=production
      - NOKOGIRI_USE_SYSTEM_LIBRARIES=true
      - JOB_RESULTS_PATH=run-results
    steps:
      - checkout
      - restore_cache:
          key: jekyll-{{ .Branch }}-{{ checksum "Gemfile" }}
      - run:
          name: Update gems
          command: gem update --system
      - run:
          name: Install dependencies
          command: bundle check --path=vendor/bundle || bundle install --path=vendor/bundle --jobs=4 --retry=3
      - save_cache:
          key: jekyll-{{ .Branch }}-{{ checksum "Gemfile" }}
          paths:
            - "vendor/bundle"
      - run:
          name: Create results directory
          command: mkdir -p $JOB_RESULTS_PATH
      - run:
          name: Build site
          command: bundle exec jekyll build --config _config.yml,_config-publish.yml 2>&1 | tee $JOB_RESULTS_PATH/build-results.txt
      - run:
          name: Remove .html suffixes
          command: find _site -name "*.html" -not -name "index.html" -exec rename -v 's/\.html$//' {} \;
      - run:
          name: Index with Algolia
          command: bundle exec jekyll algolia --config _config.yml,_config-publish.yml
      - store_artifacts:
          path: run-results/
          destination: run-results
      - persist_to_workspace:
          root: ~/jekyll
          paths:
            - _site
{%endraw%}

Deploy

Deploy takes the output from the build job and syncs it with the S3 bucket I use to publish the site. It then applies AWS S3-specific commands using the AWS CLI tool to ensure metadata, redirects, and caching are correctly set using a Python environment.

{%raw%}  deploy:
    docker:
      - image: circleci/python:2.7
    working_directory: ~/jekyll
    steps:
      - attach_workspace:
          at: ~/jekyll
      - run:
          name: Install AWS CLI
          command: sudo pip install awscli
      - run:
          name: Deploy to S3
          command: aws s3 sync _site s3://damieng-static/ --delete --content-type=text/html
      - run:
          name: Correct MIME for robots.txt automatically
          command: aws s3 cp s3://damieng-static/robots.txt s3://damieng-static/robots.txt --metadata-directive="REPLACE"
      - run:
          name: Correct MIME for sitemap.xml automatically
          command: aws s3 cp s3://damieng-static/sitemap.xml s3://damieng-static/sitemap.xml --metadata-directive="REPLACE"
      - run:
          name: Correct MIME for Atom feed manually
          command: aws s3 cp s3://damieng-static/feed.xml s3://damieng-static/feed.xml --no-guess-mime-type --content-type="application/atom+xml" --metadata-directive="REPLACE"
      - run:
          name: Redirect /damieng for existing RSS subscribers
          command: aws s3api put-object --bucket damieng-static --key "damieng" --website-redirect-location "https://damieng.com/feed.xml"
      - run:
          name: Latest Envy Code R redirect
          command: aws s3api put-object --bucket damieng-static --key "envy-code-r" --website-redirect-location "https://damieng.com/blog/2008/05/26/envy-code-r-preview-7-coding-font-released"
      - run:
          name: Latest Envy Code R redirect #2
          command: aws s3api put-object --bucket damieng-static --key "fonts/envy-code-r" --website-redirect-location "https://damieng.com/blog/2008/05/26/envy-code-r-preview-7-coding-font-released"
      - run:
          name: Latest Envy Code R download
          command: aws s3api put-object --bucket damieng-static --key "downloads/latest/EnvyCodeR" --website-redirect-location "https://download.damieng.com/fonts/original/EnvyCodeR-PR7.zip"
      - run:
          name: Correct MIME for CSS files
          command: aws s3 cp s3://damieng-static/css/ s3://damieng-static/css/ --metadata-directive="REPLACE" --recursive
{%endraw%}

GitHub Actions

So how could I go about this in GitHub Actions? I have to admit I spent far too long poking around and examining existing Jekyll actions. I have a bunch of steps here I need fine control of, especially around Algolia and S3. I finally ended up on what was quite a simple port.

Unlike the CircleCI configuration, I did not split these into two separate jobs because:

1. There is no exact equivalent to persist_to_workspace and attach_workspace
2. The alternative of storing and restoring the artifacts leaves large useless artifacts around
3. I never ended up running the jobs separately
4. GitHub Actions provides an environment with both Ruby and AWS CLI installed

So, on to the configuration which lives in .github/workflows/jekyll.yml in my case:

{%raw%}name: Build site and deploy

on:
  push:
    branches: [ master ]

jobs:
  build:
    name: Build + Deploy
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2

      - name: Setup Ruby
        uses: ruby/setup-ruby@v1
        with:
          ruby-version: 2.6.1

      - name: Ruby gem cache
        uses: actions/cache@v1
        with:
          path: vendor/bundle
          key: ${{ runner.os }}-gems-${{ hashFiles('**/Gemfile.lock') }}
          restore-keys: |
            ${{ runner.os }}-gems-

      - name: Install dependencies
        run: |
          bundle config path vendor/bundle
          bundle install --jobs 4 --retry 3

      - name: Build Jekyll site
        run:  bundle exec jekyll build --config _config.yml,_config-publish.yml

      - name: Remove .html suffixes except for index.html
        run: find _site -name "*.html" -not -name "index.html" | while read f; do mv "$f" "${f%.html}"; done

      - name: Index with Algolia
        env:
          ALGOLIA_API_KEY: ${{ secrets.ALGOLIA_API_KEY }}
        run:  bundle exec jekyll algolia --config _config.yml,_config-publish.yml

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1

      - name: Sync site with S3
        run:  aws s3 sync _site s3://damieng-static/ --delete --content-type=text/html

      - name: Correct MIME types
        run: |
          aws s3 cp s3://damieng-static/ s3://damieng-static/ --exclude "*" --include "*.txt" --metadata-directive="REPLACE"
          aws s3 cp s3://damieng-static/sitemap.xml s3://damieng-static/sitemap.xml --metadata-directive="REPLACE"
          aws s3 cp s3://damieng-static/feed.xml s3://damieng-static/feed.xml --no-guess-mime-type --content-type "application/atom+xml" --metadata-directive "REPLACE"
          aws s3 cp s3://damieng-static/css/ s3://damieng-static/css/ --metadata-directive "REPLACE" --recursive
          aws s3 cp s3://damieng-static/js/ s3://damieng-static/js/ --metadata-directive "REPLACE" --recursive

      - name: Redirect /damieng for existing RSS subscribers
        run:  aws s3api put-object --bucket damieng-static --key "damieng" --website-redirect-location "https://damieng.com/feed.xml"

      - name: Latest Envy Code R redirects
        run:  |
          aws s3api put-object --bucket damieng-static --key "fonts/envy-code-r" --website-redirect-location "https://damieng.com/blog/2008/05/26/envy-code-r-preview-7-coding-font-released"
          aws s3api put-object --bucket damieng-static --key "downloads/latest/EnvyCodeR" --website-redirect-location "https://download.damieng.com/fonts/original/EnvyCodeR-PR7.zip"

      - name: Set caching for images at 30 days
        run:  aws s3 cp s3://damieng-static/ s3://damieng-static/ --exclude "*" --include "*.svg" --include "*.ico" --include "*.jpg" --include "*.png" --include "*.webp" --include "*.gif" --recursive --metadata-directive REPLACE --expires 2034-01-01T00:00:00Z --acl public-read --cache-control max-age=2592000,public

      - name: Set caching for CSS and JS at 1 hour
        run:  aws s3 cp s3://damieng-static/ s3://damieng-static/ --exclude "*" --include "*.css" --include "*.js" --recursive --metadata-directive REPLACE --expires 2034-01-01T00:00:00Z --acl public-read --cache-control max-age=3600,public
{%endraw%}

I also took the opportunity to fix a long-running issue in that my S3 objects would lose my manually-applied cache settings (and new posts and files would not have any).

Gotchas

There were only a couple of bumps in the road once I decided on a straight-port rather than trying to leverage higher-level existing actions:

1. The secrets configured in the repo settings were not automatically exposed to the commands running in the action. Instead, you have to expose them using the ${{ secrets.KEY_NAME }} syntax.
2. I was using rename instead of mv. I don't recall why. Perhaps it was my Windows-ness creeping in. Rename has been dropped in newer distros.

Summary

The syntax is surprisingly similar with build-times about the same as CircleCI. It's just nice to have it in one place. Time to port some other repos over!

[)amien

I've recently been working on a full HTML5 conversion of the Sinclair Spectrum +3 manual with full canvas-drawn screenshots and diagrams for smooth scaling/high res displays as well as some close font matching and layout as well as cross-reference links all over the place.

My ZX Spectrum +3 Manual conversion is now available!

What I wanted to show today was the odd commands, pokes and outs that were hastily scribbled down in the back of my manual over the years.

Spectrum +3/+2A specific

• You know you can type SPECTRUM to go from +3 BASIC to 48K BASIC but, did you know you can get back? RANDOMIZE USR 23354. It leaves CHR$ 163/164 as UDGs.
• Switch off the +3 drive with POKE 23399,4. In 48K mode, you can use POKE 23611,221 if it somehow stayed on.

128K specific

• 128K machines are very compatible but the last two user-defined graphics (UDGs) T and U have become the SPECTRUM and PLAY command (for switching to 48K mode and using the AY-3-8912 sound chip). You can, however, switch them back to UDGs using POKE 23611,205.
• POKE 23611,205; RANDOMIZE USR 4867; alternative way to switch from 128K to 48K.

A couple of VTX-5000 modem specific hacks:

• OUT 8189,0 to switch the 128K ROM back in.
• POKE 23611,29 also to switch from 48K to 128K mode.

All machines

• POKE 23617,n changes the input mode for INPUT prompts. 0=Alpha (C/L), 1=Extended (E), 2=Graphics (G), 4=Keyword (K).
• POKE 23658,0 turns off CAPS LOCK while POKE 23658,1 turns it on.
• POKE 23692,255 will suppress the Scroll? prompt 255 times so you can auto-scroll.
• RANDOMIZE USR 3582 to scroll the screen one line.
• RANDOMIZE USR 3330 to scroll the entire screen.
• LET A=32768: POKE 23607,A/256: POKE 23606,A-(PEEK 23607 * 256)

A few pokes to stop people messing with your BASIC programs!

• POKE 23756,n to change the first line number to n. If you set this to 0 then, it is not editable without reversing the poke! In 128K mode it lists line 0 over and over and can cause the editor to crash while adding new lines!
• POKE 23613,82 will disable the BREAK key in your BASIC program
• POKE 23613,0 causes BREAK to crash.
• POKE 23755,100 causes the program listing to be invisible on BREAK while running (use POKE 23755,0 to get it back)

Header-less tape loading and saving...

LD A,255
LD IX,@START
LD DE,@LENGTH
SCF
JP 1218 ; Save or
JP 1366 ; Load

Enjoy,

[)amien