Bitmap v scalable fonts

Bitmap fonts are incredibly easy to make. Using a program like Softy or BitFonter you decide the size of your letters and start plotting pixels. You can see exactly how it will look because you draw every glyph (letter/symbol/number) in every size you want to support. This can obviously be very time consuming and doesn’t let you take full advantage of the resolution of the device and the capabilities it offers. A printer can handle in excess of 300 dpi while a display is typically 72 dpi (Mac) or 96 dpi (Windows) with LCD’s supporting sub-pixels due to the individual layout of the red-green and blue elements you can’t feasibly pre-plot every single combination and even if you could the file size would be rather large.

Rather than having specific set of pixels to turn on or off TrueType, OpenType and PostScript fonts contain a series of instructions that tell the computer the shape using a series of points, lines and curves. This means the computer can scale the glyph to the size that is required and then take full advantage of the device being rendered honoring the users preferences for anti-aliasing (smoothing using shades of grey), sub-pixel precision (smoothing using hints of red, green and blue to take advantage of the layout of colour elements in an LCD display), desired contrast and gamma settings etc.

Grid fitting

Such a scaled glyph won’t fit perfectly within a pixel grid and a small sizes and low resolution it can look awful. It is also necessary to ensure that the vertical part of the letter I (known as a stem) looks very similar to the stem of other letters at the same size – we don’t want some letters looking bold – and that the top of the letter o aligns nicely with the top of the i etc. (in most fonts). The glyphs themselves don’t know what is a stem, what should align with other glyphs etc.

Many renders include logic to try and improve un-hinted fonts such as the drop-out control in Windows through to the full auto-hinter in FreeType. If you’ve ever used free fonts from any of the numerous web sites around you’ve probably seen that it doesn’t get it right and it looks like this: 

The first few versions of Envy Code R looked like that because to address this problem you need to learn a process called hinting, which let’s the designer give the renderer “hints” on how to choose the pixels.

Hinting

Font hinting started off as stem and edge identification so that glyphs would maintain the right proportions when sized and rendered on these low-DPI devices. It became apparent that a much more fine-grained level of control was required and so a stack-based byte-code language was developed as part of the TrueType specification to allow designers finer control in how points are adjusted to better take advantage of the display characteristics.

A TrueType font can contain extra blocks which describes, using a sequence of bytes that represent instructions and their arguments, the process by which to align the points and therefore make decisions about how best to fit the letter into the grid by retaining and adjusting various elements.

The important blocks are:

Each instruction (opcode) has a mnemonic that is representative of what it does and these are documented in Chapters 5 through 6 of the TrueType specification (along with much other useful relevant information). Actual per-glyph instructions are stored with each glyph outline in the glyp block.

Rasterizing & rendering

There are many different ways a TrueType font can end up on your screen with a lot of variants between how vendors chose to render the font and what options they expose to developers and users to fine-tune the experience.

• Windows – User choice of 1-bit, 4-bit grey-scale anti-aliasing, ClearType, ClearType tuning and display DPI plus WPF and DirectWrite per-app options
• Mac OS X – User choice of sub-pixel anti-aliasing strength and 1-bit cut-off plus per-app 1-bit option (e.g. Terminal)
• Java – Per-application choice of 1-bit, grey-scale or sub-pixel rendering
• Flash – Per-application choice of 1-bit or grey-scale
• FreeType – Rendering library that exposes a number of runtime and compile-time settings

This is of course ignoring the other rendering engines out there such as the Adobe’s Photoshop, RiscOS, D-Type rendering engine, Font Fusion (used on BeOS) etc. and prior versions of those renderers listed above (Flash and Mac OS changed significantly). Getting it pixel-perfect on every combination is impossible but we can try :)

Tools

Instructing fonts is a painstaking process at the best of times and few people deal directly with the low-level instructions instead relying on tools, stem identification and higher-level languages to achieve the same result. Some tools that have support for hinting instructions are:

• FontLab Studio 5
  Comprehensive font-production package for Windows and Mac that includes auto-hinting and it’s own higher-level link language that it can generate TrueType instructions from but it does not support viewing or modifying existing TrueType instructions and does not handle diagonals well. Rendering preview includes mono, grey-scale and ClearType. (Commercial $649)
• Fontographer 4.1
  Rather dated font-production package for Windows and Mac. (Commercial $349)
• FontForge
  Comprehensive font-production package that runs on X11 that includes auto-hinting and the ability to disassemble and edit existing TrueType instructions as well as debug them with stepping. Includes basic mono/grey-scale rendering options. (Open source)
• Microsoft Visual TrueType
  Hinting instruction tool from Microsoft that uses it’s own higher-level VTT Talk language that compiles down to TrueType instructions that you can further edit. Includes a comprehensive set of preview rendering options but is not capable of disassembling existing instructions. (Commercial, free with signed licence agreement)
• TTX
  Python scripts that can convert a font into an editable XML representation and back including disassembly and assembly of TrueType hinting instructions. (BSD)
• TTIComp
  Command-line tool that provides an alternative C-like hinting language. (GPL)
• Xgridfit
  FontForge scripts to provide an alternative XML-based hinting language. (GPL)
• TrueTypeViewer
  Windows tool for displaying TrueType fonts and glyphs including debugging and descriptive disassembly of instructions. (GPL)
• TTHMachine
  Real-time editing of hinting instruction mnemonics and observing their effects which is useful for learning. (Free, no longer supported)

[)amien

These discoveries form the basis of much of the technology we use today and include the desktop metaphor, the graphical user interface, laser printers, object orientation and Ethernet.

Xerox manufactured a number of high-end machines including the 1973 Xerox Alto which, being GUI based, shipped with a number of proportional bitmapped fonts.

What is interesting to me however is the mono-spaced font used by the SWAT debugger (but not by the command prompts, they were proportional – ahead of their time!) and so, based on a screen-shot of SWAT, I thought it needed to live again!

I’ve had to make up a few of the symbols and letters that weren’t shown and filled out the symbols for the Windows 1252 Latin-1/ISO-8990-1 code-page and with the absence of any solid information online give it a name so here is Alto Mono!

Download Alto Mono (TrueType, Windows FON, BDF) (30 KB)

When using the TrueType version choose 6 point on Windows and 8 point on Mac OS X.

The Xerox manuals are also fun to browse though with such section headings as “Things the user doesn’t really need to know…” and “How to get out of trouble” and the comments about SWAT’s odd syntax and interface.

Don’t forget to check out my reproduction of the PalmOS system font. Not monospaced but very clear at small sizes – great for the Visual Studio output window ;-)

[)amien

It’s been a struggle but finally after countless hours here it is, the next release of my Envy Code R monospaced (fixed-width) font designed for programmers.

Many glyphs have been redrawn since preview #6 including braces, lower-case y, 6 & 9, ampersand, dollar-sign, hash etc. One pixel was removed vertically height to make the box drawing balanced and allow more lines per screen.

These new box-drawing, shading and symbols make Envy Code R a great font for the command-prompt (Consolas and Lucida Console lack box-drawing completely). To use them you will need to run the included registry file and reboot to operate correctly from a command prompt’s properties dialog.

This typeface contains over 550 glyphs providing full complements for DOS, Windows and Mac versions of the US, Western, Central Europe, Turkish, Baltic, Icelandic and Nordic code-pages. This hits several Unicode ranges including Basic Latin, Latin-1 Supplement, Latin Extended A & B, Box Drawing, Block Elements, Letterlike Symbols, Number Forms, Arrows… although not all of these ranges are complete yet.

As well as regular and bold variants this version includes a full italic version too and the obligatory italic-as-bold hack to get italic syntax highlighting in Visual Studio as shown here in my favourite 10 point with my Humane theme.

And for those of you that like the font a little larger it now looks good and the odd sizing issues are all gone!

Okay, enough with the teasing, you’ve waited far too long…

Download Envy Code R Preview #7.2 (TrueType) (169 KB)

These files are free to download and use from damieng.com but CAN NOT be redistributed either by other web sites or be included in your package, download, product or source repository at this time.

[)amien

Joining Microsoft

For those who haven’t been following my blog long I took a job at Microsoft Canada Development Centre as a developer on LINQ to SQL. It turns out my H-1B Visa has been approved and I will be moving down to Redmond in October.

Joining a company of Microsoft’s size is a daunting experience. The sheer number of people, departments, systems, procedures and intranet sites to navigate and learn plus of course the actual job of jumping into the product and seeing where we go from here. I’ve also been helping out a little on the forums and internal lists and getting involved in the regular scheduled update meetings.

Of course you also hear all sorts of interesting news just before it becomes public knowledge such as publishing XNA apps to Xbox Live! and Office getting ODF and PDF support.

On the personal front…

A whirlwind couple of weeks full of new employee orientation, relocating, getting lost, filling in forms, exploring, meeting a couple of hundred people and catching up with a few old friends including one from Guernsey all of which lead to a quiet blog.

There have been some personal stories of getting lost, baby sharks and falling in lakes which will be kept to email now – there’s no way those 500+ subscribers are here for my personal bits! I’ll be sending out an email this week so if you haven’t seen something by the weekend and we’re friends ping me and I’ll forward you on a copy.

Some photos are up on Facebook with a few more to follow.

Envy Code R

Of course what everybody really wants to know (according to my inbox) is where Envy Code R preview #7 is.

It is coming, but every time I think I’m close to a release I find another annoying glitch all related to hinting.

Hinting is the process whereby you tell the rendering system how to shape the characters to better fit into a pixel grid. It consists of a table saying at which sizes to smooth and apply instruction plus a program that adjusts the font as a whole for a given size and then a program per-glyph that tells it how to adjust the points in relation to each other with delta hints providing modifications for specific point sizes.

It’s a complicated process if you’re doing it at the lowest level with a tool such as Microsoft’s Visual TrueType but is made easier with a tool like FontLab Studio 5 which has an autohinter that often gets things wrong but is a lot easier to work with and works with hints at a higher level of abstraction.

Which is why I parted with $999 on FontLab and I’m going to investigate a donate option to try and recoup some of those costs.

The bold variant is the only one now requiring hinting and I’m hoping to have it done in the next 24-48 hours. The regular variant looks just great… as does italics.

[)amien

These look great – even more so when you consider there are no embedded bitmaps and very few delta hints.

There is still a lot of work to do – all the foreign characters, symbols and box-drawing characters (another 600 glyphs) require hinting and I should test it on the Mac, Java and Flash font rendering engines to make sure there are no show-stoppers there.

Preview 7 will consist of of just a plain style regular and bold because I need to get this out – it’s been too long since the last release. Preview 8 will add back italics and the Visual Studio italics-as-bold hack shortly afterwards.

Check out Talios’s shots using Java/Linux and Eddy Young’s shots in NetBeans.

A newer version of Envy Code R is available.

[)amien

Many articles later, some playing around with Microsoft Visual TrueType and much frustration and experimentation later I think I’m on the right path.

Here is how Envy Code R is looking on Windows right now with standard font smoothing.

ClearType doesn’t look as good and I’m still learning the black art and the implications of each type of hinting instruction.

Strangely, these hints seem to be ignored on the Mac which is still rendering everything a little too thick especially on curves. Perhaps that is why so many developers create a Mac-specific version?

Once I’m happy with how the regular version works I’ll put it online for download and then whip the bold and italic variants in to shape and any feedback into regular for the proper 0.7 release.

A newer version of Envy Code R is available.

[)amien

• ReadOnly, Greg Jandl: Comma clarified and change applied to full quotation marks, semi-colons and various accented letters
• Adrian Bool, Greg Jandl: The slash on the zero has been redrawn to be less heavy
• jxp: The Euro symbol has been redrawn from scratch
• Aristotle Pagaltzis: Braces are more curvy and a full set of box-drawing characters have been added
• IRC: Hash sign with longer legs

I have also fleshed out a number of additional symbols and accented letters that has seen the number of code pages supported increase to 12 pages and made a large number of tweaks to the italic version which was a last-minute addition to 0.6 (PR6) and had a number of errors especially round the accented letters.

Of course what you really want to know is how the new version looks in Visual Studio with that lovely Humane theme of mine:

There is still some work to do on the sizes above and below 10 point (again) as well as fleshing out a few more symbols, letters and italicising additional letters such as a curly k and rounder e which I hope will be finished towards the end of this week.

The observant followers may have noticed a pixel has been shaved off the vertical height which now brings it in line with the bitmapped Envy Code B coding font. I had intended on making the change for some time and the box characters practically demanded it to ensure the centres were whole pixels and not off-centre but some people may not like it…

A newer version of Envy Code R is available.

[)amien

One thing not too many people have yet been raving over is the lovely set of typefaces from Ascender Fonts known as the Droid family.

Hidden away into the downloadable SDK’s emulator is system.img which contains various binary files including these new fonts. Being that the image is almost certainly in Linux ext format and I found no easy way of mounting it in Mac OS X or Windows I was rather pleased when I stumbled upon Beeno’s page of the extracted files.

I have already covered Droid Sans Mono with an eye for using it for programming but thought it would be worth showing the other members of the family although I haven’t drawn direct comparisons with the Mac and Windows supplied fonts as I did with Red Hat’s Liberation fonts.

The Droid fonts in Windows XP via WordPad:

The Droid fonts in Mac OS X 10.5 via TextEdit:

Updated October 2008 from the official sources and including Apache licence

Download latest Droid Font Family from GitHub (TAR.GZ of TTF) (2.7 MB)

Download archived Droid Font Family (ZIP of TTF) (1.9 MB)

[)amien

Check out previous coverage of the well-known and lesser-known coding fonts.

There are a number of Droid fonts including Droid Sans and Droid Serif but of particular interest for developers is the Droid Sans Mono font that looks great in Visual Studio not only at my favourite 10 point… but from 7 point upwards with either ClearType or standard font smoothing although some might find the fact it smooths at all sizes a little soft (or Mac-like).

Here it is at 9 point with Rob Conery’s Vibrant Ink 2 theme:

Here it is at 11 point with my Humane theme:

And hereis 12 point in Xcode on the Mac:

The only issues are:

• the lack of a bold weight or italic variant which limits the syntax highlighting options
• the 0 is currently not slashed (there could be some other indistinguishable character pairs)

Being that the Droid family is Apache licensed no doubt somebody will fill that gap (okay, okay, I’ll give it a shot when I get some time;-)

Download Droid Sans Mono (TrueType TTF) (54 KB, 10/2008 source)

Try my free scalable coding font Envy Code R (shown below) with Visual Studio italic support, has a bold variant and distinguishable pairs 0O etc:

[)amien

Every month FontShop make a font available for free for a limited time so grab it while you can. The best option is to join their monthly newsletter so you don’t miss one!

Ralph Levien

Ralph is behind the excellent Inconsolata font which is great for printing code or if you like large programming fonts. He also has a number of classical typefaces including the great ATF Century Catalogue, Museum Caps, LeBe, ATF Bodoni and ATF Franklin Gothic.

Microsoft OpenType Font Pack

A bunch of licenced fonts available from Ascender Corporation including the stylish Kooteny and sharp Miramonte.

Jos Buivenga

Jos has produced a number of beautiful typefaces free to download. I particularly like the subtle quirks of Delicious and the flowing curves of Fertigo (sample shown_ but also check out some of the others available too.

MgOpen

These fonts were previously commercial and designed for the Greek alphabet and the four fonts available – Canonica, Cosmetica, Modata, Moderna – look great in the Latin alphabet too.

Die Gestalten

A number of experimental fonts for download but not redistribution including the rather delicious Engel Light ltd.

COM4t

These people produce a number of great free fonts including the elegant COM4t Fine Regular and Quilline Script Thin.

If you love typography then check out the ILoveTypography blog for news on great typefaces and tips on designing your own font.

[)amien
^{* some restrictions apply to usage on some of the fonts, check the licences for details.}

• Envy Code A – sizes from 7pt-12pt in both regular and bold weights
  
• Envy Code B – sizes from 9pt-10pt in regular, bold, italic and bold italic. (was the basis for Envy Code R)
  
• PalmOS – a Window port of the PalmOS system font recreated from screenshots.
  

I also have about 20 pixel fonts from my Spectrum days that I am intending on bringing across to Windows FON format in the near future.

[)amien

For reasons best known to Jeff he went with 11 point this time (previously his scheme was published with 10 point) and used the older preview of Envy Code R neglecting to mention the italic-as-bold variant to get round the no-italics limitation of Visual Studio’s highlighting syntax editor.

So here is the latest version, at the optimal 10 point utilising the italic variant and Jeff’s own colour scheme modified to show comments in ‘bold’:

Personally I am using a tweaked version of Thomas Restrepo’s dark theme at work that currently looks like:

[)amien

The last few days have been frantic ones putting the final touches to the next release of the Envy Code R typeface as I bring it closer to my idea of the perfect coding font.

Many glyphs have been redrawn, curves improved and a many additional characters and symbols added bringing the number of glyphs to 461 – enough to cover common European and US code pages including Windows/ISO 1250 & 1252 as well as MacOS Roman and a few others.

There was some interest in a bold variant and so once that was done I felt the urge to make an italic one too:

Phew!

As a bonus I’ve created a variant that overcomes Visual Studio’s aversion to italics by marking the italic font as bold. Choose ‘Envy Code R VS’ in the Font and Colors part of Visual Studio’s Options and choose bold wherever you want italics.

Here’s a sample at 10 point/ClearType with my own color scheme:

Yummy.

A few things to note:

• The new glyphs for accented characters, ligatures etc. might not be correct, they’re new to me
• Bitmaps are missing so if you don’t use smoothing/anti-aliasing/ClearType stick to the prior PR4 release
• Optimal size: Windows 10 point, Java 13 point and Mac OS X 12.5 point but looks good at larger sizes too
• Some glyphs will be improved (96?&) but others can’t (WwMm@) as there are no more pixels to play with
• At some sizes individual letters aren’t the right height, e.g. u,v,x at 12 point (damn hinting)

Download of this older version no longer available.

Feel free to leave comments and suggestions here (or better yet blog about it!)

[)amien

I have been experimenting with Envy Code R over the last few months – everything from creating a bold version to delta hinting with Visual TrueType with mixed results.

Part of the problem is that at larger sizes the odd shapes and over-emphasized curves that I drew to make the font look great at 10-point with ClearType on makes it look awkward at larger sizes.

Last weekend, I went back to the drawing board, literally, to try and improve on the shapes without impacting too much on the 10-point size ClearType version. The result is as follows;

And for comparison, the old version:

Let me know your thoughts in the comments, even if it’s just to say your prefer PR6, Consolas or something else. Be sure to mention what size and whether you have ClearType on (or if you are using a Mac).

[)amien

The primary difference is that Microsoft try to align everything to whole pixels vertically and sub-pixels horizontally.
Apple just scale the font naturally – sometimes it fits into whole pixels other times it doesn’t.
This means Windows looks sharper at the expense of not actually being a very accurate representation of the text. The Mac with it’s design/DTP background is a much more accurate representation and scales more naturally than Windows which consequently jumps around a lot vertically.

Jeff and Joel both wrote follow up posts agreeing that it is one of philosophy but both are of the opinion that the Windows pixel-grid approach is the better whilst our displays are only capable of low dots-per-inch (DPI).

What they don’t seem to appreciate is the compromise this causes.

Here is an example of Times New Roman on Windows (left) and Mac OS (right) scaled over whole point sizes with sub-pixel precision:

The two thing to note here arising from this “pixel-grid is king” approach are

1. Windows does not scale fonts linearly as the rough line points out
2. Windows scales the height and width but not the weight of the font

Neither of these may matter to a casual user but for professionals preparing material destined for high DPI (film or print) then it’s a world of difference. How can you layout a page on-screen and expect the same result on the page when the font isn’t the same width?

The issue is reminiscent of the “I hate black bars on wide-screen films” brigade who believe that the film should be chopped, panned, scaled and otherwise distorted from the artists original intention simply so that it fits better on their display.

Typography has a rich and interesting history developed and honed over centuries. It is a shame to misrepresent typefaces especially as the pixel-grid approach becomes less relevant as displays reach higher resolutions.

Update

Some additional comparisons and a note that the gamma differences between Windows and Mac will affect how you see the “other” systems rendering on your machine.

Further update (21 August 2007)

Thanks to Daring Fireball and ZDNet we’ve had a few more great comments which I’ve summarised here:

George thinks the philosophy idea is wrong because “What percentage of Mac users sit around all day doing nothing but pre-press work?” but as Fred points out Microsoft’s desktop-user optimised rendering ends up on images and videos all over the web, thus escaping the environment for which it was crippled.

George also claims that Vista’s rendering is improved, I can’t vouch for that one way or another but from looking at his screen shots the difference there could simply be the contrast level as adjusted by the ClearType tuner.

Nathaniel believes that it’s not Microsoft’s job to manipulate a typeface and that if you want on-screen readability then choose a font designed for that such as Microsoft’s own Tahoma or Apple’s Lucida Grande.

I’d go further and say that Microsoft’s own aggression in sticking to the grid kills font choice at the regular reading size of 10/11 point by optimising everything to a generic sans or serif look:

Windows XP

Mac OS X

James points to an article called Texts Rasterization Exposures that proposes a combination of using vertical hinting only and calculating horizontally to 256 levels and has some convincing screenshots showing the benefits. Probably too late for Leopard or Vista SP1 though.

[)amien