Background

If you've built a PC desktop in the last few years, you've probably been exposed to the confusing array of DDR4 information when it comes to buying RAM.

What it comes down to is not all RAM is created equal. Once you get past pin size and memory capacity, you'll have to filter down by speed. Speed isn't a simple one-figure number - you may see a rated speed like 2400MHz, but you may also see another bunch of numbers like 16-16-16-39 indicating the necessary clock cycles to perform certain types of memory operations.

While JEDEC - the people behind the DDR4 standard - ratify the well-known speeds like 2400 they've been quite behind the desires of consumers and manufacturers, so Intel created Extreme Memory Profile (XMP). XMP lets the RAM tell the PC a faster set of timings and clock speed it can support and what voltage it needs to do that. Things get a little more complicated as these memory modules are qualified only for specific speeds on tested motherboards and CPU combinations. XMP knows nothing about that - head to the RAM's Qualified Vendor List (QVL) to check it instead.

While all of this used to be only of interest to people wishing to "overclock" their systems, the latest Ryzen Zen 2 CPU's like the 3900X really benefit from increasing the memory speeds as the "Infinity Fabric" interconnect between the chiplets and the outside world runs at the speed of the memory controller (at least initially). Any Zen 2 owner wanting to take full advantage of the CPU they have purchased will need to dig in.

Thankfully memory modules for some time have a little bit of persistent memory called Serial Presence Detect (SPD) on them. It is accessed over the "I2C/SMBus" device bus in your machine so your BIOS or other tools can find out what the manufacturer has rated them for as well as who manufactured the actual memory chips themselves, when, where, and using what kind of die/process.

Reading the Serial Presence Detect

So given all that, what do you do to take a look at this info?

Well, on Windows, you can just fire up a tool like Thaiphoon Burner, click EEPROM in the menu and then Read SPD on SMBus... which gives you something like this:

You can then feed these values into something like DRAM Calculator for Ryzen to figure out good fast timings to enter into your BIOS.

What about Linux?

If you Google around, you'll find guides that tell you to run various modprobe, i2cdetect commands, and probably decode-dimms however...

DDR4 uses a different type of SPD memory chips. Linux support is new and incomplete.

Previously these chips were a type known as eeprom - electronically-erasable programmable read-only-memory (the previous tech was just known as eprom as you needed an ultraviolet box to wipe them through a little window on the top).

DDR4, however, has switched from the very dated eeprom technology to more popular flash memory.

So if you run those commands, you'll likely find decode-dimms doesn't tell you much at all. There are two possible scenarios:

1. It tells you no eeprom found meaning the existing eeprom driver can't find your SPD, and you're most likely going to need to obtain and load an alternative i2c driver.
2. It shows you some generic RAM information but says SPD is invalid, meaning i2c is communicating with the SPD, but Linux doesn't understand the DDR4 flash eeprom. You need to set up the ee1004 driver.

Identifying the right I2C bus

Either way, you're going to need to get the i2c device number your SPD is connected to by way of i2cdetect, so run:

i2cdetect -l

Which will give you some output like this:

i2c-3   i2c         i915 gmbus dpd                          I2C adapter
i2c-1   i2c         i915 gmbus dpc                          I2C adapter
i2c-4   i2c         DPDDC-E                                 I2C adapter
i2c-2   i2c         i915 gmbus dpb                          I2C adapter
i2c-0   smbus       SMBus I801 adapter at f040              SMBus adapter

The results are system-dependent, but you need to deduce which one your SPD is on. The Intel Z370 chipset my Linux box runs uses the SMBus I801 interface, so I need to make a note of i2c-0.

If you're not sure, then it may be a process of elimination based on the names. i915 is the Intel onboard CPU graphic support, so it wouldn't be that, and a quick Google also associated the DPDDC-E with that too. I'd probably also give some preference to SMBus adapter ones rather than I2C, but I couldn't find any confirmation that this is always the case.

If you're not sure which one, you can try dumping information from each to see if you can find the right-looking data using this command but changing 0 to the likely candidates to the i2c device number from the first column in i2cdetect -l.

i2cdetect -y 0

With any luck, it should look something like this. If all you see are -- then try another.

     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- 08 -- -- -- -- -- -- --
10: -- -- -- -- 14 15 -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: 30 31 -- -- 34 35 -- -- -- -- -- -- -- -- -- --
40: 40 -- -- -- 44 -- -- -- -- -- -- -- -- -- -- --
50: 50 51 52 53 -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- 6a -- -- -- -- --
70: -- -- -- -- -- -- -- --

Now there's a chance - especially if decode-dimms told you no eeprom found that you don't actually have the i2c driver loaded. The i801 driver is compatible with many Intel chipsets. To load that, you can use:

modprobe i2c-i801

If you're running an AMD CPU, you might get away with:

modprobe i2c-amd-mp2-pci

if your Linux is really up to date. If not check out the i2c-amd-mp2 repo for manual installation instructions.

You can also try running decode-dimms again, and hopefully, now you should see some generic memory information.

Introducing ee1004

Now that you have i2c working and identified, you can move on to the ee1004 flash driver for DDR4.

If so then just typing:

modprobe ee1004

Should get you up and running. If so, you can skip the next section and go straight to registering the SPD devices.

If you don't have the kernel module (Ubuntu 18 doesn't have it), you will see an error like this one:

modprobe: FATAL: Module ee1004 not found in directory /lib/modules/...

and so you'll need to install it...

Installing latest ee1004 driver

So either you don't have ee1004 or you need the latest. Let's CURL it down from the maintainers site. I wouldn't recommend building kernel drivers from untrusted sources but...

curl -O http://jdelvare.nerim.net/devel/lm-sensors/drivers/ee1004/Makefile
curl -O http://jdelvare.nerim.net/devel/lm-sensors/drivers/ee1004/ee1004.c
sudo make install

(If make doesn't work, you're going to need to install things like build-essentials etc., via apt or your package manager of choice).

Now that is compiled and copied into /lib/modules/.../kernel/drivers/misc/eeprom/ee1004.ko we need to load it. Now there's a small chance your system already has the older eeprom driver loaded, so we'll unload that as well, so it doesn't interfere:

modprobe -r eeprom
modprobe ee1004

Registering the SPD devices

OK, so you now have the ee1004 driver installed, and you can see the bus your SPD devices are on. There are just a few more steps to make sure ee1004 knows which SPD devices it's supposed to be using.

Run the following command replacing 0 with the number after i2c- you identified as the bus your SPD devices are located on:

i2cdetect -y 0

Now this will dump out that basic I2C information we saw earlier. On my system, it looks like this:

     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- 08 -- -- -- -- -- -- --
10: -- -- -- -- 14 15 -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: 30 31 -- -- 34 35 -- -- -- -- -- -- -- -- -- --
40: 40 -- -- -- 44 -- -- -- -- -- -- -- -- -- -- --
50: 50 51 52 53 -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- 6a -- -- -- -- --
70: -- -- -- -- -- -- -- --

Notice those '50 51 52 53` bytes? Those are my four DDR4 DIMM sticks, and we need to register those numbers with the ee1004 driver.

Note: I couldn't find out if the numbers that are important or the location.

You can start by just registering one to see if it works (replace i2c-0 with your i2c device from the previous section and 0x50 with the bank number of your memory module you figured out in the chart above:

echo ee1004 0x50 > /sys/bus/i2c/devices/i2c-0/new_device

now run decode-dimms and, with any luck, you will see output like this:

Memory Serial Presence Detect Decoder
By Philip Edelbrock, Christian Zuckschwerdt, Burkart Lingner,
Jean Delvare, Trent Piepho and others


Decoding EEPROM: /sys/bus/i2c/drivers/ee1004/0-0050
Guessing DIMM is in                              bank 1

---=== SPD EEPROM Information ===---
EEPROM CRC of bytes 0-125                        OK (0x05EA)
# of bytes written to SDRAM EEPROM               384
Total number of bytes in EEPROM                  512
Fundamental Memory type                          DDR4 SDRAM
SPD Revision                                     1.1
Module Type                                      UDIMM
EEPROM CRC of bytes 128-253                      OK (0x27DE)

If it did not register correctly, you should de-register the bank with the command:

echo 0x51 /sys/bus/i2c/devices/i2c-0/delete_device

If it worked, go ahead and register the rest of your modules depending on what banks you have memory sticks in. For example, on my system, the complete registration is:

echo ee1004 0x50 > /sys/bus/i2c/devices/i2c-0/new_device
echo ee1004 0x51 > /sys/bus/i2c/devices/i2c-0/new_device
echo ee1004 0x52 > /sys/bus/i2c/devices/i2c-0/new_device
echo ee1004 0x53 > /sys/bus/i2c/devices/i2c-0/new_device

Now you can finally run

decode-dimms --side-by-side

And get to see timings, manufacturer, etc.

Decoding EEPROM                                  0-0050                0-0051                0-0052                0-0053
Guessing DIMM is in                              bank 1                bank 2                bank 3                bank 4

---=== SPD EEPROM Information ===---
EEPROM CRC of bytes 0-125                        OK (0x05EA)           OK (0x699E)           OK (0x05EA)           OK (0x699E)
# of bytes written to SDRAM EEPROM               384
Total number of bytes in EEPROM                  512
Fundamental Memory type                          DDR4 SDRAM
SPD Revision                                     1.1                   1.0                   1.1                   1.0
Module Type                                      UDIMM
EEPROM CRC of bytes 128-253                      OK (0x27DE)

---=== Memory Characteristics ===---
Maximum module speed                             2400 MHz (PC4-19200)
Size                                             16384 MB
Banks x Rows x Columns x Bits                    16 x 16 x 10 x 64
SDRAM Device Width                               8 bits
Ranks                                            2
Rank Mix                                         Symmetrical
AA-RCD-RP-RAS (cycles)                           16-16-16-39
Supported CAS Latencies                          20T, 18T, 16T, 15T, 14T, 13T, 12T, 11T, 10T, 9T

---=== Timings at Standard Speeds ===---
AA-RCD-RP-RAS (cycles) as DDR4-2400              16-16-16-39
AA-RCD-RP-RAS (cycles) as DDR4-2133              15-15-15-35
AA-RCD-RP-RAS (cycles) as DDR4-1866              13-13-13-30
AA-RCD-RP-RAS (cycles) as DDR4-1600              11-11-11-26

---=== Timing Parameters ===---
Minimum Cycle Time (tCKmin)                      0.833 ns
Maximum Cycle Time (tCKmax)                      1.600 ns              1.500 ns              1.600 ns              1.500 ns
Minimum CAS Latency Time (tAA)                   13.320 ns
Minimum RAS to CAS Delay (tRCD)                  13.320 ns
Minimum Row Precharge Delay (tRP)                13.320 ns
Minimum Active to Precharge Delay (tRAS)         32.000 ns
Minimum Active to Auto-Refresh Delay (tRC)       45.320 ns
Minimum Recovery Delay (tRFC1)                   350.000 ns
Minimum Recovery Delay (tRFC2)                   260.000 ns
Minimum Recovery Delay (tRFC4)                   160.000 ns
Minimum Four Activate Window Delay (tFAW)        21.000 ns
Minimum Row Active to Row Active Delay (tRRD_S)  3.299 ns
Minimum Row Active to Row Active Delay (tRRD_L)  4.900 ns
Minimum CAS to CAS Delay (tCCD_L)                5.000 ns
Minimum Write Recovery Time (tWR)                15.000 ns             N/A                   15.000 ns             N/A
Minimum Write to Read Time (tWTR_S)              2.500 ns              N/A                   2.500 ns              N/A
Minimum Write to Read Time (tWTR_L)              7.500 ns              N/A                   7.500 ns              N/A

---=== Other Information ===---
Package Type                                     Monolithic
Maximum Activate Count                           Unlimited
Post Package Repair                              Not supported
Module Nominal Voltage                           1.2 V
Thermal Sensor                                   No

---=== Physical Characteristics ===---
Module Height                                    32 mm
Module Thickness                                 2 mm front, 2 mm back
Module Reference Card                            B revision 0

---=== Manufacturer Data ===---
Module Manufacturer                              Crucial Technology
DRAM Manufacturer                                Micron Technology
Assembly Serial Number                           0x3982077B            0xA02071FC            0x3982077D            0xA02071AC
Part Number                                      BLS16G4D240FSE.16FBD  BLS16G4D240FSE.16FAD  BLS16G4D240FSE.16FBD  BLS16G4D240FSE.16FAD

Identifying die revisions

One of the things you don't see here is the die version. If you're overclocking, it is important to determine what timings and tweaks you can get at (right now, Micron E-die is the newest hotness after Samsung B-die had a good run).

Die is not part of the specification, and it seems from playing with Thaiphoon, it's a bunch of matching bytes. I'm looking into putting together a small Linux script/binary that will identify them, but in the mean-time, you have two options if you need this extra info:

Either way, you'll need to make a Thaiphoon-compatible hex dump (replacing 0-0050 with the i2c bus number followed by a dash then the memory bank number)

od -Ax -t x1 -v /sys/bus/i2c/drivers/ee1004/0-0050/eeprom

You'll see some output like this which you should copy to the clipboard:

000000 23 11 0c 02 85 21 00 08 00 00 00 03 09 03 00 00
000010 00 00 07 0d fc 2b 00 00 6b 6b 6b 11 00 6b f0 0a
000020 20 08 00 05 00 a8 1b 28 28 00 78 00 14 3c 00 00
000030 00 00 00 00 00 00 00 00 00 00 00 00 16 36 16 36
000040 16 36 16 36 00 20 2b 0c 2b 0c 2b 0c 2b 0c 00 00
000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
000070 00 00 00 00 00 00 9c b4 c9 c9 c9 c9 e7 d6 ea 05
000080 11 11 01 01 00 00 00 00 00 00 00 00 00 00 00 00
000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 de 27
000100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
000130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
000140 85 9b 00 00 00 39 82 07 7b 42 4c 53 31 36 47 34
000150 44 32 34 30 46 53 45 2e 31 36 46 42 44 00 80 2c
000160 00 44 50 41 47 48 34 47 30 30 31 ff 00 00 00 00
000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
000180 0c 4a 05 20 00 00 00 00 00 94 00 00 07 ff 3f 00
000190 00 6a 6a 6a 11 00 6b f0 0a 20 08 00 05 00 a8 1b
0001a0 28 00 00 00 00 00 00 00 00 cf b5 ca ca ca ca d6
0001b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0001c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0001e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0001f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
000200

Now either:

1. Send it to a Windows-using friend who can run Thaiphoon
2. Run Thaiphoon via WINE (which should now work as it won't need low-level hardware access to the SMBus)

Inside Thaiphoon in the File menu, you'll see Import from Clipboard which should reveal a lot of information, including the DRAM Die Revision / Process Node you're after.

If you want to read another individuals hex dumps on your Linux box decode-dimms can do that too:

decode-dimms -x myspd1.hex

Enjoy!

[)amien

What is semantic versioning (semver) ?

Semantic versioning is a simple agreement on how packages should be versioned. It gives package developers a framework to version their software. It provides consumers of packages an expectation of how they will behave over time that they can consider.

The versioning format is major.minor.patch, described as:

• major - Large improvements and breaking-changes.
• minor - New features that are backward compatible.
• patch - Bug fixes that are backward compatible.

The thing to note here is that a breaking change should only be found in a major release.

What's wrong with that?

Well, the reality is any change could theoretically break somebody already using your package. XKCD has a famous strip about it but even not taken to that extreme; there are many ways you can unintentionally break existing users.

How can you unintentionally break existing users?

Here are a few examples of how you can break existing users with a seemingly safe bug fix or innocuous new feature:

Create a new class

A seemingly innocent thing to do - just create a new class. The problem is many languages allow you to import entire namespaces - C#, for example. Class names must be unique or fully qualified, so if your class name collides with any others in-scope and boom, compiler error.

Add a method to an existing class

Okay, this is our class. We can add a method to it. Wait, did we let the customer inherit from this class? If that well-chosen method name collides with one, they implemented in the sub-class, boom—a compiler error.

Change a default value

Convention over configuration is a well-established mantra with liberal use of defaults. What if you need to change the default? The industry guidance before was X, and now it's Y, so you want to change the default to match. The problem is you have no idea of knowing if a customer accepted X because they always want your best-intentioned default or if they specifically wanted that value and knew it was the default when they first observed it.

Speed things up

Performance improvements are always desirable, right?! Well, any speed up or slow down can cause previously rare deadlocks to increase or decrease. They can also affect back-pressure on other systems and exceed rate limits on API calls.

Upgrade a dependency

You upgraded a dependency, and now the package manager upstream is having difficulty reconciling it with the customer's project as they also have this dependency.

Is semver a dream?

Semantic versioning isn't a dream. It's an ideal, an intent, something to strive for.

We can never be sure an update won't break somebody. There are no guarantees.

When we intentionally break things, we can ensure we align it with a major release and ideally bundle it with other breaks and a set of enhancements or features to make adopting it worth the effort.

I started designing fonts around 1987 on an 8-bit Sinclair ZX Spectrum. Many years later, my involvement in the Spectrum emulation scene led aul Dunn to ask me if I could provide fonts for his excellent BASIN Sinclair BASIC for Windows. My interest in 8x8 fonts was suitably rekindled, and I ended up delivering about 60 - some even extracted from my original +3 disk images.

I wanted to get these fonts online earlier! Raw 7698-byte files, however, are only of use to BASIN users or those suitably familiar with the convoluted process and tools to get them into a Spectrum or emulator. Even trying to use them on Windows was a pain given that TrueType only cares about scalable fonts (SBIT embedded bitmaps are rare these days).

Fellow typographer and Spectrum fan Paul van der Laan gave me some tips for pixel-perfect TrueType fonts at specific multiples (8px on Windows). Armed with this knowledge, I put together an automated pipeline using a modified version of John Elliot's psftools which converts the original ZX font to PSF and then to BDF. Converting to TrueType TTF involved FontLab 5 Studio and some Python scripting while other conversions required some off-the-shelf tools and a bunch of custom ones all tied together with bash scripts.

It's impossible to get a real feel for a font from a simple run of letters. To alleviate this, I created code that re-renders hand-picked game screenshots to show them in real-world scenarios. If your browser supports HTML5 canvas, you'll see these. You can also drag any raw 768-byte Spectrum font file such as those created by BASIN to any of my ZX-Origins pages to see your font rendered with these samples.

The uploads are now mostly complete - there are 78 typefaces and well over 120 actual fonts once you include the variations, styles, and weights that some typefaces have. This number may increase from time to time if, and when, I finish off fonts in my _incomplete folder, resurrect some abandoned ones (sorry Unwanted Attention), or get inspired for something entirely new.

So without further ado, head over to ZX Origins and grab a font or two for your latest 8-bit retro game!

My site goes back to 2004 and is reasonably sized but not massive even with the comments, so waiting 30 seconds for a change to reflect is disappointing.

I investigated several options and ended up using a few in combination, skipping others and holding out for one.

Incremental build

Launching with the --incremental switch, for example, bundle exec jekyll serve --incremental, can speed things up a bit, but there are issues here. One is that it still builds many pages, tags, and categories as it updates, and the second is that if you're using collections, it misses rebuilding pages that reference that collection.

Verdict: Not great.

WSL2 or a Mac?

My primary dev box these days is a beefy Windows box. Jekyll doesn't seem to want to run under native Windows, and Windows Subsystem for Linux (WSL) can be pretty slow compared to a Mac. Microsoft announced Windows Subsystem for Linux 2 (WSL2) which will be much faster but, it isn't available until June 2019 to insiders.

Verdict: Hold tight.

Caching

Hubber extraordinaire Ben Balter developed the Jekyll Include Cache which can help speed up building.

The approach it uses is that you can switch out a regular include x for include_cached x, providing that x doesn't depend on any state/variables. That meant my footer, sidebar, and navbar could all easily be parsed and stored just once by changing things like: {% include footer.html %} to {% include_cached footer.html %}

There are, however, places where state is required, and this plugin can still help there. For example, my header is called using {% include head.html %} and contained code like this:

<meta
  name="description"
  content="{{ page.excerpt | markdownify | strip_html | truncatewords:40 }}"
/>
<title>{{ page.title }} &raquo; DamienG</title>
<meta name="twitter:url" content="{{ page.url | absolute_url }}" />

What you can do here is pass parameters to include_cached and read them the other side. Replacing the include with:

{% include_cached head.html title=page.title url='page.url | absolute_url' excerpt='page.excerpt | markdownify | strip_html | truncatewords:40' %}

And changing the include itself to:

<meta name="description" content="{{ include.excerpt }}" />
<title>{{ include.title }} &raquo; DamienG</title>
<meta name="twitter:url" content="{{ include.url }}" />

Which made the header easy to read as the title, description, and URL are used multiple times for the various types of metadata (Twitter, OpenGraph, etc.)

Verdict: Some good improvement.

No paging, tags, and collections for local dev

I never actually browse the pages, tags, or collections locally but, they're essential for the CI and publishing. I've manually turned them off in the past but, it's error-prone and a pain. But there is a way to avoid that.

Move the parts of your configuration you don't want to be part of your local build into a new configuration file. I moved the following from my _config.yml into a new _config-publish.yml:

autopages:
  enabled: true
  categories:
    enabled: true
  tags:
    enabled: true

pagination:
  enabled: true

I also moved my Aloglia configuration block here too. Now a local build with unpublished content avoids turning up in the search index!

It's important to remember to add --config _config.yml,_config-publish.yml to any Jekyll build steps on your CI server. For example, mine has:

- run:
    name: Build site
    command: bundle exec jekyll build --config _config.yml,_config-publish.yml 2>&1 | tee $JOB_RESULTS_PATH/build-results.txt

and:

- run:
    name: Index with Algolia
    command: bundle exec jekyll algolia --config _config.yml,_config-publish.yml

Verdict: Awesome, drops from 30s to 17s for full build.

Turn off comments for local

I thought this would have more of an impact but, it only shaves a few seconds off the site build times.

To do this add the following first and last lines to your comments.html include (I'll probably add this to the Jekyll Blog Comments system)

{% if site.comments.enabled %} ... (existing comments.html contents here) {%
endif %}

and remember to add this to your _config-publish.yml:

comments:
  enabled: true

Verdict: Okay, drops from 17s to 15s for full build.

Conclusion

With these steps, my site has gone from almost 50 seconds to about 17 for a full build without incremental. With incremental, that now means about 1-2 seconds!

I can't wait for WSL2 to push this over the edge!

[)amien