Posts in category blogging - page 2

WordPress to Jekyll part 1 - My history and reasoning

Part of my series on migrating from WordPress to Jekyll.

  1. My history & reasoning
  2. Comments & commenting
  3. Site search
  4. Categories & tags
  5. Hosting & building

It’s hard to believe it was 13 years ago back in a cold December on the little island of Guernsey when I decided to start blogging. I’d had a static site with a few odd musings on it since 2000 but this was to be conversational, regularly updated and with more technical content. Blogspot seemed the easiest way to get started.

Briefly hosted at home

Within 18 months of regular blogging I’d moved over to Subtext which being a .NET app required Windows hosting so threw it on a small Shuttle PC on my home DSL. This is where I started using it as an experiment for CSS and web techniques but within a year I’d had my 1MB DSL brought to it’s knees twice through articles being featured on Boing Boing.

I did however contribute a little to the project and started chatting with the maintainer - Phil Haack - who I’d end up meeting when we both joined Microsoft years later and is a friend to this day.

Landing on WordPress

DamienG theme in 2008 In 2007 I migrated to a PHP based CMS that was making a name for itself called WordPress. My blog would remain on WordPress for 10 years across shared hosting, VMs and dedicated servers.

One server was caught in an explosion at the ISP, another time my site got pwned through a WordPress vulnerability. I switched themes several times before creating my own super-light MootStrap theme based around the BootStrap 2 layout and nav bar. I messed with wp-SuperCache trying to improve performance and scalability before switching out the PHP engine for HHVM as well as using NGINX instead of Apache and MariaDB instead of MySQL all in an attempt to eek out a bit of extra performance.

While my theme lives on today - for now at least - MootStrap and PHP are no more as I switched over to the Jekyll static site generator earlier this month after a long meandering journey to get there.

Why Jekyll?

I’ve had a lot of success with Jekyll on some other sites I run. Hosting it on GitHub pages or S3 with a CloudFront brings a lot of benefit:

  1. Cost - S3 and CloudFront cost pennies rather than $40+ a month
  2. Security - there’s no code running to be exploited, no WordPress plug-in back-doors
  3. Speed - CloudFront is a geo-distributed CDN and S3 is no slouch either
  4. Editing - text files are easier to process, find, manipulate and markdown much easier to write

The price aspect is definitely worth mentioning again. With the occasional bursts in traffic my site hosting generally worked out around $40 a month for a decent VM. On AWS I’m expecting it to max out at $3 despite these improvements and benefits.

Of course part of the other reason is static site generators are interesting and I like to play.

Some challenges

Jekyll is a static site generator. That is you run the tool somewhere and it produces plain html files with zero server-side code left in them. By its very nature is going to not have support for:

  • Comments - No way to accept or render them
  • Search - No site search facility
  • URL control - Difficult to match the paging/tags/categories with default plugins

Surprisingly however there are blog-friendly facilities where static generation can support it, specifically:

[)amien

Five steps to blog (re)design

For some time I’ve wanted to refresh the design here at damieng.com which evolved out of the Redoable theme with my own tweaks to colors, typography, images and background until it was almost my own.

Almost, but not quite.

As an engineer I prefer incremental improvement over a complete redesign but I was at the point where the code-base (HTML, CSS and PHP) was unworkable. Partly my fault, partly the sheer complexity of a general-purpose theme with wide-ranging plugin support I didn’t need.

These are the steps that helped me get to a  clean minimal code-base that did exactly what I wanted and would like to share the experience for friends about to go through a similar process.

1. Decide on a layout

Layout of DamienG.comGetting layouts right across browsers can be tricky.

Thankfully a number of individuals have put together a number of reusable CSS frameworks that provide grid-based approaches to layout, compensating for differences in browser defaults and including some minimalistic styling.

These let you hit the ground running but they are not complete themes unless you’re into minimalistic design.

I’m not a fan of fixed-width sites that contribute to the years of wear on my scrolling mouse finger so settled on the Fluid 960 Grid System which scales a 12 or 16 column grid system out to the full browser width while being incredibly easy to use and tweak.

At this point you’ll want to copy the CSS they provide into your own development folder and take the existing HTML sample page and cut it down to the bare minimum of <head> and its necessary <link…> elements to setup the CSS plus the <body> and any intermediate child elements required to get the basic structure in place.

Now think about how many columns each section needs to cover and whether it should continue taking columns to the right or whether it should start on a new row after the existing section has finished.

Given I wanted the layout shown above I needed just three containers – the primary left area (red) would span 12 columns and would contain the branding, navigation and actual content. The right area (blue) would span 4 columns and contain the sidebar. As I didn’t want my footer too high on pages with small amounts of content I created another section (not pictured) that also spanned 12 columns but would appear after the content in both sections. Simply:

<body>
  <div class="container_16">
    <div class="grid_12">
      <a href="//damieng.com/" />
      <ul class="nav" />
      <div id="maincontent" />
    </div>
    <div class="grid_4" id="sidebar" />
    <div class="clear" />
    <div class="grid_12" id="footer" />
  </div>
</body>

2. Add your content

Unless you’re planning on throwing away any of your content (not recommended if you have it), now is the time to drop in actual content, navigation and any chrome. At this stage real static HTML content is best – the combination of PHP, remotely uploading files and hooking in to dynamic content can really slow you down while you are trying to get the basics sorted.

The easiest thing to do here is to just view source a page on your blog and copy out the required blocks. For me this was:

  • Two blog posts from the main page, they sit in the maincontent div above, each contained within a new <div class=”article”> so I can provide article-specific styling
  • A brand new set of nested navigation elements created by hand based on the page structure – initially two levels deep to support the drop-down CSS menus
  • My five sidebar elements – Search, Stay in touch, Topics, My most popular and Interesting finds

Bear in mind that you are going to need to keep the structure and content of your existing pages and articles unless you have the time to either go-back and edit each one or apply some bulk-changes via SQL UPDATE statements. For me, this meant keeping my post sub-headings as <h3> and sticking with my class="alert", class="code" and  class="download" blocks I use for notifications, code snippets and downloads which required bringing some CSS elements in from the old theme.

3. Style the page

Design inspiration can come from many places – here are some best-of designs, portfolios and showcases to draw from:

Now you have some basic HTML and CSS displaying real content it’s easy to see what works and what doesn’t with minimal resistance to experimenting.

I was happy with the existing look and I plan on further tweaks now my code-base is easy to understand so for now took the basic colors, background and icons then adjusted the layout, spacing, positioning, wording and elements like the content metadata. This was also an opportunity to experiment with some CSS techniques available but considering CSS3 browser support is limited right now I kept this to:

  • @font-face for headings – Font Squirrel have a great set of @font-face kits including fonts with the CSS snippets
  • text-shadow for headings and hyperlink hover-overs to achieve a glow effect
  • border-radius for curved blocks and navigation bar (also used prior to the redesign)
  • JavaScript to upgrade the search-box from type="text" to type="search" in Safari

The result is something that looks quite unique on Safari and Chrome yet still looks good on Internet Explorer 7.

4. Convert to a theme

Now our CSS and HTML looks great as a single page we need to create a theme for the blog engine. This consists of several specially-named files to dynamically construct the page as required.

You can build this from scratch but a basic theme like Starkers ‘naked’ WordPress theme is recommended. Merge the layout and style of your theme with the dynamic tags that drop in content, navigation and titles in from the basic theme. A quick start would be to modify:

  • header.php with your HTML up to opening your main content div but:
    • remove <link type=”text/css”> elements to the CSS and replace with the one line <link> from the theme
    • replace the static navigation elements with a call to a function to dynamically build it such as <?php wp\_list\_pages(depth=3&title_li=); ?>
    • links for shortcut icons and ensure they point to files that exist, you don’t want to be serving up 404 error pages every time
    • meta tags – take the dynamic ones with <?php ?> elements (content-type, description etc). from the theme and merge them with your sites existing ones if you have any
  • sidebar.php with your HTML from closing the main content div through to closing the sidebar div
  • footer.php with your HTML footer div (and in my case the preceding clear div) through to the end of the file
  • single.php with your HTML that appears around each article with the necessary tags (like <?php the_content(); ?> taken from the naked theme)
  • style.css with relative references to your CSS files and your new metadata, e.g.
/*
Theme Name: DamienGNew
Theme URI: //damieng.com
Description: Minimalistic theme based on Fluid 960 Grid.
Version: 1

Author URI: //damieng.com
*/
@import "css/reset.css";
@import "css/grid.css";
@import "css/damieng.css";
@import "css/navigation.css";

Rename the theme folder and upload it to your blog in the right location – for WordPress this is wp-content/themes.

5. Test drive, tweak, rinse and repeat

Now you want to test the theme without interrupting the existing one so don’t go and set it to be the default just yet.

If you are using WordPress a few tips:

  1. WP Super Cache enabled sites – check Don’t cache pages for logged in users so you don’t see cached pages in the original theme pages or serve up pre-cached new themed pages for other visitors
  2. Theme Test Drive – lets you specify a new theme and what access level it applies to (e.g. 10 for admin) so you will see it while you are logged in for testing
  3. Page templates –  leave them to last as WordPress will try to use them in the default theme too
  4. Keep an eye out for error_log files – they contain details of when pages failed

It’s worth testing your site in popular browser and operating system combinations from your own analytics or you don’t have any of your own stats check out usage share of web browsers at Wikipedia. The major engines are:

  • Internet Explorer (Windows)
  • WebKit (Safari on OS X, Chrome on Windows & Konqueror on Linux)
  • Gecko (Firefox cross-platform & Camino on Mac)
  • Opera (cross-platform)

Bear in mind that each browser can behave quite differently between versions – Internet Explorer 6 will likely render quite differently to 7 or 8.

Sites like BrowserShots can help by rendering your page on a multitude of browsers for you and present screen-shots (once the theme is live) to compare. Validating your HTML and CSS is a good idea because the errors and warnings it spots are the likely cause for odd rendering problems.

Now you see a single blog post rendered it’s time to go through the rest of the files and apply the same merging process before you sit back, smile and make your new theme the default.

[)amien

Importing BlogML into WordPress

I’ve been trying to get my content out of Subtext and into WordPress – a process that shouldn’t be difficult however Subtext only supports the blog-independent BlogML format and whilst WordPress supports a number of import formats BlogML isn’t one of them. For export WordPress only supports it’s own WordPress WXR format although the BlogML guys have an exporter available.

The first idea was to put together an XSL transform to convert BlogML to WXR.

BlogML format

BlogML posts look like this although Subtext fails to populate the views attribute or even a tag for the user-email as at 1.9.3. It also doesn’t include a field for a commenter’s IP addresses. These two limitations mean no Gravatars or Identicons at the other end right now.

<post id="1" date-created="2006-04-24T04:07:00" date-modified="2006-04-25T11:55:00" approved="true"
    post-url="https://damieng.com/blog/archive/2008/01/30/Test.aspx" type="normal" hasexcerpt="false" views="0">
  <title type="text"><![CDATA[This is a test]]></title>
  <content type="text"><![CDATA[Just testing content]]></content>
  <post-name type="text"><![CDATA[ThisIsATest]]></post-name>
  <categories>
    <category ref="1" />
  </categories>
  <authors>
    <author ref="1" />
  </authors>
</post>

WXR format

WXR posts are extended RSS items and annoyingly doesn’t have a field for view counts at all.

<item>
  <title>This is a test</title>
  <link>https://damieng.com/blog/archive/2008/01/30/Test.aspx</link>
  <pubDate>Thu, 04 Apr 2006 04:07:00 +0000</pubDate>
  <dc:creator>Damien Guard</dc:creator>
  <guid isPermaLink="false">https://damieng.com/blog/archive/2008/01/30/Test.aspx</guid>
  <description></description>
  <content:encoded><![CDATA[Just testing content]]></content:encoded>
  <wp:post_id>1</wp:post_id>
  <wp:post_date>2006-04-24 04:07:00</wp:post_date>
  <wp:post_date_gmt>2006-04-24 04:07:00</wp:post_date_gmt>
  <wp:comment_status>open</wp:comment_status>
  <wp:ping_status>open</wp:ping_status>
  <wp:post_name>about</wp:post_name>
  <wp:status>publish</wp:status>
  <wp:post_parent>0</wp:post_parent>
  <wp:menu_order>0</wp:menu_order>
  <wp:post_type>post</wp:post_type>
</item>

Convert BlogML to WXR using XSLT

There are a few things to bear in mind when using this transform:

  • Link and guid tags are populated but WordPress seems to ignore them. Will investigate soon!
  • Time-zone conversion does not take place – hand-code +offsets in the XSLT to deal with your zone.
  • Track-backs not yet considered.
  • Default namespace in BlogML is not handled – remove the xmlns=”…” declaration from your BlogML file before transforming.
  • HTML within comments is not supported – when I enabled this WordPress treated the HTML as text.
  • Embedded attachments are not supported.
  • Edit the primary site link at channel/link in the transformed file to match your site – BlogML doesn’t include it.

Multiple authors and categories should work just fine so throw this file and your BlogML export through an XSLT processor and presto, WXR content ready for import.

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                              xmlns:dc="http://purl.org/dc/elements/1.1/"
                              xmlns:wp="http://wordpress.org/export/1.0/"
                              xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <xsl:output method="xml" indent="yes" cdata-section-elements="content:encoded"/>

  <xsl:template match="/">
    <rss version="2.0"
        xmlns:content="http://purl.org/rss/1.0/modules/content/"
        xmlns:wfw="http://wellformedweb.org/CommentAPI/"
        xmlns:dc="http://purl.org/dc/elements/1.1/"
        xmlns:wp="http://wordpress.org/export/1.0/">
      <xsl:apply-templates />
    </rss>
  </xsl:template>

  <xsl:template match="blog">
    <channel>
      <title><xsl:value-of select="title"/></title>
      <link>http://addsitehere</link>
      <description><xsl:value-of select="sub-title"/></description>
      <pubDate>
        <xsl:call-template name="topubdate">
          <xsl:with-param name="date" select="@date-created" />
        </xsl:call-template>
      </pubDate>
      <generator>DamienG's BlogML to WordPress transform</generator>
      <language>en</language>
      <xsl:apply-templates />
    </channel>
  </xsl:template>

  <xsl:template match="blog/categories/category">
    <wp:category>
      <wp:category_nicename>
        <xsl:value-of select="title"/>
      </wp:category_nicename>
      <wp:category_parent></wp:category_parent>
      <wp:posts_private>0</wp:posts_private>
      <wp:links_private>0</wp:links_private>
      <wp:cat_name>
        <xsl:value-of select="@description"/>
      </wp:cat_name>
    </wp:category>
  </xsl:template>

  <xsl:template match="post">
    <item>
      <title>
        <xsl:value-of select="title"/>
      </title>
      <link>
        <xsl:value-of select="@post-url"/>
      </link>
      <pubDate>
        <xsl:call-template name="topubdate">
          <xsl:with-param name="date" select="@date-created" />
        </xsl:call-template>
      </pubDate>
      <dc:creator>
        <xsl:variable name="authorref" select="authors/author/@ref" />
        <xsl:value-of select="//author[@id=$authorref]/title"/>
      </dc:creator>
      <xsl:apply-templates select="categories" />
      <guid isPermaLink="false">
        <xsl:value-of select="@post-url"/>
      </guid>
      <description></description>
      <content:encoded>
        <xsl:value-of select="content" disable-output-escaping="yes"/>
      </content:encoded>
      <wp:post_id>
        <xsl:value-of select="@id"/>
      </wp:post_id>
      <wp:post_date>
        <xsl:value-of select="translate(@date-modified,'T',' ')"/>
      </wp:post_date>
      <wp:post_date_gmt>
        <xsl:value-of select="translate(@date-modified,'T',' ')"/>
      </wp:post_date_gmt>
      <wp:comment_status>open</wp:comment_status>
      <wp:ping_status>open</wp:ping_status>
      <wp:post_name>
        <xsl:value-of select="post-name"/>
      </wp:post_name>
      <wp:status>
        <xsl:choose>
          <xsl:when test="@approved='true'">publish</xsl:when>
          <xsl:otherwise>draft</xsl:otherwise>
        </xsl:choose>
      </wp:status>
      <wp:post_parent>0</wp:post_parent>
      <wp:menu_order>0</wp:menu_order>
      <wp:post_type>post</wp:post_type>
      <xsl:apply-templates />
    </item>
  </xsl:template>

  <xsl:template match="comment">
    <wp:comment>
      <wp:comment_id>
        <xsl:value-of select="@id"/>
      </wp:comment_id>
      <wp:comment_author>
        <xsl:value-of select="@user-name"/>
      </wp:comment_author>
      <wp:comment_author_email></wp:comment_author_email>
      <wp:comment_author_url>
        <xsl:value-of select="@user-url"/>
      </wp:comment_author_url>
      <wp:comment_author_IP></wp:comment_author_IP>
      <wp:comment_date>
        <xsl:value-of select="translate(@date-created,'T',' ')"/>
      </wp:comment_date>
      <wp:comment_date_gmt>
        <xsl:value-of select="translate(@date-created,'T',' ')"/>
      </wp:comment_date_gmt>
      <wp:comment_content>
        <xsl:value-of select="content"/>
      </wp:comment_content>
      <wp:comment_approved>
        <xsl:choose>
          <xsl:when test="@approved='true'">1</xsl:when>
          <xsl:otherwise>0</xsl:otherwise>
        </xsl:choose>
      </wp:comment_approved>
      <wp:comment_type></wp:comment_type>
      <wp:comment_parent>0</wp:comment_parent>
    </wp:comment>
  </xsl:template>

  <xsl:template match="post/categories/category">
    <category>
      <xsl:variable name="catref" select="@ref" />
      <xsl:value-of select="/blog/categories/category[@id=$catref]/title"/>
    </category>
  </xsl:template>

  <xsl:template name="topubdate">
    <xsl:param name="date" />
    <xsl:value-of select="substring($date,9,2)" />
    <xsl:value-of select="' '" />
    <xsl:call-template name="monthname">
      <xsl:with-param name="month" select="substring($date,6,2)" />
    </xsl:call-template>
    <xsl:value-of select="' '" />
    <xsl:value-of select="substring($date,1,4)" />
    <xsl:value-of select="' '" />
    <xsl:value-of select="substring($date,12,8)" /> +0000
  </xsl:template>

  <xsl:template name="monthname">
    <xsl:param name="month" />
    <xsl:choose>
      <xsl:when test="$month='01'">Jan</xsl:when>
      <xsl:when test="$month='02'">Feb</xsl:when>
      <xsl:when test="$month='03'">Mar</xsl:when>
      <xsl:when test="$month='04'">Apr</xsl:when>
      <xsl:when test="$month='05'">May</xsl:when>
      <xsl:when test="$month='06'">Jun</xsl:when>
      <xsl:when test="$month='07'">Jul</xsl:when>
      <xsl:when test="$month='08'">Aug</xsl:when>
      <xsl:when test="$month='09'">Sep</xsl:when>
      <xsl:when test="$month='10'">Oct</xsl:when>
      <xsl:when test="$month='11'">Nov</xsl:when>
      <xsl:when test="$month='12'">Dec</xsl:when>
    </xsl:choose>
  </xsl:template>
  <xsl:template match="text()" />
</xsl:stylesheet>

In conclusion

I really don’t want to give up email addresses and IP addresses which gives me two options:

  1. Write an ASPX page that rips the content directly out of the subtext tables and formats it as WXR bypassing BlogML
  2. Extend the Subtext export facility to add the missing fields and transform them from there

I’ll let you know where I go from here…

[)amien

Subtext .NET blogging system 1.9 released

My favorite .NET blogging system, Subtext, is celebrating it’s first .NET 2.0 compatible release known as Subtext 1.9.

Haack and the team have worked hard on getting it out so here’s a thanks to them.

I’ve deployed it here with only a couple of minor issues which are most likely due to me previously running a self-modified version off the Subversion trunk. As a result it didn’t quite detect that it needed to upgrade the database nor that my skin was missing and it should use something else for now.

If anyone else has the same problem simply remove the 1.9.0 record from your subversion_version table and it should upgrade the tables and stored procedures appropriately.

Now Subtext is a Web Application Project you can add a deployment project so deploying to live is only two clicks away (you’ll need to install Web Application Project and Web Deployment Project upgrades to VS 2005 first until SP1 gets here)

  1. Right-click on Subtext.Web and choose Add Web Deployment Project
  2. Accept the defaults by clicking OK
  3. Right mouse button on your new Subtext.Web.csproj_deploy deployment project and choose Property Pages
  4. Set the output folder to the path to your blog’s home on your live server and hit OK

So now we have almost what we need but we don’t want the live server being deployed to every time we build so:

  1. Right-click on the SubtextSolution and choose Configuration Manager
  2. Un-check the build for Subtext.Web.csproj_deploy and press Close

Now you can modify, run, debug and test safely on your dev machine until you are happy it’s ready for live.

To deploy to live either build in Release mode or right-click on Subtext.Web.csproj_deploy and choose build.

You can also have the deployment project replace sections of the web.config on the target machine – useful when your live server connects to a different database for example.

[)amien