A blog comment receiver for Cloudflare Workers

24 October 2023 Development Cloudflareweb development

Background

A number of years back I switched to a static site generator for damieng.com, firstly with Jekyll, and then with Nuxt when I wanted more flexibility. I've been happy with the results and the site is now faster, cheaper, more secure and easier for me to maintain.

The tricky part was allowing user comments without a third-party service like Disqus. I came up with a solution that has comments into markdown files just like the rest of the site content so they can be published as part of the build process.

The accepting new comments side of things was an Azure function that received a comment via a form post and created a PR. If I approve the PR it gets merged and that automatically triggers a site rebuild to bring it in.

Perfect! Or so I thought...

Azure problems

I could go into a lot of detail here but suffice to say that while a million executions per month are free for a function storing it's data in Azure Storage is not and Microsoft will bill you $0.02-$0.03 a month. That's fine until your credit card expires and you discover Azure's billing system is hard-wired to only accept credit cards matching the country you signed up with and you can't change country. (I've moved from the US to the UK since setting up my account there)

So because Microsoft couldn't charge me $0.03 they notified me of intent to shut down my service and the simplest solution was just to rewrite the function to run elsewhere (it was in C#).

Given card processors charge almost 10x this to process the payment Microsoft must lose money by billing people these trivial amounts.

I do not recommend using Azure if you might move country.

Cloudflare Workers to the rescue

So I went ahead and rewrote my comment receiver on my current provider of choice - Cloudflare. I've had success with their Pages product with Nuxt and their free tiers are very generous (100,000 per day for workers).

This time however I thought I'd code it directly for Cloudflare Workers in TypeScript without using Nuxt - just as an experiment more than anything else.

Here's how things went.

Project setup

You start by using the templating tool for Cloudflare by running:

npm create cloudflare@2

Then give your function a name, choose "Hello World" Worker as the template and yes to Typescript.

This will create a new folder with a package.json and tsconfig.json and a src/index.ts file with a simple worker that returns a Hello World response.

Now we'll install one dependency (GitHub's API is trivially easy to use without a library but we're not going near home-rolled YAML encoding) and open up VS Code (or your editor of choice):

cd your-project-name
npm install yaml --save-dev
code .

Okay, time to edit some files!

The code

Replace the contents of src/index.ts with the following which is also available as a Gist:

import { stringify } from "yaml"

export default {
  async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
    // Make sure this is a POST to /post-comment
    if (request.method !== "POST" || new URL(request.url).pathname !== "/post-comment") {
      return new Response("Not found", { status: 404 })
    }

    // We only accept form-encoded bodies
    if (request.headers.get("content-type") !== "application/x-www-form-urlencoded") {
      return new Response("Bad request", { status: 400 })
    }

    // Get and validate the form
    const form = await request.formData()
    const validationError = validateForm(form)
    if (validationError) {
      return validationError
    }

    // Validate the Turnstile recaptcha if configured to do so
    if (env.TURNSTILE_SECRET_KEY) {
      const passedTurnstile = await isTurnstileValid(form.get("g-recaptcha-response") ?? "")
      if (!passedTurnstile) {
        return new Response("Failed Turnstile validation", { status: 403 })
      }
    }

    // Details required for the branch/filename
    const commentId = crypto.randomUUID()
    const postId = form.get("post_id")?.replace(invalidPathChars, "-")

    // Get the starting point for the github repo
    const repository = await github()
    const defaultBranch = await github(`/branches/${repository.default_branch}`)

    // Create a new branch for the comment
    const newBranchName = `comments-${commentId}`
    await github(`/git/refs`, "POST", {
      ref: `refs/heads/${newBranchName}`,
      sha: defaultBranch.commit.sha,
    })

    // Create a new file for the comment
    const frontmatter = {
      id: commentId,
      date: new Date().toISOString(),
      name: form.get("name") ?? undefined,
      email: form.get("email") ?? undefined,
      avatar: form.get("avatar") ?? undefined,
      url: form.get("url") ?? undefined,
    }

    await github(`/contents/content/comments/${postId}/${commentId}.md`, "PUT", {
      message: `Comment by ${form.get("name")} on ${postId}`,
      content: btoa("---\n" + stringify(frontmatter) + "---\n" + form.get("message")),
      branch: newBranchName,
      author: {
        name: form.get("name"),
        email: form.get("email") ?? env.FALLBACK_EMAIL,
      },
    })

    // Create a pull request for it
    await github(`/pulls`, "POST", {
      title: `Comment by ${form.get("name")} on ${postId}`,
      body: form.get("message"),
      head: newBranchName,
      base: repository.default_branch,
    })

    // Redirect to the thanks page
    return Response.redirect(env.SUCCESS_REDIRECT, 302)

    async function github(path: string = "", method: string = "GET", body: any | undefined = undefined): Promise<any> {
      const request = new Request(`https://api.github.com/repos/${env.GITHUB_REPO}${path}`, {
        method: method,
        headers: {
          Accept: "application/vnd.github+json",
          Authorization: `Bearer ${env.GITHUB_ACCESS_TOKEN}`,
          "User-Agent": "Blog Comments via PR",
          "X-GitHub-Api-Version": "2022-11-28",
        },
        body: body ? JSON.stringify(body) : undefined,
      })

      const response = await fetch(request)
      if (!response.ok) {
        throw new Error(`GitHub API returned ${response.status} ${response.statusText}`)
      }
      return await response.json()
    }

    async function isTurnstileValid(clientTurnstile: string): Promise<boolean> {
      const form = new FormData()
      form.set("secret", env.TURNSTILE_SECRET_KEY)
      form.set("response", clientTurnstile)
      form.set("remoteip", request.headers.get("CF-Connecting-IP") ?? "")
      const response = await fetch("https://challenges.cloudflare.com/turnstile/v0/siteverify", {
        body: form,
        method: "POST",
      })

      if (!response.ok) return false
      const json = (await response.json()) as any
      return json.success === true
    }
  },
}

function validateForm(form: FormData): Response | undefined {
  if (form === null) return new Response("Form not decoded", { status: 400 })

  // Validate the form fields
  if (isMissingOrBlank(form.get("post_id"))) return new Response("post_id must not be empty.", { status: 422 })

  if (reservedIds.test(form.get("post_id") ?? ""))
    return new Response("post_id must not use reserved Windows filenames.", {
      status: 422,
    })

  if (isMissingOrBlank(form.get("message"))) return new Response("message must not be empty.", { status: 422 })

  if (isMissingOrBlank(form.get("name"))) return new Response("name must not be empty.", { status: 422 })

  // Validate the email if provided
  if (!isMissingOrBlank(form.get("email"))) {
    if (!validEmail.test(form.get("email") ?? ""))
      return new Response("email must be a valid email address if supplied.", {
        status: 422,
      })
  }

  // Validate the website URL if provided
  if (!isMissingOrBlank(form.get("url"))) {
    try {
      new URL(form.get("url") ?? "")
    } catch {
      return new Response("url must be a valid URL if supplied.", {
        status: 422,
      })
    }
  }
}

function isMissingOrBlank(str: string | null): boolean {
  return str === null || str === undefined || str.trim().length === 0
}

export interface Env {
  FALLBACK_EMAIL: string
  SUCCESS_REDIRECT: string
  GITHUB_REPO: string
  GITHUB_ACCESS_TOKEN: string
  TURNSTILE_SECRET_KEY?: string
}

const invalidPathChars = /[<>:"/\\|?*\x00-\x1F]/g
const validEmail = /^[^\s@]+@[^\s@]+\.[^\s@]+$/
const reservedIds = /CON|PRN|AUX|NUL|COM[1-9]|LPT[1-9]/i

Configuring API keys

Now you're going to obtain some API keys. You can initially store these in the .dev.vars file using a KEY=value format one per line. It will look a little like:

GITHUB_ACCESS_TOKEN=github_pat_[rest of your personal access token]
TURNSTILE_SECRET_KEY=0x[rest of your Turnstile secret key]

GitHub access token

Now you'll need to generate a GitHub personal access token with:

  • Read access to commit statuses and metadata
  • Read and Write access to code and pull requests

Store the access token GitHub gives you with the KEY GITHUB_ACCESS_TOKEN.

If you want to use the Cloudflare Turnstile service to validate the recaptcha then you'll need to generate a private key and store it in the file with the KEY TURNSTILE_SECRET_KEY. You'll also need to follow the Turnstile client-side render guide on how to integrate the client-side portion and ensure you send the value you get back from the service client side to the server with the name g-recaptcha-response.

Set this in the config file with the key TURNSTILE_SECRET_KEY.

Non-secret configuration

We're almost done now you need to modify the wranger.toml file in your project root with the contents:

name = "create-comment-pr"
main = "src/index.ts"
compatibility_date = "2023-10-10"

[vars]
FALLBACK_EMAIL = "[email protected]"
GITHUB_REPO = "myorg/mysiterepo"
SUCCESS_REDIRECT = "https://mywebsite.com/thanks"

Testing

Now you can start the local test server (which will also handle the Typescript without needing a manual build step) with:

wrangler dev

And test it either by submitting to localhost:8787 or by using a too like Thunder Client in VS Code.

Deploying

Now you're ready to deploy to production!

wrangler secret put GITHUB_ACCESS_TOKEN
[enter your GitHub personal access token]
wrangler secret put TURNSTILE_SECRET_KEY
[enter your Turnstile secret key]
wrangler deploy

Now change your testing tool to published URL and try it out and if that works update your form post to the new location.

Enjoy!

Damien

0 responses

  1.