Posts tagged with aws

Random tips for PowerShell, Bash & AWS

Now that I am again freelancing, I find myself solving unusual issues, many of which had no online solutions.

Given these no doubt plague other developers, let’s share!

Pass quoted args from BAT/CMD files to PowerShell

Grabbing args from a batch/command files is easy – use %* – but have you ever tried passing them to PowerShell like:

powershell "Something" "%*"

Unfortunately, if one of your arguments has quotes around it (a filename containing a space perhaps), it becomes two separate arguments. e.g. "My File.txt" now becomes My and File.txt.

PowerShell will only preserve it if you use the -f option (to run a .PS1 file) and that requires a relaxed policy via Set-ExecutionPolicy so is a no-go for many people.

Given you can’t make PowerShell do the right thing with the args the trick here is - to not pass them as args at all!

SET MYPSARGS=%*
...
powershell -ArgumentList "$env:MYPSARGS"

Get Bash script path as Windows path

While Cygwin ships with cygpath to convert /c/something to c:\Something etc. MSYS Bash shells do not have this. You can get it another way there however:

#!/bin/sh
pushd "$(dirname "$0")" > /dev/null
if command -v "cygpath" > /dev/null; then
  WINPWD=""$(cygpath . -a -w)""
else
  WINPWD=""$(pwd -W)""
fi
popd > /dev/null
echo $WINPWD

This solution works by switching the working directory to the one the script is in "$(dirname "$0")" and then capturing the print-working-directory command output using the -W option that grabs it in Windows format. It then pops the working directory to make sure it goes back to where it was.

Note that this uses forward slashes as a directory separator still. Many tools and apps are okay with that but some older ones are not.

JSON encoding in API Gateway mapping templates

If you use Amazon’s AWS Lambda you’ll also find yourself touching API Gateway. While most of it is great, the mapping templates are deficient in that they do not encode output by default despite specifying the MIME types.

All of Amazon’s example templates are exploitable via JSON injection. Just put a double-quote in a field and start writing any JSON payload.

Amazon must fix this – encode by default like other templating systems have done, such as ASP.NET Razor. Until then some recommend the Amazon-provided $util.escapeJavaScript() however while it encodes " as \" it also produces illegal JSON by encoding ' as \' .

The mapping language is Apache Velocity Template Language (VTL), and while not extendable, the fine print reveals that it internally uses Java strings and does not sandbox them which let’s us utilize Java’s replace functionality:

#set($i = $input.path('$'))
{
   "safeString": "$i.unsafeString.replaceAll("\""", "\\""")
}

Show active known IPs on the local network

I’m surprised more people don’t know how useful arp -a is, especially if you pipe it into ping…

Bash

arp -a | grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' | xargs -L1 ping -c 1 -t 1 | sed -n -e 's/^.*bytes from //p'

PowerShell

(arp -a) -match "dynamic" | Foreach { ping -n 1 -w 1000 ($_ -split "\s+")[1] } | where { $_ -match "Reply from " } | % { $_.replace("Reply from ","") }

Wrapping up

I just want to mention that if you are doing anything on a command-line, be it Bash, OS X, PowerShell or Command/Batch then SS64 is a site worth visiting as they have great docs on many of these things!

[)amien

Monitoring URLs for free with Google Cloud Monitor

As somebody who runs a few sites, I like to keep an eye on them and make sure they’re up and responding correctly.

My go-to for years has been Pingdom, but this year they gutted their free service (update 2021 it’s toally killed and now owned by Solarwinds… yeah, the people who got hacked and unknowingly distributed a back door to all their customers) so maybe not that service.

The free service with Pingdom also had limited alerting options and can only monitor a single endpoint. Instead, I went looking for something better as $15 a month to monitor a couple of personal low-volume sites is not money well spent.

Google Cloud

I’ve played with the Google Cloud Platform offerings for a while, and like many others, theirs includes a monitoring component called unsurprisingly Google Cloud Monitoring.

It’s currently free in beta and is based on StackDriver - acquired by Google in 2014. I can imagine more integration and services to continue to come through as they have a complete product that also monitors AWS.

Uptime checks

Screenshot showing uptime check options

You can create HTTP/HTTPS/TCP/UDP checks, and while designed to monitor the services you’re running on Google Cloud, will happily take arbitrary URLs to services running elsewhere.

Checks can be run every 1/5/10 or 15 minutes, use custom ports, look for specific strings in the response and setting custom headers and authentication credentials.

Each URL is monitored and reported from six geographical locations. They are split between three in the USA (east, central and west), Europe, Asia and Latin America. For example:

damieng.com

  • Virginia responded with 200 (OK) in 357 ms
  • Oregon responded with 200 (OK) in 377 ms
  • Iowa responded with 200 (OK) in 330 ms
  • Belgium responded with 200 (OK) in 673 ms
  • Singapore responded with 200 (OK) in 899 ms
  • Sao Paulo responded with 200 (OK) in 828 ms

Alerting policies

Here’s where Google’s offering surprised me. It has alerting options for SMS and Email, obviously, but also HipChat, Slack, Campfire, and PagerDuty. You can specify combinations together, mixing and matching with different uptime checks etc.

Screenshot of alerting policy options

Incidents

Like Pingdom, if the endpoint monitored goes down, an incident is opened. You can write details (comments) to the incident, and like Pingdom, the incident is closed once the endpoint starts responding again.

Graph & dashboard

The cloud monitoring product has a configurable dashboard geared around monitoring Google Cloud specific services. There is an uptime monitoring component that still provides some value.

You can download the JSON for a graph, an API as well as iframe sharing functionality.

Final thoughts

I’m very impressed with this tool given the lack of limitations in a free product. I am using it for my sites, but it has no SLA right now!

Any other recommendations for free URL monitoring?

[)amien