farblog

Google Compute Engine is Google’s “run a virtual machine on Google infrastructure” product. It’s broadly similar to Amazon’s EC2, in that you get an unmanaged (Linux) virtual machine that you can run pretty much anything on, one difference being that it seems to be aimed at larger workloads: 16-core machines with hundreds of GB of RAM, 5TB disks, that kind of thing.

While I’d been meaning to look at it for a while, I didn’t think I had any reason to use it; I certainly don’t have any workloads of the scale people seem to be talking about. A short while ago, a friend at work mentioned he was using it to run a private Minecraft server, which seemed pretty small to me, so I thought perhaps I’d take another look.

It turns out that Compute Engine is just as suited to small-scale workloads as large ones, and while you do have to pay to use it, it works out to be pretty inexpensive. Having spent a little time with it now, I figured it was time to document what I found out.

Boring disclaimer time first, though: I don’t work on Compute Engine, so this isn’t anything official, just some guy on the internet. Also, in the interests of full disclosure: I’m getting an employee discount on the cost of using Compute Engine (though it’s cheap enough that I’d be happy paying full price anyway). With that in mind…

Stalkers and readers with good memories will recall that I started proxying this site via Google’s PageSpeed Service a little over two years ago. PageSpeed Service is a reverse proxy running in Google’s data centres that applies various performance rewrites to the original content (minifying CSS, and so on), and it does a pretty good job overall. As an additional benefit, it’s a (short TTL) caching proxy, so nobody needs depend directly on the copy of Apache running at the end of a DSL pipe on my server at home.

However, I’ve always been slightly bothered by the fact that that dependency still exists. There’s the usual “home network isn’t very reliable” problem¹, but rather more importantly, that server’s on my home network, and given the choice, I’d rather not have it running a public copy of Apache as well as everything else.

Anyway, it turns out that I’m going to need to reinstall that server in a bit anyway, so I figured that it might be a good time to see whether Compute Engine was a good fit to run a simple low-traffic Apache server like the one that serves this site (spoiler: yes).

I was hoping that I’d have something clever to say about what I needed to do to set it up, but in truth the Compute Engine quickstart is almost embarrassingly easy, and ends up with a running copy of Apache, not far from where I needed to be.

One thing I did decide to do while experimenting was to script the whole install, so that a single script creates the virtual machine (the “instance”), installs everything I need, and sets up Apache to serve this site. Partly² this was to make sure I recorded what I’d done, and partly so that I could experiment and reset to a clean state when I messed things up.

That may have been a bit excessive for a simple installation, but it does mean that I now have good documentation that I can go into some detail about.

First, create your instance

With Compute Engine, the first thing you need to do (assuming you’ve completed the setup in the quickstart) is to create an instance, which is what Compute Engine calls a persistent virtual machine. My script ended up using something like the following, which creates an instance together with a new persistent disk to boot from:

$ gcutil --project=farblog addinstance www \
         --machine_type=f1-micro \
         --zone=us-central1-b \
         --image=debian-7 \
         --metadata_from_file=startup-script:startup.sh \
         --authorized_ssh_keys=myuser:myuser.pub \
         --external_ip_address=8.35.193.150 \
         --wait_until_running

gcutil is the command-line tool from the Google Cloud SDK that allows you to configure and control everything related to Compute Engine (other tools in the SDK cover App Engine, Cloud Storage, and so on, but I didn’t need to use any of those).

Taking it from the top, --project specifies the Google Developers Console project ID (these projects are just a way to group different APIs for billing and so on; in this case, I’m only using Compute Engine). You can also ask gcutil to remember the project (or any other flag value) so that you don’t need to keep repeating it.

addinstance is the command to add a new instance, and www is my (unimaginative) instance name. Everything after this point is optional: the tool will prompt for the zone, machine type, and image, and use sensible defaults for everything else.

The machine type comes next: f1-micro is the smallest machine type available, with about 600MB RAM and a CPU reservation suitable to occasional “bursty” (rather than continuous) workloads. That probably wouldn’t work for a server under load, but it seems to be absolutely fine for one like mine, with a request rate measured in seconds between requests, rather than the other way around.

Next is the zone (us-central1-b), where the machine I’m using will be physically located. This currently boils down to the choice of few different locations in the US and Europe (at the time of writing, four different zones across two regions named us-central1 and europe-west1). As with Amazon, the European regions are slightly more expensive (by about 10%) than the US ones, so I’m using a zone in a US region.

While Compute Engine was in limited preview, the choice of zone within a region was a bit more important, as different zones had different maintenance schedules, and a maintenance event would shut down the whole zone for about two weeks, requiring you to bring up another instance somewhere else. However, the US zones no longer have downtime for scheduled maintenance: when some part of the zone needs to be taken offline, the instances affected will be migrated to other physical machines transparently (i.e. without a reboot).

This is pretty awesome, and really makes it possible to run a set-and-forget service like a web server without any complexity (or cost) involved in, for example, setting up load balancing across multiple instances.

After the zone, I’ve specified the disk image that will be used to initialise a new persistent root disk (which will be named the same as the instance). Alternatively, rather than creating a new disk from an image, I could have told the instance to mount an existing disk as the root disk (in either read/write or read-only mode, though a given disk can only be mounted read/write by one instance at any time).

The image really is just a raw disk image, and it appears can contain pretty much anything that can run as an x86-64 KVM guest, though all the documentation and tools currently assume you’ll be running some Linux distribution, so you may find it it a little challenging to run something else (though plenty of people seem to be).

For convenience, Google provides links to images with recent versions of Debian and CentOS (with RHEL and SUSE available as “premium” options), and above I’m using the latest stable version of Debian Wheezy (debian-7, which is actually a partial match for something like projects/debian-cloud/global/images/debian-7-wheezy-v20131120).

Continuing with the options, --metadata_from_file and --authorized_ssh_keys are two ways to specify key/value metadata associated with the instance. In this case, the first option sets the metadata value with the key startup-script to the contents of the file startup.sh, while the second sets the metadata value with the key sshKeys to a list of users and public keys that can be used to log into the instance (here, myuser is the username, and myuser.pub is the SSH public key file).

Both of these are specific to the instance (though it’s also possible to inherit metadata set at the project level), and can be queried — along with a host of other default metadata values — from the instance using a simple HTTP request that returns either a text string or JSON payload.

I’m not going to go into metadata in any detail other than the above built-ins, but it looks to be pretty powerful if you need any kind of per-instance customisation.

The startup-script metadata value is used to store the contents of a script run (as root) after your instance boots. In my case, I’m just using this to set the hostname of my instance, which was otherwise unset³, which in turn makes a bunch of tools throw warnings. I found the easiest way to fix this was to specify a startup script containing just hostname www.

The sshKeys metadata value is used to store a list of users and SSH public keys. This is read by a daemon (installed as /usr/share/google/google_daemon/manage_accounts.py) that ensures that each listed user continues to exist (with a home directory, .ssh/authorized_keys containing the specified key, etc), and also ensures that each listed user is a sudoer (present in /etc/sudoers)⁴.

Note that you don’t need to specify any of this at all. By default, Compute Engine creates a user account on your instance with a name set to your local login name, and creates a new ssh keypair that it drops into ~/.ssh/google_compute_engine{,.pub} on the machine you created the instance from. You can then simply use gcutil ssh instance-name to ssh into the instance.

This is helpful when you’re getting started, but it does mean that if you want to ssh from anywhere else, you either need to copy those keys around, or do something like the above to tell Compute Engine to accept a given public key. Since I wanted to be able to ssh programmatically from machines that didn’t necessarily have gcutil installed, I found it simpler to just create an ssh keypair manually, specify it as above, and use standard ssh to ssh to the instance.

--external_ip_address allows you to choose a “static” external IP address (from one you’ve previously reserved). Otherwise, the instance is assigned⁵ an ephemeral external IP address chosen when the instance is created. This is is reclaimed if you delete the instance, so you probably don’t want to rely on ephemeral IP addresses as the target of e.g. a DNS record.

However, you can promote an ephemeral address that’s assigned to an instance so that it becomes a static address, and there’s no charge for (in-use) static addresses, so there’s no problem if you start using an ephemeral address and later want to keep it. (Strictly speaking, external IP addresses are actually optional, as all of your instances on the same “network” can talk to each other using internal addresses, but this isn’t something simple installations are likely to use, I wouldn’t have thought.)

Compute Engine doesn’t currently have support for IPv6, oddly, though there’s a message right at the top of the networking documentation saying that IPv6 is an “important future direction”, so hopefully that’s just temporary. (EC2, for what it’s worth, doesn’t support IPv6 on their instances either, though their load balancers do, so you can use a load balancer as a [costly] way to get IPv6 accessibility.)

Finally (phew!), --wait_until_running won’t return until the instance has actually started booting (typically about 25 seconds; you can add a brand new instance and be ssh’d into a shell in less than a minute.) Note that the machine won’t have any user accounts until the initial boot has finished, so if you’re scripting this you’ll need to spin a bit until ssh starts working.

Configure your instance

I did need to spend a fair amount of time working out how to configure the instance once it existed, but that was mostly because I’m not too familiar with Debian.

There isn’t a great deal to say about this part (and obviously it’ll depend upon what you’re doing), but in my case I simply ran sudo apt-get install to get the packages I needed (apache2 and mercurial, and a few others like less and vim), downloaded and installed mod_pagespeed, the Apache module that does the same thing as PageSpeed Service, built my site, and set up Apache to serve it.

There are still two things I’m not quite happy with:

• Automated updates. This has nothing to do with Compute Engine, just Debian: I’m currently using the unattended-upgrades Debian package, which I believe I’ve now configured to apply security updates correctly, but I don’t fully understand what options I have here, or whether I have in fact configured it correctly.
• Mail delivery. For probably-reasonable reasons, Compute Engine blocks anything on an outbound SMTP port. I don’t need to send mail generally, but I’d like a way to forward notices to root@. The official solution involves setting up an account with a third-party and then using an SMTP smart host running on a non-standard port. This would work, but it sounds like massive overkill for notices-to-root, so I’m currently thinking about using something like fetchmail (on another machine) in ODMR mode to inject mail to an existing account.

Cost

I’d estimate the monthly charge for my single instance to be around $15 + VAT (before the discount I’m getting). If I have the numbers right, that’s about what I’m paying for electricity to run my (not very efficient) server at home at present.

That price is dominated by the cost of the machine, which from the pricing documentation is currently $0.019/hour; the disk and network cost (for me) is going to end up at significantly less than a dollar a month.

I mention VAT above because something that’s not currently clear from the pricing documentation is that individuals in the EU pay VAT on top of the quoted prices (likewise true for EC2). Businesses are liable for VAT too, but they’re responsible for working out what to pay themselves, and do so separately.

One other aspect of the tax situation is a bit surprising (for individuals again, not businesses): VAT for Compute Engine is charged at the Irish VAT rate (23%), because when you’re in the EU, you’re paying Google Ireland. (This is in contrast to Amazon, who charge the UK rate even though you’re doing business with Amazon Inc. - tax is complicated.) Admittedly, the difference on the bill above is less than 30p/month, but it still took a little bit of time to figure out what was going on.

tl;dr

Despite all the talk of “big data” and large-scale data processing, is Compute Engine a viable option for running small-scale jobs like a simple static web server? Absolutely.

And while it’s easy to get started, it also looks like it scales naturally: I haven’t looked at load balancing (or protocol forwarding) in any detail, but everything else I’ve read about seems quite powerful and easy to start using incrementally.

From the management side, I’m impressed by the focus on scriptability: gcutil itself is fine as far as it goes, but the underlying Compute Engine API is documented in terms of REST and JSON, and the Developers Console goes out of its way to provide links to show you the REST results for what it’s doing (as it just uses the REST API under the hood). There are also a ton of client libraries available (gcutil is written against the Python API, for example), and support from third-party management tools like Scalr.

I still don’t think that I personally have any reason to use Compute Engine for large-scale processing, but I’m quite happy using it to serve this content to you.