One of the most common questions I’ve heard asked from Eucalyptus users is this one: “how easy is it to convert an AMI to an EMI?” And the answer is: not as easy as it should be.
We’ve got some process guidelines on our wiki, thanks to Tim Gerla — but we should have the ability to do much of this automagically. So that’s the tool I’ve started work on. In the spirit of “release early release often”, I’ve uploaded a very early iteration of this tool. Find the brand new Github repo here.
It’s quite a ways yet from being prime-time, but it’s allowed me to do quite a bit of testing. Some assumptions I’ve started with:
* First, I pulled a list of all public AMIs on us-east-1. There were about 20,000 public images available there as of mid-November.
* Then I selected the subset of AMIs that were built with a PV-GRUB kernel, and I’m importing them to an instance of Eucalyptus running the kexec-loader kernel. In both cases, the AKI/EKI is just a bootstrapping mechanism that then hands control over to the image’s own kernel, so we shouldn’t get caught up by kernel incompatibilities. Using only the subset of kernels with PV-GRUB AKIs leaves us with about 7000, and picking one particular AKI (aki-825ea7eb) gives us about 1700.
* From there, we’re working with individual distros, and there will be idiosyncracies between the various distros out there, so it makes sense to pick one and go with it. There’s a lot of Ubuntu out there on AWS, so I just grepped on AMIs with “ubuntu” in the name. That’s 1067 images in my dataset.
* Now that we’ve got a reasonable set of images to examine, here’s the process that the scripts walk through. For each AMI, we start an AWS instance, ssh to that instance, install euca2ools if they aren’t already installed, scp Euca credentials to the instance, and bundle the instance to the specified Euca cloud. Then we fire up the resultant EMI on the Eucalyptus side and see if we can ssh in. And we bail with appropriate error messages at various places along the line.
I’ve run through a few hundred images at this point. Not one of them has been completely successful from start to finish. About 10% so far have yielded a bundled EMI that boots and yields a Eucalyptus instance in a running state. Can’t ssh into any of them yet, though.
The good news is that the failures are all quite specific and predictable, and the next steps are clear. Do a better job of guessing login IDs. Figure out why fstabs fail. Look for rogue kernel modules. Make sure we’re doing key injection properly on the Euca end. Comb through the results of euca-get-console-output and look for patterns. The big win is having tools that allow us to do that work in minutes, rather than in hours or days. Every step gets us closer to the goal of fully automated conversions on the fly.
Oh, and an apology: a lot of this should probably have been written using Eutester, instead of as a bunch of shell scripting. The fact is, I’m a terrible hack. But I’m just leading the charge temporarily; when I’ve figured out the basics, the real coders can swoop in and do things the right way. In the meantime… patches welcome.
Greg DeKoenigsberg Speaks 