Being on the napkin

Alistair, I loved your napkin-based comparison of the private cloud players. ¬†It’s hilarious, and I agree with most of it. Yes, the private cloud discussion definitely shares parallels with the platform wars of the past. It’s a very useful lens for viewing our little world… even if that lens happened to be the bottom of a wine bottle. ūüôā

First of all, I’ve got to say that it’s great to see Eucalyptus in the center of your napkin. Look at the size of those other players: put together the combined market cap of the OpenStack companies, VMWare, and Citrix… and then look at little old us. We must be doing something right!

What really interests me, though, is where you tried to fit Eucalyptus into your napkin analysis. I suspect that you weren’t quite sure where to put us, so you chose OS/2, found some surface similarities, and moved on to the other parts of the napkin.

In the immortal words of the great sage Jules Winnfield, “allow me to retort.”

“Integration happens when people rally around one thing.” Yes, this is exactly the right argument — and customers are rallying around AWS. Those guys are farther to the upper right in the latest Gartner Magic Quadrant than I’ve ever seen. ¬†They’re doing more public cloud business than all the other players in that market combined right now. Which is why we, and our users and customers, are rallying around one thing: AWS compatibility.

“Apps matter more than legacy protocols.” This, also, is exactly right. Developers are writing apps for the cloud — and that generally means writing them for AWS first. And developers are busy, which means that despite their good intentions to make their apps portable, portability generally comes somewhere between localization and database normalization on the priority list. Which is, of course, why legacy players like VMWare are fighting tooth-and-nail to protect their own legacy protocols. Just listen to Pat Gelsinger, CEO of VMWare: “if a workload goes to Amazon, you lose, and we have lost forever.” ¬†That’s precisely why the value we add is so useful to our users and customers: the switching costs from AWS to Eucalyptus (and back) are orders of magnitude simpler than any other option.

“IBM spent a lot of time making OS/2 work with legacy mainframe protocols and existing enterprise environments.”

Wait… did you just compare AWS to the IBM System/360?

“…and Eucalyptus is OS/2.”



OK, story time.

I worked at IBM way back in the day, fresh out of no college. I was the designated worldwide global support guru for the Audiovation¬†Sound Card , for Microchannel, for OS/2. This was a combination of hardware and software that was never tested, and thus never worked. So being “worldwide global support guru” basically meant picking up the phone and saying “that combination doesn’t work, send the card back, here’s your case number, have a nice day.” And customers would always ask, irritatedly, “you made all these products — how is it possible that they don’t work together?” And my inability to lie about the answer to that question — “because none of these three components are important individually in the market, let alone collectively” — was probably a contributing factor to my getting fired from that job.

In my opinion, the chief failure of OS/2 was in its attempts to be too many things to too many people — being OK at everything, but good at nothing in particular.

That is precisely the opposite of the Eucalyptus strategy, which is to be insanely great at interoperability with the AWS API, the de facto standard for talking to the world’s dominant cloud platform.

I’ve said this before, and I’ll say it again: it’s about focus for us. We are focusing on a very particular pain point, that developers are feeling ever more acutely: the potential for AWS lock-in. Our goal is to provide an open source alternative for those users to mitigate those potential lock-in risks. Focus, focus, focus.

We see the benefits of this focused approach every day. Our roadmap is spread out before us in great detail — and that roadmap, combined with some of the best cloud engineers on the planet, gives us a feature velocity¬†that no one else in the private cloud world can currently match. ¬†Which is why we’re on your napkin, despite being a fraction of the size of the napkin’s other inhabitants.

But in thinking about it, maybe your comparison to old school mainframe connectivity is right, and you’ve just got your timeframes wrong.

Maybe it’s 1965, and OpenStack is Multics, and Red Hat is GE, and Amazon is IBM, and AWS is the brand new System/360, ready to dominate the computing landscape for the next two decades.

Which would make Eucalyptus the open source little brother that the System/360 never had, that has no analogue in the history books, and that could have changed everything. Trouble is, I’m not sure how to fit that on your napkin.

Being on the napkin

Hobos and Vagrants and Quality

I don’t know how many people have actually met Vic Iglesias, our¬†Quality Hobo. ¬†Here’s what he looks like in his natural habitat:

Quality Hobo: both smarter and better looking than you.

It’s really super important not to make Quality Hobo angry. Let me¬†assure you from personal experience: no one wants that.

Here are some things that make Quality Hobo angry (and that you should, therefore, avoid):

  • Stealing Quality Hobo’s cigarettes, electronic or otherwise.
  • Remarking upon Quality Hobo’s resemblance to a certain celebrity.
  • Wasting Quality Hobo’s time with questions about which test cases¬†are most recent.
  • Wasting Quality Hobo’s time by writing tests that don’t integrate¬†into Eutester.
  • Wasting Quality Hobo’s time with questions on how to build your QA¬†environment.
  • Wasting Quality Hobo’s time in any way at all.

Here’s what I’m saying: it was only a matter of time before our Hobo¬†got mixed up with a Vagrant.

Vagrant¬†is incredible, and Mitchell Hashimoto¬†is incredible for¬†creating it. ¬†It’s the sort of tool you try out for a particular¬†reason, and then once you’ve used it, you find a million other reasons¬†to use it. ¬†Last night, as I chased down a bug in Faststart, I got¬†to try out Micro-QA, which is a mashup of Vagrant¬†and Eutester¬†and Jenkins¬†and Ansible and all kinds of stuff, built on top of¬†one of the standard Vagrant boxes for Centos 6.4.

Here’s the full instructions ¬†for getting a fully updated, complete testing environment for Eucalyptus installed on your laptop:

  1. Install Vagrant+Virtualbox on your laptop.
  2. Run “git clone” on the Micro-QA¬†repo.
  3. Run “vagrant up”.
  4. Point your browser to http://localhost:8080.

Aaaaand you’re done.

Micro-QA in action. YOUR TEST IS A FAILURE.

Ease of automated testing is one of those force multipliers that¬†doesn’t seem super-exciting, but really is amazingly¬†super-exciting. ¬†Because here’s the thing: When the cost of¬†testing is higher than basically zero, people don’t bother with it —¬†or, they do a really half-assed job of it and then say “oh yeah,¬†I totally tested that.” Which is waaaaaay worse.

In Micro-QA, we have a tool that brings the¬†cost of automated QA to near-zero. ¬†And not just for QA folks: it’s a tool that can be used by our QA¬†team, our engineering team, our support team, our customers, and our¬†community, all with comparatively little knowledge required. It’s a huge win for us.

And here’s the kicker: it’s not only a QA tool; it’s also a great tool for hybrid cloud¬†diagnostics. ¬†Set up your Euca environment; set up your AWS environment; bake in some tests that run against each; run them on a regular basis; scream when something breaks. ¬†It’s kinda sorta magic.

Anyway. If you’ve got a Euca install, go get Micro-QA running on¬†your laptop, bring it up in your browser, pick some test cases to run, drop the contents of your eucarc file into your test case, and run it. ¬†If¬†it breaks, ping us on Freenode (#eucalyptus-qa) and let us know what broke.

Vic announced Micro-QA¬†less than six months ago. ¬†It was cool at the time, but now it’s way past cool. ¬†I’m really impressed by how far it’s come¬†in such a short time. ¬†It’s like I’m living in the future.

(For the record: Vic is really a super sweetheart of a guy, and I¬†don’t think he actually lives under a bridge at present. I think he¬†may be living Between Two Ferns, though.)

Hobos and Vagrants and Quality

Who cares about AWS compatibility?

Simon Wardley is never shy to share a provocative opinion. ūüôā

A summary of his latest missive: OpenStack is already doomed because of their inability, or unwillingness, to produce AWS clones. There’s nothing new in Simon’s position there, but it’s his bluntest statement of opinion yet on OpenStack’s prospects.

I’m not going to presume to agree or disagree with Simon’s prediction — but in his blog post and the ensuing conversation, I saw a few opportunities to clarify how I think about Eucalyptus and AWS fidelity.

* * * * *

First, on proving AWS fidelity.

Obviously, at Eucalyptus we think deeply about the AWS fidelity problem, and how to approach it. Simon suggests one possible model:

So could CloudStack, Eucalyptus, Open Nebula and some of the OpenStack party create a rich set of AWS compatible environments — of course. But the problem becomes you have to define one thing as the ‘reference’ model. The only way around this that I know is for the groups to create a massive set of test scripts and provide some sort of AWS compatibility service and define that as the reference model and each show compatibility to it and it to AWS. It’s possible, I’ve hinted enough times that people could try that route but there’s no takers so far.

I can’t speak for the other projects, but we find that the best tests of AWS compatibility can be found in the AWS ecosystem itself — which is one of the key advantages of working in such an ecosystem. Chasing a full API is an exhausting process, and can be discouraging, but we’ve had good success by first ensuring compatibility with the most popular open source tools in the AWS world. By moving progressively through these tools, we will cover ever-expanding sections of the API, leaving the dustiest corners of the API for last (perhaps never even to be implemented; after all, an API is ultimately only as useful as the tools that exercise it.)

The Netflix OSS toolchain is a great example of this. The team at Netflix has taken quite a bit of heat for relying so heavily on the AWS family of services, but their decision to open source all of their tools has been a boon to us. They are smart users who exercise AWS at a scale that other users can scarcely imagine, so it’s a safe bet that they’re exercising many of the most interesting parts of the AWS API. We’ve learned much, and proved much, by following their trail.

Of course, we also have our own automated test suite for AWS/Eucalyptus fidelity; we call it Eutester. It’s designed, at least in part, to run identical test cases against both Eucalyptus and AWS, and anyone who wants to test the AWS fidelity of their own IaaS can pick up that code and run with it. That codebase will continue to grow as Eucalyptus grows. Patches welcome, as they say.

* * * * *

Second, on architecting for AWS fidelity.

It seems to be assumed — among some, anyway — that AWS API compatibility is something that can simply be dropped into OpenStack at any time. The trouble is, no one will know how true that is, or isn’t, until they actually do the work.¬†Geoff Arnold’s comments hit the nail on the head:

Load balancing is a great example. For good or ill, Amazon’s Elastic Load Balancer is a cornerstone of web-tier cloud applications architecture. If the OpenStack community was serious about AWS compatibility, the LBaaS team would have established ELB compatibility as a fundamental requirement. It didn’t. On the contrary, much of the preliminary documentation focused on all of the cool features that LBaaS would support that were not available with ELB. By Grizzly, all that we had to show for our efforts was a proof of concept based on a single instance of haproxy. Elastic provisioning was officially out of scope for the core LBaaS effort.

Software design is about choices, and with every choice you make, there’s a chance that you’ve made some other choice impractical, or even impossible. We know that there are, at the very least, syntactic differences between OpenStack and AWS; it is also quite likely that there are deeper semantic mismatches. ¬†It may be that the OpenStack community will be able to bridge both syntactic and semantic mismatches between OpenStack and AWS with ease — but given our experiences, it doesn’t seem likely. The devil is in the details, and one must care deeply about the details in order to conquer that devil.

* * * * *

Which brings me to the last point, which is caring about AWS fidelity.

As Thierry Carrez says,

EC2 API support has always been there in OpenStack. It just never found (yet) a set of contributors that cared enough to make it really shine. Canonical promised it (with AWSOME) then let it go. More recently Cloudscaling promised it, but I’ve seen nothing so far. The next in line might just deliver.

Maybe. ¬†There is great power in being the one who “cares enough”. And Thierry’s response here begs the question: why doesn’t the OpenStack community care more about supporting the various AWS APIs? ¬†(Since EC2 is just the tip of the iceberg.)

That’s a question for the OpenStack community to answer. ¬†In the meantime, I can assure you that, at Eucalyptus, we care deeply about AWS compatibility — as do our users. We work towards that goal tirelessly, every day, and I think it’s safe to say it’s because of that passion that we have taken the lead. And it’s a lead that we have every intention of extending.

Anyway. See some of you at Netflix tonite.

Who cares about AWS compatibility?

Demo of Eucalyptus hotness: 3.3 milestone 6

Our demo day for milestone 6 was yesterday, and it was choice. We’re at feature completeness at this point, and we’re now on final approach for release sometime Soon-ish, as soon as we shake out all the code nasties. We’ve got some good stuff to show off on Vimeo. The basic transcript:

  • 0:00 Eric Choi, Product Mktg Manager, with agenda/housekeeping.
  • 1:25 Tim Cramer, VP of Engineering, sets the table for demos.
  • 3:45 Yours Truly, VP of Community, talks (a little too long) about Eucalyptus compatibility with the AWS Ruby SDK.
  • 16:00 Vic Iglesias, QA Lead, talks about EucaLobo, an amazing fork of ElasticWolf that provides an autoscaling UI for both Eucalyptus and AWS. (Watch this one twice; it’s really the star of the show.)
  • 24:40 Vic Iglesias talks about Micro-QA, the self-contained environment for running automated testing on Eucalyptus installations.
  • 31:30 Chris Grzegorczyk, Co-founder and Chief Architect, shows Asgard and other NetflixOSS tools running on Eucalyptus.
  • 41:25 David Kavanagh, Senior Engineer, shows tagging and other latest updates to the Eucalyptus 3.3 User Console.
  • 45:00 Colby Dyess, Partner Manager, shows Eucalyptus compatibility with the AWS Toolkit for Eclipse.

Exciting, and more proof that when it comes to AWS compatibility in a private IaaS, we’re the only game in town.

Demo of Eucalyptus hotness: 3.3 milestone 6

Proof: Netflix OSS on Eucalyptus

The Netflix OSS team sure knows how to throw a good party. ¬†We’ve been to their meetup twice now, most recently at their event last Wednesday, at which they unveiled the Netflix OSS Prize.¬† At this rate, given the amount of interest they’re generating, they’ll need to rent out hotel space for their next event.

What draws the crowd? It’s a good question. The simplest answer, I guess, is that they’re a cool company doing cool stuff — and people want to be like them.

At Eucalyptus, we certainly enjoy working with Netflix, and we enjoy hanging out with them generally, because they‚Äôre cool. ¬†(And their headquarters are awesome — ‚Äúspa-like‚ÄĚ, one might say.) ¬†But we also have very specific interests in following the Netflix approach:

1. Netflix understands cloud.  Surely this is obvious; Netflix is one of the most advanced user of cloud services on the planet. They were the first to understand the true value of the AWS model, and in taking full advantage of it, they’ve developed a reputation for being industry leaders who work at true cloud scale.  Other leaders now seek to learn from them and emulate their practices.

What does that mean, though — “working at true cloud scale”? In a nutshell: Netflix has embraced the reality that sometimes services just go away. ¬†This is the single biggest shift that one must make when moving into the world of cloud: not just accepting, but embracing the idea that sometimes your systems just go away.¬† That’s the point of cloud, and if you don’t build your systems with that mindset from Day One, you’re Doing It Wrong. Netflix has demonstrated an understanding here that few organizations can match.¬† They’ve moved from Chaos Monkey, which randomly knocks over instances, to Chaos Gorilla, which randomly knocks over entire availability zones — and they’ve made these tools, and others like them, available for those who dare to follow their lead.

2. Netflix understands open source. Not just as users, but as producers. ¬†They are willing to share some pretty amazing software, because they understand the difference between software that provides differentiating value for their business (the engine) and software that provides non-differentiating value (the plumbing). ¬†The more they can share the cost of maintaining the plumbing, the more resources they can commit to the engine. ¬†This is a highly strategic choice, which makes them all the more committed to it — they have 26 projects in their Github repo and counting, with no sign of slowing down anytime soon.

3. Netflix OSS requires real AWS API fidelity, and Eucalyptus provides it.¬† Netflix is completely committed to the AWS model, and all of their code currently assumes that you’re using AWS.¬† If you want to figure out if your own AWS compliant IaaS “just works”, the Netflix OSS tools represent the best possible tests of AWS API fidelity. The “Asg” in Asgard, for instance, stands for Autoscaling Groups — so as we get our own autoscaling functionality up to speed for the Eucalyptus 3.3 release, it makes perfect sense to use Asgard as the benchmark to test against.¬† Which is precisely why we demonstrated at the Netflix OSS event: Chaos Monkey for knocking down instances, Asgard for Autoscaling to replace the destroyed instances, and Edda for auditing the whole process — and all working on Eucalyptus precisely as it would work against AWS.

Here’s the point, and it’s a simple one: it’s easy and cool to claim that your private cloud is “AWS API compatible”. ¬†But it’s another thing entirely to prove it. ¬†At Eucalyptus, we stake our entire reputation on proving it, with every single release. At the Netflix meetup, people walked up to our demo station, and they could *see* Netflix OSS on a private cloud. Edda, Asgard, and Chaos Monkey, all running in the cloud that was sitting right there on the table next to them.

Don’t trust hype. ¬†Trust proof.


Annotation of Euca demo 3.3 milestone 4

We’ve got two hours worth of demo video from yesterday’s show and tell session for Eucalyptus 3.3 milestone 4 — with every milestone, we continue to improve the most compatible AWS private cloud platform.¬† ELB, autoscaling and cloudwatch are pretty much in the bag; now it’s mostly spit and polish. For those who don’t want to watch the whole two hours, but are interested in the progress of various features, here’s the timings of the various demos:

At 2:38, David Kavanagh talks about data management features in the UI.

At 11:00, Ean Schuessler talks about data layer architecture changes in the UI.

At 25:42, Jeff Uphoff at 25:42 talks about migrating instances from one node controller to another.

At 45:15, Vasya Kochergin talks about migrating instances in vmware.

At 51:35, Swathi Gangisetty talks about support for NetApp cluster mode.

At 1:19:30, Steve Jones talks about autoscaling improvements.

At 1:28:56, Ken Edwards talks about cloudwatch improvements.

At 1:37:35, Evan Thomas talks about cloudwatch alarms.

At 1:49:20. Sang-Min Park talks about elastic load balancing.

At 2:05:45, Matt Spaulding talks about the elastic load balancer VM.

Enjoy. As always, if you have any qyestions, feel free to ask on IRC (#eucalyptus on freenode) or join our mailng list.

P.S. we should have Silvereye builds of 3.3m4 available early next week.

Annotation of Euca demo 3.3 milestone 4

Eucalyptus 3.3 Show and Tell. Y’all come!

After every sprint, the Eucalyptus engineers hold a Show and Tell session, where they share what they’ve been working on with everyone else in the company.¬† Technically, these Show and Tell sessions are for “product management validation acceptance” or some such blah blah blah — but I like it because it lets all of the engineers strut their stuff, so they can be stars inside the company.

Well, bleep that.¬† I say let ’em be stars outside of the company too!

Thus: on Wednesday March 13th, at noon Pacific time, we will be opening our Show and Tell to the entire world.¬† It’ll probably run a couple of hours or so.¬† Some of the goodies we’ll be showing for 3.3 Sprint 4:

* nearly full implementations of ELB, Cloudwatch, and Autoscaling

* mechanisms for node evacuation and instance migration

* user console improvements and features

* next generation storage adapters

We’ll run a Gotomeeting session for the audio and screen sharing, and we will be on #eucalyptus-meeting on freenode for those who want to ask questions of the Eucalyptus engineering team — time allowing, of course, since we’re going to have a lot to show off. ūüôā

Sign up now and we’ll see you online on Wednesday.

Eucalyptus 3.3 Show and Tell. Y’all come!

Extending the Eucalyptus lead in AWS compatibility

Recently, we released our latest milestone build for Eucalyptus 3.3. ¬†Go take it for a spin. This is a big one, since it incorporates, for the first time, functional versions of the “Big Three” AWS services we’ll be releasing later this spring: Autoscaling, Elastic Load Balancing, and Cloudwatch. ¬†It also presents a good opportunity to step back and look again at our AWS compatibility story.

It’s no secret that Eucalyptus believes in the power of the Amazon Web Services API. Amazon continues to be the dominant public cloud, and they continue to widen the gap between themselves and the other public cloud providers. ¬†This is thanks, in large part, to an API that is well considered, well documented, and has therefore spawned a powerful and growing ecosystem. ¬†The rule of thumb for those who are developing code for the public cloud has thus become “make it work for Amazon first” — to such a degree that even VMware is running scared.

Such is the strength of the de facto standard. ¬†The abstractions that are now being developed by every other cloud provider are either strongly influenced by, or derived directly from, the abstractions pioneered by AWS. ¬†Every single cloud provider has their own alternatives to EC2 and S3 — but AWS, having mastered these services long ago, is now moving quickly through a set of higher level abstractions. ¬†It is possible — perhaps even likely — that the AWS API, as the continual representative of the newest and best cloud abstractions, will emerge as a dominant standard for cloud applications in much the same way that the LAMP stack emerged as the dominant standard for web applications. ¬†It will simply be assumed that if you want to provision applications rapidly at scale, you will either write an application that uses the AWS API directly, or you will depend upon a PaaS that works first and best with the AWS API.

To us, it has always made perfect sense to follow Amazon’s strong lead.  To build the strongest private cloud, build the best possible complement to the strongest public cloud.  And to ensure that users always have options, make sure that the private cloud in question is open source.

The next, obvious question: how do we do that, exactly?

We’ve been refining our approach for more than half a decade. ¬†Which is about a thousand cloud years. ¬†Our approach can now be boiled down to two fundamentals:

1. Implement services as closely as possible to the way Amazon implements them. ¬†Which means paying close attention to every detail of every API interaction. It means closely tracking all compatibility issues we find, and treating them as critical. ¬†It means writing tons of in-depth tests that can run against both Eucalyptus and AWS. And it means using the AWS WSDL to construct service stubs, which dramatically improves the speed with which we can produce new features, as we’re doing with this release.

2. Leverage the AWS ecosystem to define and test the limits of compatibility. Which means relentlessly testing third-party tools and libraries against Eucalyptus, and being satisfied only when those tools work as well against Eucalyptus as they would work against AWS. Compatibility is a journey, and we will consider ourselves “compatible” when our users can treat Eucalyptus as their own personal region of AWS. ¬†Think of it a hybrid cloud Turing test.

All that said, we strongly encourage the diversity of tools to create a high-level compatibility between AWS and other public cloud services, and we see some value in tools like AWSome and Deltacloud and CloudBridge and the like.  Over time, as other cloud vendors continue to refine their own versions of the AWS abstractions, it will be easier to make common high-level tools that encompass actions across all various public and private clouds, and that may make it a bit easier for developers to write code that is truly portable across many different clouds.  (Then again, maybe not.)  In the meantime, though, we will let others focus on Broad Compatibility, while we continue our laser focus on Deep Compatibility, and the advantages that we will provide to our users as a result.

Anyway. Go install the latest milestone build for yourself.¬† We’ve also got some great demo scripts that will allow you to try out the latest and greatest functionality. Don’t hesitate to drop by on IRC (#eucalyptus on freenode) and let us know what you think.

Extending the Eucalyptus lead in AWS compatibility


The ami2emi project has been moving along. As in, it actually works for a number of cases now. Configure cloud parameters for AWS and Euca, run a script, and boom: your chosen AMIs are brought to life on your Euca cloud. When it works, it Just Works. It’s cool. ūüôā

Note that these cases do *not* yet cover configuration of the applications themselves — just the images.¬† The bits might still need to be twiddled to get the apps working properly, but all of the actual bits are successfully transferred into the new image, and the new image successfully spins up instances, and you can ssh to them and everything.

The AMIs that can currently be auto-slurped into Eucalyptus successfully share certain characteristics:

1. They carry their own kernels inside the image. On the AWS side, that means they’re linked against the stock pv-grub kernels, and we link them similarly to the kexec-loader kernels on the Euca side.

2. They’re instance-store images, rather than EBS-based images. At least so far.

You can find the list of currently tested images here. That list will expand rapidly as we have time to run more test cases.

Note the large number of Bitnami instances in this list. That must be because they’re Crazy Awesome.

So, give it a whirl. Patches exceptionally welcome, since it’s heinous bash scripting and I can use all the help I can get. At least I document my code, sorta. ūüôā


Converting AMIs to EMIs

One of the most common questions I’ve heard asked from Eucalyptus users is this one: “how easy is it to convert an AMI to an EMI?” And the answer is: not as easy as it should be.

We’ve got some process guidelines on our wiki, thanks to Tim Gerla — but we should have the ability to do much of this automagically.¬† So that’s the tool I’ve started work on.¬† In the spirit of “release early release often”, I’ve uploaded a very early iteration of this tool.¬† Find the brand new Github repo here.

It’s quite a ways yet from being prime-time, but it’s allowed me to do quite a bit of testing.¬† Some assumptions I’ve started with:

* First, I pulled a list of all public AMIs on us-east-1.  There were about 20,000 public images available there as of mid-November.

* Then I selected the subset of AMIs that were built with a PV-GRUB kernel, and I’m importing them to an instance of Eucalyptus running the¬†kexec-loader kernel.¬† In both cases, the AKI/EKI is just a bootstrapping mechanism that then hands control over to the image’s own kernel, so we shouldn’t get caught up by kernel incompatibilities.¬† Using only the subset of kernels with PV-GRUB AKIs leaves us with about 7000, and picking one particular AKI (aki-825ea7eb) gives us about 1700.

* From there, we’re working with individual distros, and there will be idiosyncracies between the various distros out there, so it makes sense to pick one and go with it.¬† There’s a lot of Ubuntu out there on AWS, so I just grepped on AMIs with “ubuntu” in the name.¬† That’s 1067 images in my dataset.

* Now that we’ve got a reasonable set of images to examine, here’s the process that the scripts walk through.¬† For each AMI, we start an AWS instance, ssh to that instance, install euca2ools if they aren’t already installed, scp Euca credentials to the instance, and bundle the instance to the specified Euca cloud.¬† Then we fire up the resultant EMI on the Eucalyptus side and see if we can ssh in.¬† And we bail with appropriate error messages at various places along the line.

I’ve run through a few hundred images at this point.¬† Not one of them has been completely successful from start to finish.¬† About 10% so far have yielded a bundled EMI that boots and yields a Eucalyptus instance in a running state.¬† Can’t ssh into any of them yet, though.

The good news is that the failures are all quite specific and predictable, and the next steps are clear.¬† Do a better job of guessing login IDs.¬† Figure out why fstabs fail.¬† Look for rogue kernel modules.¬† Make sure we’re doing key injection properly on the Euca end.¬† Comb through the results of euca-get-console-output and look for patterns.¬† The big win is having tools that allow us to do that work in minutes, rather than in hours or days. Every step gets us closer to the goal of fully automated conversions on the fly.

Oh, and an apology: a lot of this should probably have been written using Eutester, instead of as a bunch of shell scripting.¬† The fact is, I’m a terrible hack.¬† But I’m just leading the charge temporarily; when I’ve figured out the basics, the real coders can swoop in and do things the right way.¬† In the meantime… patches welcome.

Converting AMIs to EMIs