The Perfect Time to Try Eucalyptus

Eucalyptus 4.0 is coming soon, and you can try it in beta, on a single system, right now.

To take your own AWS-compatible cloud-in-a-box for a spin, here’s what you do:

  1. Install CentOS 6.5 minimal on a box that supports virtualization. Give it a fixed IP address.
  2. Set aside a range of contiguous IP addresses to play with. 20 should be plenty.
  3. As root, run the following command:

bash <(curl -s https://raw.githubusercontent.com/eucalyptus/eucalyptus-cookbook/master/faststart/cloud-in-a-box.sh)

That’s it.

It’s still in beta, so if it breaks, you get to keep both pieces — but it’s pretty stable for me.

The installation script is based on Chef Solo. The goal is to provide a very simple installation experience that results in either a running cloud, or a very clear explanation of why you do not have a running cloud. Once CentOS minimal is installed, a typical install takes about 15 minutes. (NOTE: do *not* install the Desktop version; NetworkManager and PackageKit get in the way.)

If you have any problems getting your cloud-in-a-box up and running, don’t hesitate to reach out to me on Twitter (@gregdek) or on freenode (gregdek, #eucalyptus). At this stage, bad news is the best news, so if you have bugs, let’s see ’em.

I especially invite my friends who work for OpenStack vendors to see how the other half lives.  😉

Advertisements
The Perfect Time to Try Eucalyptus

Being on the napkin

Alistair, I loved your napkin-based comparison of the private cloud players.  It’s hilarious, and I agree with most of it. Yes, the private cloud discussion definitely shares parallels with the platform wars of the past. It’s a very useful lens for viewing our little world… even if that lens happened to be the bottom of a wine bottle. 🙂

First of all, I’ve got to say that it’s great to see Eucalyptus in the center of your napkin. Look at the size of those other players: put together the combined market cap of the OpenStack companies, VMWare, and Citrix… and then look at little old us. We must be doing something right!

What really interests me, though, is where you tried to fit Eucalyptus into your napkin analysis. I suspect that you weren’t quite sure where to put us, so you chose OS/2, found some surface similarities, and moved on to the other parts of the napkin.

In the immortal words of the great sage Jules Winnfield, “allow me to retort.”

“Integration happens when people rally around one thing.” Yes, this is exactly the right argument — and customers are rallying around AWS. Those guys are farther to the upper right in the latest Gartner Magic Quadrant than I’ve ever seen.  They’re doing more public cloud business than all the other players in that market combined right now. Which is why we, and our users and customers, are rallying around one thing: AWS compatibility.

“Apps matter more than legacy protocols.” This, also, is exactly right. Developers are writing apps for the cloud — and that generally means writing them for AWS first. And developers are busy, which means that despite their good intentions to make their apps portable, portability generally comes somewhere between localization and database normalization on the priority list. Which is, of course, why legacy players like VMWare are fighting tooth-and-nail to protect their own legacy protocols. Just listen to Pat Gelsinger, CEO of VMWare: “if a workload goes to Amazon, you lose, and we have lost forever.”  That’s precisely why the value we add is so useful to our users and customers: the switching costs from AWS to Eucalyptus (and back) are orders of magnitude simpler than any other option.

“IBM spent a lot of time making OS/2 work with legacy mainframe protocols and existing enterprise environments.”

Wait… did you just compare AWS to the IBM System/360?

“…and Eucalyptus is OS/2.”

<blink>

LOL.

OK, story time.

I worked at IBM way back in the day, fresh out of no college. I was the designated worldwide global support guru for the Audiovation Sound Card , for Microchannel, for OS/2. This was a combination of hardware and software that was never tested, and thus never worked. So being “worldwide global support guru” basically meant picking up the phone and saying “that combination doesn’t work, send the card back, here’s your case number, have a nice day.” And customers would always ask, irritatedly, “you made all these products — how is it possible that they don’t work together?” And my inability to lie about the answer to that question — “because none of these three components are important individually in the market, let alone collectively” — was probably a contributing factor to my getting fired from that job.

In my opinion, the chief failure of OS/2 was in its attempts to be too many things to too many people — being OK at everything, but good at nothing in particular.

That is precisely the opposite of the Eucalyptus strategy, which is to be insanely great at interoperability with the AWS API, the de facto standard for talking to the world’s dominant cloud platform.

I’ve said this before, and I’ll say it again: it’s about focus for us. We are focusing on a very particular pain point, that developers are feeling ever more acutely: the potential for AWS lock-in. Our goal is to provide an open source alternative for those users to mitigate those potential lock-in risks. Focus, focus, focus.

We see the benefits of this focused approach every day. Our roadmap is spread out before us in great detail — and that roadmap, combined with some of the best cloud engineers on the planet, gives us a feature velocity that no one else in the private cloud world can currently match.  Which is why we’re on your napkin, despite being a fraction of the size of the napkin’s other inhabitants.

But in thinking about it, maybe your comparison to old school mainframe connectivity is right, and you’ve just got your timeframes wrong.

Maybe it’s 1965, and OpenStack is Multics, and Red Hat is GE, and Amazon is IBM, and AWS is the brand new System/360, ready to dominate the computing landscape for the next two decades.

Which would make Eucalyptus the open source little brother that the System/360 never had, that has no analogue in the history books, and that could have changed everything. Trouble is, I’m not sure how to fit that on your napkin.

Being on the napkin

Cloud Horror Stories at OSCON

Running a private cloud can be like being the victim in a horror movie. You stand up the cloud and everything’s just fine. You get some workloads running. You’re scaling up. You’re scaling down. All is well. And then the spooky music starts, and little things start going wrong — and then suddenly the chairs are spinning in the air and you’re running for your life.

If you’re in Portland for OSCON on Monday night, come on by the Cloud Horror Stories birds-of-a-feather session. We’ll have people there with scary stories to tell, so if you’re interested in learning how things can go terrible wrong in cloud-land, come on by. Even better — if you’ve got a story of your own to share, we’d love to hear you tell it, preferably in your spookiest voice.

See you around the campfire. BOOOOOO!

(And no, we’re not actually going to start a campfire in the convention center. Definitely, almost certainly, maybe not. We can do that flashlight-under-the-chin thing, anyway.)

Cloud Horror Stories at OSCON

Who cares about AWS compatibility?

Simon Wardley is never shy to share a provocative opinion. 🙂

A summary of his latest missive: OpenStack is already doomed because of their inability, or unwillingness, to produce AWS clones. There’s nothing new in Simon’s position there, but it’s his bluntest statement of opinion yet on OpenStack’s prospects.

I’m not going to presume to agree or disagree with Simon’s prediction — but in his blog post and the ensuing conversation, I saw a few opportunities to clarify how I think about Eucalyptus and AWS fidelity.

* * * * *

First, on proving AWS fidelity.

Obviously, at Eucalyptus we think deeply about the AWS fidelity problem, and how to approach it. Simon suggests one possible model:

So could CloudStack, Eucalyptus, Open Nebula and some of the OpenStack party create a rich set of AWS compatible environments — of course. But the problem becomes you have to define one thing as the ‘reference’ model. The only way around this that I know is for the groups to create a massive set of test scripts and provide some sort of AWS compatibility service and define that as the reference model and each show compatibility to it and it to AWS. It’s possible, I’ve hinted enough times that people could try that route but there’s no takers so far.

I can’t speak for the other projects, but we find that the best tests of AWS compatibility can be found in the AWS ecosystem itself — which is one of the key advantages of working in such an ecosystem. Chasing a full API is an exhausting process, and can be discouraging, but we’ve had good success by first ensuring compatibility with the most popular open source tools in the AWS world. By moving progressively through these tools, we will cover ever-expanding sections of the API, leaving the dustiest corners of the API for last (perhaps never even to be implemented; after all, an API is ultimately only as useful as the tools that exercise it.)

The Netflix OSS toolchain is a great example of this. The team at Netflix has taken quite a bit of heat for relying so heavily on the AWS family of services, but their decision to open source all of their tools has been a boon to us. They are smart users who exercise AWS at a scale that other users can scarcely imagine, so it’s a safe bet that they’re exercising many of the most interesting parts of the AWS API. We’ve learned much, and proved much, by following their trail.

Of course, we also have our own automated test suite for AWS/Eucalyptus fidelity; we call it Eutester. It’s designed, at least in part, to run identical test cases against both Eucalyptus and AWS, and anyone who wants to test the AWS fidelity of their own IaaS can pick up that code and run with it. That codebase will continue to grow as Eucalyptus grows. Patches welcome, as they say.

* * * * *

Second, on architecting for AWS fidelity.

It seems to be assumed — among some, anyway — that AWS API compatibility is something that can simply be dropped into OpenStack at any time. The trouble is, no one will know how true that is, or isn’t, until they actually do the work. Geoff Arnold’s comments hit the nail on the head:

Load balancing is a great example. For good or ill, Amazon’s Elastic Load Balancer is a cornerstone of web-tier cloud applications architecture. If the OpenStack community was serious about AWS compatibility, the LBaaS team would have established ELB compatibility as a fundamental requirement. It didn’t. On the contrary, much of the preliminary documentation focused on all of the cool features that LBaaS would support that were not available with ELB. By Grizzly, all that we had to show for our efforts was a proof of concept based on a single instance of haproxy. Elastic provisioning was officially out of scope for the core LBaaS effort.

Software design is about choices, and with every choice you make, there’s a chance that you’ve made some other choice impractical, or even impossible. We know that there are, at the very least, syntactic differences between OpenStack and AWS; it is also quite likely that there are deeper semantic mismatches.  It may be that the OpenStack community will be able to bridge both syntactic and semantic mismatches between OpenStack and AWS with ease — but given our experiences, it doesn’t seem likely. The devil is in the details, and one must care deeply about the details in order to conquer that devil.

* * * * *

Which brings me to the last point, which is caring about AWS fidelity.

As Thierry Carrez says,

EC2 API support has always been there in OpenStack. It just never found (yet) a set of contributors that cared enough to make it really shine. Canonical promised it (with AWSOME) then let it go. More recently Cloudscaling promised it, but I’ve seen nothing so far. The next in line might just deliver.

Maybe.  There is great power in being the one who “cares enough”. And Thierry’s response here begs the question: why doesn’t the OpenStack community care more about supporting the various AWS APIs?  (Since EC2 is just the tip of the iceberg.)

That’s a question for the OpenStack community to answer.  In the meantime, I can assure you that, at Eucalyptus, we care deeply about AWS compatibility — as do our users. We work towards that goal tirelessly, every day, and I think it’s safe to say it’s because of that passion that we have taken the lead. And it’s a lead that we have every intention of extending.

Anyway. See some of you at Netflix tonite.

Who cares about AWS compatibility?

Shoes for the Cobbler’s kids

We’re big fans of the Cobbler project here at Eucalyptus. We think it’s the best tool in the open source world for bare metal provisioning.  We’ve invested in a gigantic QA environment for continual integration testing, and Cobbler is one of the linchpins of that environment. It’s the kind of tool that’s best appreciated by sysadmins who deal with *a lot* of systems.

I’m sort of attached to Cobbler personally, since I watched it grow out of Red Hat’s Emerging Technologies team several years ago. Now it’s grown past its Red Hat roots to become a truly independent project — and independent projects need support from time to time.

The Cobbler folks have set up an Indiegogo campaign to raise some funds for some much-needed infrastructure, and as proud Cobbler users, we are proud to help them out. Their goal is to raise $4000, and Eucalyptus will match every donation, dollar for dollar, until they reach their goal.

If you’ve used Cobbler and it’s helped you do your job, pitch in. The campaign is running for two more weeks; let’s help put them over the top.

Shoes for the Cobbler’s kids

Step two: put your cloud in that box!

(I’m sure you all know that step one is “cut a hole in the box”.)

We’ve been continually working to improve the install process of Eucalyptus over the past few months.   In particular, we’ve been working on a project that we call Silvereye.  Our most recent goal: make it trivial to install a fully-running Eucalyptus cloud on a single machine.

A cloud on one machine?  Why bother?  Well, lots of reasons, actually.  The biggest: the developer workstation.  If you’re hacking on Eucalyptus, it’s pretty awesome to have Eucalyptus on a single system that you tear down and rebuild in 15 minutes.

Anyway: mission accomplished.  Go to our Silvereye downloads directory and get the latest build (right now it’s silvereye_centos6_20121004.iso).  Burn it to DVD, boot your target system, and choose the “Cloud-in-a-box” option from the Centos-based installer.  Answer some simple questions.  Boom, in 15 minutes you’ve got a cloud-in-a-box!

(Note #1: a helpful README can be found in the Github repo for Silvereye: github.com/eucalyptus/silvereye.)

(Note #2: in the cloud-in-a-box config, when you log in as root for the post-install config, it’ll say “hey, do you want to install the frontend now?”  Answer yes.  It automatically installs the node controller for you.)

(Note #3: Silvereye is not supported. At all. If you use it, there are ABSOLUTELY NO GUARANTEES that it won’t burn down your house, steal your pickup truck, or throw your mother into a wood-chipper.)

Silvereye is mostly the work of sysadmin-par-excellence Bill Teachenor, based on the original Faststart installer written by David Kavanagh — but various folks are now working on it; Andy Grimm, Graziano Obertelli, and Andrew Hamilton have all been pushing the cloud-in-a-box on various distros, and Scott Moser of Canonical did some great proof-of-concept work on the UEC code. So thanks to all of them, and everyone else who’s played with it.

Give it a spin; it really is dead-easy.  We still need to round off a few corners before we can call it the official installer of record, but we’re quite close now.

Want that AWS-compatible cloud on your laptop?  Of course you do.  Now go get it.

Step two: put your cloud in that box!