Proof: Netflix OSS on Eucalyptus

The Netflix OSS team sure knows how to throw a good party.  We’ve been to their meetup twice now, most recently at their event last Wednesday, at which they unveiled the Netflix OSS Prize.  At this rate, given the amount of interest they’re generating, they’ll need to rent out hotel space for their next event.

What draws the crowd? It’s a good question. The simplest answer, I guess, is that they’re a cool company doing cool stuff — and people want to be like them.

At Eucalyptus, we certainly enjoy working with Netflix, and we enjoy hanging out with them generally, because they’re cool.  (And their headquarters are awesome — “spa-like”, one might say.)  But we also have very specific interests in following the Netflix approach:

1. Netflix understands cloud.  Surely this is obvious; Netflix is one of the most advanced user of cloud services on the planet. They were the first to understand the true value of the AWS model, and in taking full advantage of it, they’ve developed a reputation for being industry leaders who work at true cloud scale.  Other leaders now seek to learn from them and emulate their practices.

What does that mean, though — “working at true cloud scale”? In a nutshell: Netflix has embraced the reality that sometimes services just go away.  This is the single biggest shift that one must make when moving into the world of cloud: not just accepting, but embracing the idea that sometimes your systems just go away.  That’s the point of cloud, and if you don’t build your systems with that mindset from Day One, you’re Doing It Wrong. Netflix has demonstrated an understanding here that few organizations can match.  They’ve moved from Chaos Monkey, which randomly knocks over instances, to Chaos Gorilla, which randomly knocks over entire availability zones — and they’ve made these tools, and others like them, available for those who dare to follow their lead.

2. Netflix understands open source. Not just as users, but as producers.  They are willing to share some pretty amazing software, because they understand the difference between software that provides differentiating value for their business (the engine) and software that provides non-differentiating value (the plumbing).  The more they can share the cost of maintaining the plumbing, the more resources they can commit to the engine.  This is a highly strategic choice, which makes them all the more committed to it — they have 26 projects in their Github repo and counting, with no sign of slowing down anytime soon.

3. Netflix OSS requires real AWS API fidelity, and Eucalyptus provides it.  Netflix is completely committed to the AWS model, and all of their code currently assumes that you’re using AWS.  If you want to figure out if your own AWS compliant IaaS “just works”, the Netflix OSS tools represent the best possible tests of AWS API fidelity. The “Asg” in Asgard, for instance, stands for Autoscaling Groups — so as we get our own autoscaling functionality up to speed for the Eucalyptus 3.3 release, it makes perfect sense to use Asgard as the benchmark to test against.  Which is precisely why we demonstrated at the Netflix OSS event: Chaos Monkey for knocking down instances, Asgard for Autoscaling to replace the destroyed instances, and Edda for auditing the whole process — and all working on Eucalyptus precisely as it would work against AWS.

Here’s the point, and it’s a simple one: it’s easy and cool to claim that your private cloud is “AWS API compatible”.  But it’s another thing entirely to prove it.  At Eucalyptus, we stake our entire reputation on proving it, with every single release. At the Netflix meetup, people walked up to our demo station, and they could *see* Netflix OSS on a private cloud. Edda, Asgard, and Chaos Monkey, all running in the cloud that was sitting right there on the table next to them.

@joeemison @adrianco The AWS API is open enough to be emulated, as demonstrated by @eucalyptus running Edda, Simian Army, and Asgard.

— Joe Sondow (@joesondow) March 15, 2013

Don’t trust hype.  Trust proof.

Next Euca hackfest: OpenShift integration

We’re going to be starting up our weekly IRC hackfests on #eucalyptus-devel next week.

There’s a lot of cool integration work of various kinds that we want to do with Eucalyptus, and it’s the kind of work that’s best done with many hands.  A lot of it is just “getting X to run on Eucalyptus,” and we want to fill in as many possible values of X as we can.  Thus, hackfests.

The goal is to have at least a couple of hours of non-interrupted hacking time every week, and we’re going to aim for end of week, either Thursday or Friday afternoon.  Figuring out timing is always an issue, so far now we’re just going to pick a time and see how it works out.  The first hackfest will be noon-2pm Pacific time on Thursday, August 2nd on #eucalyptus-devel.  This will overlap somewhat with the standing recipes meeting, but since we’ll likely be working on recipes much of the time, I think we can swing it.  We expect to have a few core people present at these hackfests every single week, but of course, the more the merrier.  It’s also perfectly fine for people to drop in and drop out as they may be available.

Our first target will be OpenShift Origin integration — so we’ll be all over the #openshift channel on freenode, and dragging as many of you as we can to #eucalyptus-devel in the process.  🙂

(update: what we’re working on is actually integration of “OpenShift Origin” — the bits that are used to make the OpenShift service, which is trademarked by Red Hat, etc., etc.  Must respect the brand. Post updated accordingly.)

Today is the day.

Eucalyptus 3.1 is open for business.

No more artificial separation between Enterprise and Community.  No more frenzied checkins to the “enterprise edition” while the separate-but-equal “community version” atrophies.  No more working on new features behind closed doors for months on end.  No more wondering about what’s on the roadmap.  No more going weeks without any publicly visible check-ins.  No more.

Today is the day that we release Eucalyptus 3.1, and reassert our position as the world’s leading open source cloud software company.  With the emphasis on open source.  We’ve been working to get to this day for months, and now, the day has come.

For those who want to get started with the new bits immediately, the Faststart installer can be found here.  With two virt-capable laptops installed with Centos 6.2 minimal, you can have a private cloud running in 15 minutes if you follow the directions — and a few hours if you don’t.  🙂

Package repositories for the various distributions can be found here.

Source code is on Github. Here’s a look at all the recent commit activity.

Anyone who has questions can ask them here or here.

A list of all currently known bugs in 3.1 can be found here.

The list of features we’re currently scoping for 3.2 can be found here.

We have lots of other projects moving forward on Github as well.  Projects like Eutester for automated testing of Eucalyptus (and Amazon) instances, Recipes for automated deployments of Eucalytpus (and Amazon) instances, our nextgen installer Silvereye, and many others.

All of these projects are open to community participation and transparently managed.  We hold weekly meetings on IRC.  You can find the weekly meeting schedule here.  Minutes for all meetings for the past six months can be found here.

We’re also hiring.

“Build together. Run together. Manage together.”  That’s been the mantra for this release, and it speaks directly to the culture of our company.   If I learned anything at Red Hat, it’s that company culture matters.  It literally makes or breaks the company.  Especially in open source: either you’re an open source company, or you’re not.  We are deeply committed to the open source model, because we believe that it creates the best software, and we’re going to prove it.

The most exciting thing about today’s release, to me, is that we’re only getting started.  It’s been a long climb to get to this plateau. We’ve still got a lot of mountain yet to climb, though, and we’re looking forward to the challenge — but that can wait for another day. Maybe two.  Today is about appreciating where we’ve been, and enjoying the view.

Well done, Eucalyptians.  Well done.

Eucalyptus 3.1 Beta is now on Github!

It’s taken a while, but the move is complete.  The source code for Eucalyptus 3.1 Beta is open and publicly available in Github.  It’s actually been there for a while now, but we’ve done enough housekeeping and we’re ready to open the doors.

Build instructions can be found in the INSTALL file, but they are still in flux; comments and patches are welcome.  Don’t hesitate to join us on #eucalyptus on freenode or on our community mailing list if you have questions.

Packages for the beta will be available for various distros in the coming days.  Special props go to Debian Partner company Credativ for their impressive work on the Google Web Toolkit libraries.

We’re also working on our new bug tracker; we’re in private beta to work through various auth and workflow kinks.  If you’re interested, ask for access on IRC or the mailing list, and we will set you up.  After this beta period is concluded, we will open the new bugtracker to anyone and everyone — but we’re happy to give early access to anyone who asks.

This is another critical step in our evolution as an open source company.  But we’re not done yet.  Stay tuned.

Building a culture of openness at Eucalyptus

Tomorrow, Eucalyptus Chief Talent Officer and Co-Founder, Woody Rollins, is going to be running an IRC meeting on #eucalyptus-meeting on freenode.  The topic: Eucalyptus Culture and Talent.  It’s a discussion with company and community about how we build and maintain a strong community-centric culture, and it looks like it’s going to be a regular thing.

I’m pretty excited.  If you have any questions about Eucalyptus, come on by.  Tuesday the 20th, 1pm Eastern US time, 10am Pacific.