Euca School, first day of class!

I’m excited for our first #eucalyptus-classroom session on freenode on Monday (noon Eastern US time). Expect it to be the first of many.

The topic for this first meeting is Eutester, which is the new Eucalyptus harness for writing test cases.  This is A Big Deal, since Eutester is based on boto and paramiko, which are also compatible with AWS.  For users who are planning to script interactions between Eucalyptus and AWS, Eutester will be absolutely invaluable.  We’re also going to be using it to write the entire nextgen Eucalyptus testing suite (which is already immense btw.)

Be sure to take a look at Vic’s blog post about setting up the Eutester environment.

Hope to see you there.

Euca School, first day of class!

Building a culture of openness at Eucalyptus

Tomorrow, Eucalyptus Chief Talent Officer and Co-Founder, Woody Rollins, is going to be running an IRC meeting on #eucalyptus-meeting on freenode.  The topic: Eucalyptus Culture and Talent.  It’s a discussion with company and community about how we build and maintain a strong community-centric culture, and it looks like it’s going to be a regular thing.

I’m pretty excited.  If you have any questions about Eucalyptus, come on by.  Tuesday the 20th, 1pm Eastern US time, 10am Pacific.

Building a culture of openness at Eucalyptus

Coming Soon: Eucalyptus

The first release of Eucalyptus 3 is out, and it’s time to get ready for Eucalyptus 3.1.

There will be a feature listing for Eucalyptus 3.1 on our website somewhere at some point, but that feature list won’t really tell the story.  Here’s the story: Euca 3.1 is the point at which we return to developing code the open source way.

With Eucalyptus 2, we introduced our Enterprise Edition (EE).  That code was a fork of the open source edition, and included proprietary features that were not available in the open source release.  What was the nature of these proprietary features?  Basically, they were hooks to allow Eucalyptus to talk to enterprise-y things, like Big Proprietary SANs and Big Proprietary Virtual Machine Managers. These are the kinds of hooks that large enterprises, who use a combination of open source and proprietary technologies, badly need — in fact, they are hard requirements for these kinds of environments.

The ideal way to add these hooks would have been to add them as well-isolated modules — but in the fast-moving startup world, the ideal way is not always possible.  The decision was made to manage the two separate codebases side by side, with the good intention that when a feature would go into one version, it would be added into the other at the same time.

We all know the old saying about good intentions, don’t we?  🙂

Looking back, it’s probably not accurate to say that the decision was a “mistake”, per se, but it’s definitely true that the open source version suffered.  In retrospect, it was inevitable; whenever a hard choice needed to be made about the allocation of scarce resources, the choice was always, always, always to solve the customer’s problem.  So the subscription version was patched and tested, while the open source version atrophied, with the lack of commit activity leading many observers to conclude that Eucalyptus Was Dying.  Which was completely wrong, of course; the customer base grew the whole time, and the company grew with it.  The new version of the product marched along, with a shared understanding among Eucalyptoids that when the time came, the codebase would be rebuilt the right way.  The proprietary hooks would be pulled out into modules.  Open source would never again be treated as a second-class citizen.

It’s been a hard and frustrating wait for a company that views itself as open source to the bone.

So, starting with Eucalyptus 3.1, the two trees will be joined into a single tree, with a handful of modules available to subscription customers only.

The road has been long.  As the engineers pushed tirelessly towards the GA of Eucalyptus 3.0, Andy took a machete to mainline and created a parallel branch in which the proprietary bits were unceremoniously hacked out.  It wasn’t necessarily pretty, but it worked.  He also made sure that the bits would sync to this new branch on a nightly basis.  Graziano then set it up so that the new branch would be pulled into Launchpad, where we called it “devel”. Now the time has come to turn our attention to fixing that code in earnest.  We recently renamed the branch 3.1, and set it up as the mainline for the next release.  Go check it out.

What comes next?  Glad you asked!

1. We’re cleaning up the migration from MySQL to Postgres.  We decided to move to Postgres because we believe that this will make it easier to redistribute to the open source community.  The Postgres bits have been there for a while; we just turned them on when we branched, and everything seems to work so far, but there’s always a bit of housekeeping to do with such changes.

2. The entire codebase will be reexamined for proprietary bits that may have been left in the 3.1 tree.  This will require some refactoring to ensure a separation that is truly clean.  We will then make those modules available to our subscription customers only.

3. QA, QA, QA.  We will put the 3.1 release through the same rigorous QA cycle that we’ve previously reserved for the subscription version.  We’ll also be working on opening up more and more of this testing code, so that people can build their own QA tools for testing their Eucalyptus cloud (and EC2 as well).  See the eutester project for more info.

4. We will release the documentation under a Creative Commons license.  The documentation for Eucalyptus 3 is available right now, but we’ve still got some work to do on trademarks and licensing questions.  Once we do, we’ll make the DocBook source files available, and we may even kick off our first translation projects.  See the documentation project for more info.

5. We will make a lot of engineering decisions about how to open up the development process.  Issue tracking, revision control, patch management, feature process, bug triaging, release management, distro packaging — we’ll make key decisions about all of these issues in the coming weeks, and we’ll be sharing them with the Eucalyptus community every step of the way.

So when will the 3.1 release happen?  Tough to say.  Once we start putting the code through QA, we will have a good idea of how functional the code is; from what we know now, it’s certainly good enough to try out in non-production environments, with a reasonable expectation that things will mostly Just Work.  But the difference between that state, and the desired state of Open Source Goodness, will take a little while to bridge.

It’s an exciting time.  I’m proud of what we’re accomplishing in a short timeframe, and the future looks very bright.

Coming Soon: Eucalyptus