Greg DeKoenigsberg Speaks

Coming Soon: Eucalyptus 3.next

Posted in Uncategorized by Greg DeKoenigsberg on March 5, 2012

The first release of Eucalyptus 3 is out, and it’s time to get ready for Eucalyptus 3.1.

There will be a feature listing for Eucalyptus 3.1 on our website somewhere at some point, but that feature list won’t really tell the story.  Here’s the story: Euca 3.1 is the point at which we return to developing code the open source way.

With Eucalyptus 2, we introduced our Enterprise Edition (EE).  That code was a fork of the open source edition, and included proprietary features that were not available in the open source release.  What was the nature of these proprietary features?  Basically, they were hooks to allow Eucalyptus to talk to enterprise-y things, like Big Proprietary SANs and Big Proprietary Virtual Machine Managers. These are the kinds of hooks that large enterprises, who use a combination of open source and proprietary technologies, badly need — in fact, they are hard requirements for these kinds of environments.

The ideal way to add these hooks would have been to add them as well-isolated modules — but in the fast-moving startup world, the ideal way is not always possible.  The decision was made to manage the two separate codebases side by side, with the good intention that when a feature would go into one version, it would be added into the other at the same time.

We all know the old saying about good intentions, don’t we?  :)

Looking back, it’s probably not accurate to say that the decision was a “mistake”, per se, but it’s definitely true that the open source version suffered.  In retrospect, it was inevitable; whenever a hard choice needed to be made about the allocation of scarce resources, the choice was always, always, always to solve the customer’s problem.  So the subscription version was patched and tested, while the open source version atrophied, with the lack of commit activity leading many observers to conclude that Eucalyptus Was Dying.  Which was completely wrong, of course; the customer base grew the whole time, and the company grew with it.  The new version of the product marched along, with a shared understanding among Eucalyptoids that when the time came, the codebase would be rebuilt the right way.  The proprietary hooks would be pulled out into modules.  Open source would never again be treated as a second-class citizen.

It’s been a hard and frustrating wait for a company that views itself as open source to the bone.

So, starting with Eucalyptus 3.1, the two trees will be joined into a single tree, with a handful of modules available to subscription customers only.

The road has been long.  As the engineers pushed tirelessly towards the GA of Eucalyptus 3.0, Andy took a machete to mainline and created a parallel branch in which the proprietary bits were unceremoniously hacked out.  It wasn’t necessarily pretty, but it worked.  He also made sure that the bits would sync to this new branch on a nightly basis.  Graziano then set it up so that the new branch would be pulled into Launchpad, where we called it “devel”. Now the time has come to turn our attention to fixing that code in earnest.  We recently renamed the branch 3.1, and set it up as the mainline for the next release.  Go check it out.

What comes next?  Glad you asked!

1. We’re cleaning up the migration from MySQL to Postgres.  We decided to move to Postgres because we believe that this will make it easier to redistribute to the open source community.  The Postgres bits have been there for a while; we just turned them on when we branched, and everything seems to work so far, but there’s always a bit of housekeeping to do with such changes.

2. The entire codebase will be reexamined for proprietary bits that may have been left in the 3.1 tree.  This will require some refactoring to ensure a separation that is truly clean.  We will then make those modules available to our subscription customers only.

3. QA, QA, QA.  We will put the 3.1 release through the same rigorous QA cycle that we’ve previously reserved for the subscription version.  We’ll also be working on opening up more and more of this testing code, so that people can build their own QA tools for testing their Eucalyptus cloud (and EC2 as well).  See the eutester project for more info.

4. We will release the documentation under a Creative Commons license.  The documentation for Eucalyptus 3 is available right now, but we’ve still got some work to do on trademarks and licensing questions.  Once we do, we’ll make the DocBook source files available, and we may even kick off our first translation projects.  See the documentation project for more info.

5. We will make a lot of engineering decisions about how to open up the development process.  Issue tracking, revision control, patch management, feature process, bug triaging, release management, distro packaging — we’ll make key decisions about all of these issues in the coming weeks, and we’ll be sharing them with the Eucalyptus community every step of the way.

So when will the 3.1 release happen?  Tough to say.  Once we start putting the code through QA, we will have a good idea of how functional the code is; from what we know now, it’s certainly good enough to try out in non-production environments, with a reasonable expectation that things will mostly Just Work.  But the difference between that state, and the desired state of Open Source Goodness, will take a little while to bridge.

It’s an exciting time.  I’m proud of what we’re accomplishing in a short timeframe, and the future looks very bright.

About these ads
Tagged with: , ,

11 Responses

Subscribe to comments with RSS.

  1. hspencer77 said, on March 5, 2012 at 12:59 pm

    Reblogged this on More Mind Spew-age from Harold Spencer Jr. and commented:
    Nice! Eucalyptus 3.1…Can’t Wait!!

  2. Jef Spaleta said, on March 5, 2012 at 2:37 pm

    Greg,

    A suggestion. Since you legitamately can’t establish public goalpost deadlines for the 3.1 release and its engineering decision making.. can you instead provide some sort of visualization of incramental progress on the goals? I think you and I both know (from shared project experience) that the process for the engineering decisions that have to be made as part of project management can look very much like a blackhole of unprogress to externals outside the corporate fenceline. Good luck.

    -jef

    • Greg DeKoenigsberg said, on March 5, 2012 at 8:45 pm

      Jef, I don’t think we’re that far short of some public goalpost deadlines. I think we’ll see a lot of progress in that regard in the next month or so — public roadmap, bugs and features in public issue trackers, etc. If not, then yes, we may want to come up with a better way of reporting our progress.

  3. Roland Fouad said, on March 5, 2012 at 4:53 pm

    OpenStack!!! rah troll sorry

  4. Tim Wessels said, on March 6, 2012 at 9:02 am

    Eucalyptus has been criticized in the past for not really being open source and failing to develop an open source ecosystem. So I think it was a good thing to admit that the previous choice was a mistake and make the necessary correction. I also believe that the current development environment requires Eucalyptus to go back to its open source roots. This move can only help Eucalyptus survive as a viable “cloudstack” in the future. Today there are several dozen or more “cloudstacks” out there and not all of them are destined for longer term success. We now have OpenStack, CloudStack, Open Nebula and Eucalyptus as open source projects. My other concern about Eucalyptus is the burn rate of their venture funding given the size of the organization. OpenStack is propelling itself forward based on the number of people contributing code, tons of raw enthusiasm and over 140 “endorsements” by commercial entities. CloudStack has Citrix standing behind it and just released the next version of CloudStack with their “brand” on it.

    • Greg DeKoenigsberg said, on March 13, 2012 at 10:25 am

      Tim, we have customers and revenue and strong partnerships and a strong product that fills a particular need (AWS compatibility) exceptionally well. I can’t speak publicly about questions of burn rate and so forth, but I will say that I’m quite comfortable with the path we’re on.

      Still, the criticisms about not building an ecosystem have been warranted, which is why we are purposeful about fixing them. Fortunately, we’re early in the cloud game, with time to fix the issues we face.

  5. Steven Lee said, on March 27, 2012 at 10:39 am

    So if I read your post correctly, open source version of Eucalyptus 3.1 will be available the same day as the enterprise version because they are essentially the same code. Open source version just doesn’t have the enterprise modules, right?

    • Greg DeKoenigsberg said, on March 27, 2012 at 11:01 am

      Steven, that’s mostly right. To be *crystal* clear:

      The Enterprise version of 3.0 is already out. We’re in the process of disentangling enterprise-only bits.

      When the Enterprise version of 3.1 comes out, yes, it will release at the same time as our open source version of 3.1, because at that point, they will be essentially the same code, minus the enterprise modules.

  6. [...] This is all changing with the upcoming 3.1 release, as our good friend and colleague Greg outlines here.  This is a really big deal for Eucalyptus and also very exciting, we’re going back to our [...]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 35 other followers

%d bloggers like this: