Does anyone still do blogs?
Does anyone read this one?
Say hi in the comments if you do. 🙂
A good year for Ansible users
About a year ago, Stephen O’Grady of Redmonk published a comparison of the community metrics of the major configuration management tools. It’s a good read, and I won’t rehash its points. Go read it first.
Today I’d like to take a look at where Ansible is, a year later, using last year’s report as a benchmark. I think it’s fair to say that we’ve done pretty well for our users in 2014.
Debian Popcon
Debian’s Popularity Contest is an opt-in way for Debian users to share information about the software they’re running on their systems. Although it represents only a small sample of the Linux distro world, it’s useful because it’s one of the few places where we can really see an apples-to-apples comparison of install bases of the various tools.
First, though, a note about the original Popcon analysis: for Ansible, it was an apples-to-oranges analysis. Stephen’s report compared the Ansible control tool — the functional equivalent of a “server” — to Puppet/Chef/Salt agents. When comparing the Ansible package to the server packages of the other configuration management tools, the picture is quite different, and more in line with what we’re seeing elsewhere:
Strong growth demonstrated by Ansible in 2014. Important caveats about the above chart:
- The Puppet line shows far more variability than do the other lines, which may be an artifact of collection method.
- It appears that Chef has moved away from the distro distribution model for their server, so they are significantly under-represented. libchef-ruby was chosen because it appeared to be the best proxy. (If anyone has a better “chef server” package for Popcon comparison purposes, let me know.)
- Ansible is also under-represented to some degree, since a significant amount of Ansible’s user base installs Ansible from PyPi.
- Puppet and Salt could also be under-represented for similar reasons.
Because of Ansible’s agentless model, it’s impossible to get a realistic picture of how many systems are under active management by Ansible. However, If we look at “systems immediately addressable by each configuration management system”, then we would use openssh-server as our “agent”, and the comparison looks like this:
Obviously, not every system that has openssh installed is currently being managed by Ansible. Still, I include the graph because it’s a compelling picture of the power of not having to bootstrap an agent.
Github Metrics
Stephen tracked three metrics last year for the four projects: stars, forks, and merged PRs in the last 30 days. A key caveat for all of these graphs: because Ansible and Salt are both Github-native projects, the Github numbers for both projects are understandably higher than their older counterparts.
First, stars. Ansible has opened up its lead from last year, and now has 2-3x more stars than its counterparts.
Second, merged pull requests in last 30 days. This graph looks very similar proportionally to last year, and represents different stages of maturity and different upstream philosophies among the projects. In number of lifetime contributors, Ansible (935) and Saltstack (934) significantly outpace Chef (363) and Puppet (344).
Third, forks. Ansible had a slight lead at the end of 2013; that gap has widened in 2014.
Hacker News Jobs
In his analysis from last year, Stephen referenced a metric he called “Hacker mentions adjusted for growth.” I was unable to replicate that metric, so instead I used Ryan Williams’ excellent “Hacker News Hiring Trends” metric, which pulls from the “whoishiring” discussion threads. It’s a narrow metric, but it shows steadily growing demand for Ansible knowledge among the Hacker News crowd.
Caveat: neither “Salt” nor “Saltstack” appear to be tracked in Ryan’s dataset.
Indeed.com Job Trends
On last year’s chart from indeed.com, Ansible was practically invisible. This year’s chart shows significant growth, even though there’s a long way to go to catch up with incumbents Chef and Puppet.
Caveat: Stephen’s original search used “technology” as the term to try to weed out extraneous data. In my search I used the term “devops” instead, which shows similar results but also allows for the inclusion of Salt.
LinkedIn.com User Groups
Again, as a function of time, Ansible lags behind the incumbents Chef and Puppet, but has made significant strides in the past year, almost tripling the number of users in Linkedin.com user groups.
StackOverflow Questions
The total number of StackOverflow questions tagged with the different terms. Interestingly, although Ansible’s numbers are higher here than last year, there have still been fewer questions asked about Ansible than about any of its competitors.
Given the evidently increasing popularity of Ansible along all other metrics, it’s a curious stat — but I like to think that the gap can be explained by Ansible’s ease of use, strong documentation, and large and helpful user community. That explanation may even be reasonable. 🙂
Ansible demonstrated strong growth in 2014, and people have noticed. Thoughtworks moved us rapidly from “trial” to “adopt” in 2014 in their technology radar, with some very kind words of endorsement. Opensource.com ranked us as one of the top ten open source projects of 2014.
Which means that the bar has been raised. SDTimes called us the #1 company to watch in 2015. We’ll see about that.
As always, our success is a direct reflection of the success of our passionate community of users and contributors. Thanks to all of you. We’re looking forward to a great new year.
European friends: speak about Ansible at cfgmgmtcamp.eu or FOSDEM!
Are you an Ansible fan and want to talk about the great stuff you’re doing with Ansible, and you’re also planning to be at FOSDEM and/or cfgmgmtcamp.eu in Belgium in late January / early February?
Submit a talk! Both have extended their calls for talks until December 10th, 2014 — and both could use more Ansible users, sharing their knowledge and expertise.
It doesn’t have to be a long talk to be useful; some of the most useful talks are when people share what they’ve learned. From novice to expert, all Ansible users have valuable stories to share.
For FOSDEM, you can submit a talk for the configuration management room:
https://lists.fosdem.org/pipermail/fosdem/2014-December/002082.html
For cfgmgmtcamp.eu, you can submit a talk for the Ansible room:
https://docs.google.com/spreadsheet/viewform?formkey=dHNXcHRvTWowTUVZQVhPYndxSkxkWXc6MA
If you submit a talk and it gets accepted, you’ll get a discount to AnsibleFest London, and you’ll get some Ansible swag, and you may also get kinda famous for being an Ansible expert, which never hurts. 😉
If you have any questions, feel free to ask me at greg@ansible.com.
Trunk Club. Wow.
I am privileged to be able to visit so many of the Ansible meetups around the world — but rarely am I so privileged as I was last night to attend the Ansible meetup in Chicago.
Dean Strelau and Rick Pollak of Trunk Club invited us to host our inaugural Ansible Chicago meetup at their headquarters in downtown Chicago. This is often how it happens: a company that uses Ansible volunteers to host a meetup, and gets the benefit of being seen as a technology leader in their community; we get to show the local community how a prominent user puts Ansible to best use. Everybody wins! We’ve done similar meetups in New York, San Francisco, London, and many other cities. (And maybe yours soon! It’s easy to start your own meetup; take a look.)
Trunk Club, though, was one of the most fascinating yet. For those who aren’t familiar with the business model, check out their site for a detailed description. The short version: they talk to you about what you like, they use business intelligence to help their stylists pick out the best clothes for you, and then they send you a trunk full of clothes they think you’ll like. And then you keep what you like, send back what you don’t, and they charge you appropriately. Great model, and lots of room in there for IT automation.
Their headquarters, though, was like nothing I’d ever seen. We went up the elevator to the top floor of their building, and it was like an immaculate department store and a swanky bar and a hi-tech startup — all in the same room. Seriously. There were racks full of clothes, and customers trying things on with the help of their stylists, and then, literally in the same room, tables full of well-dressed geeks at their workstations, surely working on the software that makes Trunk Club happen. I should have taken pictures. Next time I will.
The talk was fun, too. A little bit of intro talk, a little bit of a demo of a pet use case of mine (using Ansible to set up the ELK stack to do Twitter analysis, and hi to our Elasticsearch friends!) and a lot of good pizza and beer and conversation.
Thanks to everyone who came out, and thanks again to Trunk Club for being such great hosts. Dean says the rooftop will be available for meetups when the weather improves; I can’t wait.
Look kids! Big Ben! Parliament!
London’s a great city, and it’s full of great advocates for Ansible. To everyone I met at the meetup earlier this month: thanks for coming. It was a delight to be able to meet all of you.
I wasn’t expecting to speak at this inaugural meetup, but due to a last minute cancellation (Richard, hope the family is doing better) I found myself pressed into service. I’m not really much for presentations anyway, so it was no hardship; Ali Asad Lotia and Mudassar Mian provided the technical firepower, and then when it was my turn to speak, I fell back on my tried and true strategy of asking questions and listening. This was the first opportunity a lot of people in London have had to talk with someone from Ansible in the flesh, and I’ve learned how very important it is just to meet people where they are and share stories with them.
The first thing I generally do when speaking with a roomful of users, whether I’m giving a formal presentation or not, is to survey the room for expertise. I tend to ask the same series of questions: “who has heard of X? who uses X? who uses X extensively?” What impressed me most about the crowd in London was how many of the hands stayed up through all of those questions. These were serious, serious users of Ansible. Granted, it was an Ansible meetup, so I expected more expertise than at a typical generic meetup — but it seemed as though there were hardly any novices at all.
Except for me, of course. 🙂
There were a lot of questions; I think we had more than an hour of Q+A. The audience was engaged, not only with me, but with one another; the crowd frequently came to my aid when I didn’t have an answer, and by the end of the evening, I was pretty much an afterthought as people in the crowd carried on their own conversations.
The evening also confirmed the good sense of our decision to hold our first AnsibleFest outside of North America in London. Stay tuned for details on that front soon.
Thanks again to all who attended. Looking forward to my next pint and curry.
Back to the Future
In September of 2007, I sat down at Three Cups, a coffee shop in downtown Chapel Hill, NC, with three friends from Red Hat: Michael DeHaan, Adrian Likins, and Seth Vidal. We’d all been lamenting the poor state of systems management tools, and figured that we could do better. Rather: they figured they could do better, but wanted me along to help them with the “community stuff”. I contributed exactly two things: a dogged insistence on simplicity and modularity, and the name: Func.
Here’s the blog post I wrote about it at the time:
https://gregdekspeaks.wordpress.com/2007/10/01/we-want-the-func-ow-gotta-have-the-func-ow/
The key quote from that blog post, which now looks prescient: “The takeaway: it’s not enough to write code and say ‘look, it’s open source.’ If you want to build communities around useful open source projects, you must architect them the right way to begin with.”
In February of 2012, Michael DeHaan started the Ansible project, building on the lessons he’d learned from Func and elsewhere. Here’s the very first check-in:
https://github.com/ansible/ansible/commit/f31421576b00f0b167cdbe61217c31c21a41ac02
The key quote from that check-in, which now looks prescient: “As Func, which I co-wrote, aspired to avoid using SSH and have its own daemon infrastructure, Ansible aspires to be quite different and more minimal, but still able to grow more modularly over time.”
Simplicity, modularity, community: tested in Func, perfected in Ansible.
* * *
In a little more than two years, Ansible has built one of the largest and most passionate communities of users in the open source world. Just a few examples of that passion (and really, just a few, because there are hundreds):
Pablo Roman
@pabloroman
.@TheNextWeb is moving to a brand new server infrastructure with the help of @ansible. I’ve never been so in love with a tool.
11:07 AM – 17 Jun 2014Hart Hoover
@hhoover
OH: “if a vegan does crossfit and uses Ansible, what do they talk about first?”
8:57 AM – 18 Mar 2014Florian Heigl
@FlorianHeigl1
I’ll have to admit I like watching this @ansible playbook updating servers a lot more than #soccer
Finals: “All patched”
4:13 PM – 12 Jun 2014James Conroy-Finn
@jamesconroyfinn
How have I only just found out how awesome @ansible is?!
3:38 PM – 15 Jun 2014Kozure Okami
@likwid
@ansible Thanks for making dealing with #Heartbleed extremely easy. Patched ~100 servers in no time at all. Deployed new SSL keys as well.
4:33 PM – 8 Apr 14Erik Anderson
@erikanderson
Hey @ansible – you do most everything else I need except for completing my income taxes. Would you accept a pull request? 🙂
3:43 PM – 7 Apr 14
That’s the kind of excitement you can’t buy with marketing dollars. It’s the kind of excitement you can’t manufacture with press releases and trade show gimmicks. It’s a genuine enthusiasm that people feel when they discover something insanely great that makes their lives easier.
To have a significant role in building such a thing would be a once-in-a-lifetime opportunity for any technologist.
That is why, with humility and great excitement, I’m joining Ansible today. It’s literally a dream job for me, with a dream company and a dream community.
We’re going to accomplish great things together. I can’t wait to get started.
* * *
p.s. if you’re ever in Durham, stop by the office for a visit! Here are the directions:
* Drive to downtown Durham.
* Look up at the Lucky Strike smokestack.
* Park your car somewhere close to it.
* Walk towards it until you’re close enough to touch it.
* Walk through our front door. 🙂
See you at a meetup soon.
The Perfect Time to Try Eucalyptus
Eucalyptus 4.0 is coming soon, and you can try it in beta, on a single system, right now.
To take your own AWS-compatible cloud-in-a-box for a spin, here’s what you do:
- Install CentOS 6.5 minimal on a box that supports virtualization. Give it a fixed IP address.
- Set aside a range of contiguous IP addresses to play with. 20 should be plenty.
- As root, run the following command:
bash <(curl -s https://raw.githubusercontent.com/eucalyptus/eucalyptus-cookbook/master/faststart/cloud-in-a-box.sh)
That’s it.
It’s still in beta, so if it breaks, you get to keep both pieces — but it’s pretty stable for me.
The installation script is based on Chef Solo. The goal is to provide a very simple installation experience that results in either a running cloud, or a very clear explanation of why you do not have a running cloud. Once CentOS minimal is installed, a typical install takes about 15 minutes. (NOTE: do *not* install the Desktop version; NetworkManager and PackageKit get in the way.)
If you have any problems getting your cloud-in-a-box up and running, don’t hesitate to reach out to me on Twitter (@gregdek) or on freenode (gregdek, #eucalyptus). At this stage, bad news is the best news, so if you have bugs, let’s see ’em.
I especially invite my friends who work for OpenStack vendors to see how the other half lives. 😉
The Simplest Way to Learn About Eucalyptus Code
We get lots of people who want to use Eucalyptus as a way to learn about how cloud computing works at a code level. Which is great: Freedom to Learn is one of the fundamental Free Software guarantees.
So what’s my advice to users who want to learn about Eucalyptus? It’s pretty simple.
1. Get a tiny Euca cloud running. The perfect tool for this is eucadev. If you’ve got a laptop that supports Vagrant, you’ve got a Euca cloud. It’s a small cloud, to be sure, but it’s got all of the key features required for cloud orchestration, and it’s a tool that our own developers use.
2. Find a problem to solve! The best way to learn about a codebase is to dig into it with a clear goal in mind. If you don’t yet have a clear goal, we’ve got a great list of open bugs that are tagged as “fruit” (of the low-hanging variety) to get you started.
3. Create your own local branch of the Eucalyptus source, and start hacking! An explanation of how to do this can be found in the README for Eucadev.
There’s really no substitute for getting your hands into code. Dig in. If you get stuck, swing by #eucalyptus-devel on freenode and ask for help. (After you’ve read through the docs and Googled a bit, of course.)
Being on the napkin
Alistair, I loved your napkin-based comparison of the private cloud players. It’s hilarious, and I agree with most of it. Yes, the private cloud discussion definitely shares parallels with the platform wars of the past. It’s a very useful lens for viewing our little world… even if that lens happened to be the bottom of a wine bottle. 🙂
First of all, I’ve got to say that it’s great to see Eucalyptus in the center of your napkin. Look at the size of those other players: put together the combined market cap of the OpenStack companies, VMWare, and Citrix… and then look at little old us. We must be doing something right!
What really interests me, though, is where you tried to fit Eucalyptus into your napkin analysis. I suspect that you weren’t quite sure where to put us, so you chose OS/2, found some surface similarities, and moved on to the other parts of the napkin.
In the immortal words of the great sage Jules Winnfield, “allow me to retort.”
“Integration happens when people rally around one thing.” Yes, this is exactly the right argument — and customers are rallying around AWS. Those guys are farther to the upper right in the latest Gartner Magic Quadrant than I’ve ever seen. They’re doing more public cloud business than all the other players in that market combined right now. Which is why we, and our users and customers, are rallying around one thing: AWS compatibility.
“Apps matter more than legacy protocols.” This, also, is exactly right. Developers are writing apps for the cloud — and that generally means writing them for AWS first. And developers are busy, which means that despite their good intentions to make their apps portable, portability generally comes somewhere between localization and database normalization on the priority list. Which is, of course, why legacy players like VMWare are fighting tooth-and-nail to protect their own legacy protocols. Just listen to Pat Gelsinger, CEO of VMWare: “if a workload goes to Amazon, you lose, and we have lost forever.” That’s precisely why the value we add is so useful to our users and customers: the switching costs from AWS to Eucalyptus (and back) are orders of magnitude simpler than any other option.
“IBM spent a lot of time making OS/2 work with legacy mainframe protocols and existing enterprise environments.”
Wait… did you just compare AWS to the IBM System/360?
“…and Eucalyptus is OS/2.”
<blink>
LOL.
OK, story time.
I worked at IBM way back in the day, fresh out of no college. I was the designated worldwide global support guru for the Audiovation Sound Card , for Microchannel, for OS/2. This was a combination of hardware and software that was never tested, and thus never worked. So being “worldwide global support guru” basically meant picking up the phone and saying “that combination doesn’t work, send the card back, here’s your case number, have a nice day.” And customers would always ask, irritatedly, “you made all these products — how is it possible that they don’t work together?” And my inability to lie about the answer to that question — “because none of these three components are important individually in the market, let alone collectively” — was probably a contributing factor to my getting fired from that job.
In my opinion, the chief failure of OS/2 was in its attempts to be too many things to too many people — being OK at everything, but good at nothing in particular.
That is precisely the opposite of the Eucalyptus strategy, which is to be insanely great at interoperability with the AWS API, the de facto standard for talking to the world’s dominant cloud platform.
I’ve said this before, and I’ll say it again: it’s about focus for us. We are focusing on a very particular pain point, that developers are feeling ever more acutely: the potential for AWS lock-in. Our goal is to provide an open source alternative for those users to mitigate those potential lock-in risks. Focus, focus, focus.
We see the benefits of this focused approach every day. Our roadmap is spread out before us in great detail — and that roadmap, combined with some of the best cloud engineers on the planet, gives us a feature velocity that no one else in the private cloud world can currently match. Which is why we’re on your napkin, despite being a fraction of the size of the napkin’s other inhabitants.
But in thinking about it, maybe your comparison to old school mainframe connectivity is right, and you’ve just got your timeframes wrong.
Maybe it’s 1965, and OpenStack is Multics, and Red Hat is GE, and Amazon is IBM, and AWS is the brand new System/360, ready to dominate the computing landscape for the next two decades.
Which would make Eucalyptus the open source little brother that the System/360 never had, that has no analogue in the history books, and that could have changed everything. Trouble is, I’m not sure how to fit that on your napkin.
Hobos and Vagrants and Quality
I don’t know how many people have actually met Vic Iglesias, our Quality Hobo. Here’s what he looks like in his natural habitat:
It’s really super important not to make Quality Hobo angry. Let me assure you from personal experience: no one wants that.
Here are some things that make Quality Hobo angry (and that you should, therefore, avoid):
- Stealing Quality Hobo’s cigarettes, electronic or otherwise.
- Remarking upon Quality Hobo’s resemblance to a certain celebrity.
- Wasting Quality Hobo’s time with questions about which test cases are most recent.
- Wasting Quality Hobo’s time by writing tests that don’t integrate into Eutester.
- Wasting Quality Hobo’s time with questions on how to build your QA environment.
- Wasting Quality Hobo’s time in any way at all.
Here’s what I’m saying: it was only a matter of time before our Hobo got mixed up with a Vagrant.
Vagrant is incredible, and Mitchell Hashimoto is incredible for creating it. It’s the sort of tool you try out for a particular reason, and then once you’ve used it, you find a million other reasons to use it. Last night, as I chased down a bug in Faststart, I got to try out Micro-QA, which is a mashup of Vagrant and Eutester and Jenkins and Ansible and all kinds of stuff, built on top of one of the standard Vagrant boxes for Centos 6.4.
Here’s the full instructions for getting a fully updated, complete testing environment for Eucalyptus installed on your laptop:
- Install Vagrant+Virtualbox on your laptop.
- Run “git clone” on the Micro-QA repo.
- Run “vagrant up”.
- Point your browser to http://localhost:8080.
Aaaaand you’re done.
Ease of automated testing is one of those force multipliers that doesn’t seem super-exciting, but really is amazingly super-exciting. Because here’s the thing: When the cost of testing is higher than basically zero, people don’t bother with it — or, they do a really half-assed job of it and then say “oh yeah, I totally tested that.” Which is waaaaaay worse.
In Micro-QA, we have a tool that brings the cost of automated QA to near-zero. And not just for QA folks: it’s a tool that can be used by our QA team, our engineering team, our support team, our customers, and our community, all with comparatively little knowledge required. It’s a huge win for us.
And here’s the kicker: it’s not only a QA tool; it’s also a great tool for hybrid cloud diagnostics. Set up your Euca environment; set up your AWS environment; bake in some tests that run against each; run them on a regular basis; scream when something breaks. It’s kinda sorta magic.
Anyway. If you’ve got a Euca install, go get Micro-QA running on your laptop, bring it up in your browser, pick some test cases to run, drop the contents of your eucarc file into your test case, and run it. If it breaks, ping us on Freenode (#eucalyptus-qa) and let us know what broke.
Vic announced Micro-QA less than six months ago. It was cool at the time, but now it’s way past cool. I’m really impressed by how far it’s come in such a short time. It’s like I’m living in the future.
(For the record: Vic is really a super sweetheart of a guy, and I don’t think he actually lives under a bridge at present. I think he may be living Between Two Ferns, though.)
Greg DeKoenigsberg Speaks 
