The Rocket Ship to Havana – OpenStack Summit Spring 2013

I started this blog post as “The Road to Havana”, but immediately it struck me that the term “road” just doesn’t do this summit justice.

Day 1 was full of Heat for me. As a recent addition to the Heat core reviewer team, it was quite helpful and a pleasure to meet most of the other developers in person. This happened about 10 minutes before our first session together. It never ceases to impress me how easy it is to meet somebody in real life whom you’ve been corresponding with over only IRC and email. In this case, I felt face to face contact just added warmth and depth to already warm and friendly professional relationships.

Heat is on a path toward being a really great solution for managing large application deployments in OpenStack clouds. Continue reading

Juju and Nagios, sittin’ in a tree.. (Part 1)

Monitoring. Could it get any more nerdy than monitoring? Well I think we can make monitoring cool again…

 

If you’re using Juju, Nagios is about to get a lot easier to leverage into your environment. Anyone who has ever tried to automate their Nagios configuration, knows that it can be daunting. Nagios is so flexible and has so many options, its hard to get right when doing it by hand. Automating it requires even more thought. Part of this is because monitoring itself is a bit hard to genercise. There are lots of types of monitors. Nagios really focuses on two of these:

  • Service monitoring – Make a script that pretends to be a user and see if your synthetic monitor sees what you expect.
  • Resource monitoring – Look at the counters and metrics afforded a user of a normal system.

The trick is, the service monitoring wants to interrogate the real services from outside of the machine, while the resource monitoring wants to see things only visible with privileged access. Continue reading

Juju constraints unbinds your machines

This week, William “I code more than you will ever be able to” Reade announced that Juju has a new feature called ‘Constraints’.

This is really, really cool and brings juju into a new area of capability for deploying big and little sites.

To be clear, this allows you to abstract things pretty effectively.

Consider this:

juju deploy mysql --constraints mem=10G
juju deploy statusnet --constraints cpu=1

This will result in your mysql service being on an extra large instance since it has 15GB of RAM. Your statusnet instances will be m1.small’s since that will have just 1 ECU.

Even cooler than this is now if you want a mysql slave in a different availability zone:

juju deploy mysql --constraints ec2-zone=a mysql-a
juju deploy mysql --constraints ec2-zone=b mysql-b
juju add-relation mysql-a:master mysql-b:slave
juju add-relation statusnet mysql-a

Now if mysql-a goes down

juju remove-relation statusnet mysql-a
juju add-relation statusnet mysql-b

Much and more is possible, but this really does make juju even more compelling as a tool for simple, easy deployment. Edit: fixed ec2-zone to be the single character, per William’s feedback.

Configurate your juju’s

I was reading Jorge’s Stomp Box earlier today, and somebody mentioned how it would be an even better trick if it were easier to configure juju quickly.

Ask and ye shall receive. I hacked a new sub-command into the experimental ‘juju-jitsu’ wrapper. I’ll let the scrape from my terminal do the talking. You can get it with:


bzr branch lp:juju-jitsu

And try it with


juju-jitsu/wrap-juju
juju setup-environment

Continue reading

Precise is coming

Almost 2 years ago, I stepped out of my comfort zone at a “SaaS” web company and joined the Canonical Server Team to work on Ubuntu Server development full time.

I didn’t really grasp what I had walked into, joining the team right after an LTS release. The 10.04 release was a monumental effort that spanned the previous 2 years. Call me a nerd if you want, but I get excited about a Free, unified desktop and server OS built entirely in the open, out of open source components, fully supported for 5 years on the server.

Winter, and the Precise Pangolin, are coming

And now, we’re about to do it again. Precise beta1 is looking really solid, and I am immensely proud to have been a tiny part of that. Continue reading

But will it scale? – Taking Limesurvey horizontal with juju…

One of the really cool things about using the cloud, and especially juju, is that it instantly enables things that often times take a lot of thought to even try out in traditional environments. While I was developing some little PHP apps “back in the day”, I knew eventually they’d need to go to more than one server, but testing them for that meant, well, finding and configuring multiple servers. Even with VMs, I had to go allocate one and configure it. Oops, I’m out of time, throw it on one server, pray, move to next task.

This left a very serious question in my mind.. “When the time comes, will my app actually scale?”
Continue reading

Juju ODS Demo – The Home Version

A few weeks ago I gave a live demo during Canonical CEO Jane Silber’s keynote at the Essex OpenStack Conference, which was held in Boston October 4-7 (See my previous post for details of the conference and summit). The demo was meant to showcase our new favorite cloud technology at Canonical, juju. In order to do this, we deployed hadoop on top of our private OpenStack cloud (also deployed earlier in the week via juju and Ubuntu Orchestra) and fed it a “real” workload (a big giant chunk of data to sort) in less than 5 minutes.

I’ve had a few requests to explain how it works, so, here is a step by step on how to repeat said demo.

First, you need to setup juju to be able to talk to your cloud. The simplest way to do this is to sign up for an AWS account on Amazon, and get EC2 credentials (a secret key and a key ID is needed).

If you install juju in Ubuntu 11.10, or from the daily build PPA in any other release, you’ll get a skeleton environments.yaml just by running ‘juju’.

Once this is done, edit ~/.juju/environments.yaml to add your access-key: and secret-key:. Optionally, you can set them in AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in the environment.

Now, you need the “magic” bit that turns juju status changes into commands for the “gource” source code visualization tool. Its available here:

http://bazaar.launchpad.net/~clint-fewbar/juju/gource-output/view/head:/misc/status2gource.py

(wgettable here)

http://bazaar.launchpad.net/~clint-fewbar/juju/gource-output/download/head:/status2gource.py-20110908235607-pfnddi4d114nl8qd-1/status2gource.py

You’ll also need to install the ‘gource’ visualization tool. I only tried this on Ubuntu 11.10, but it is available on other releases as well.

Make sure your desired target environment is either the only one in .juju/environments.yaml, or set to be the default with ‘default: xxxx’ at the root of the file. You need ‘juju status’ to return something meaningful (after bootstrap) for status2gource.py to work.

Now, in its own terminal, run this, note that cof_orange_hex.png is part of the official Ubuntu logo packs, but I forget where I got that. You may omit that commandline argument if you like, and a generic “person” image will be used.

python -u status2gource.py | gource --highlight-dirs \
--file-idle-time 1000000 \
--log-format custom \
--default-user-image cof_orange_hex.png \
--user-friction 0.5 \
-

This will not show anything until juju bootstrap is done and ‘juju status’ shows the machine 0 running. If you already have services deployed, it should build the tree rapidly.

So next if you haven’t done it already

juju bootstrap

Once your instance starts up, you should see a gource window pop up and the first two bits, the bootstrap node and the machine 0 node, will be added.

Continue reading

OpenStack – an amoeba on a mission

According to NASA, 70% of the earth is covered by clouds. Apparently, at least 70% of our computing needs can be covered by clouds as well. That seems to be the shared belief by the rather large crowd that gathered in Boston last week for the Essex edition of the OpenStack Design Summit and subsequent OpenStack Conference.

The amount of energy and corporate investment in OpenStack is staggering when one considers that it didn’t exist 2 years ago, and didn’t really do much more than spawn VM’s and store objects until this month with the Diablo release, which added some more capabilities, but from my point of view, mostly just refined those abilities and set the stage for the future.
Continue reading