FewBar.com – Make it good

One of the really cool things about using the cloud, and especially juju, is that it instantly enables things that often times take a lot of thought to even try out in traditional environments. While I was developing some little PHP apps “back in the day”, I knew eventually they’d need to go to more than one server, but testing them for that meant, well, finding and configuring multiple servers. Even with VMs, I had to go allocate one and configure it. Oops, I’m out of time, throw it on one server, pray, move to next task.

This left a very serious question in my mind.. “When the time comes, will my app actually scale?”
Continue reading

Just a quick note.. I was watching Artur Bergman’s rant about Full Stack awareness at Velocity Europe with glee, until I saw how we, the Ubuntu devs, had drawn his ire for breaking something so simple, ‘restart’.

I’ve heard this before, and I realized that its true! Restart should ignore that the service is stopped, and just start it, otherwise just restart it as I requested.

So, first off, instead of:

/etc/init.d/apache2 restart

or (for upstart controlled services)

restart mysql

get in the habit of doing:

service apache2 restart

This command will do the right thing more often than not (including clearing out the environment for sysvinit jobs, which can help solve the “why does the service only work when I restart it manually”). And as of 12.04, it will start doing the right thing with restart too.

Its not perfect, and there are still bugs in upstart, but the version of ‘sysvinit’ that I just uploaded to Precise Pangolin (the future 12.04 / LTS release of Ubuntu) at least gives scripters and sysadmins a chance at uniformity by making upstart jobs and sysvinit scripts do the same thing with restart. I don’t think we can backport this all the way to 10.04, so sorry for that. However, please, users, keep the rants coming.. we need them!

A few weeks ago I gave a live demo during Canonical CEO Jane Silber’s keynote at the Essex OpenStack Conference, which was held in Boston October 4-7 (See my previous post for details of the conference and summit). The demo was meant to showcase our new favorite cloud technology at Canonical, juju. In order to do this, we deployed hadoop on top of our private OpenStack cloud (also deployed earlier in the week via juju and Ubuntu Orchestra) and fed it a “real” workload (a big giant chunk of data to sort) in less than 5 minutes.

I’ve had a few requests to explain how it works, so, here is a step by step on how to repeat said demo.

First, you need to setup juju to be able to talk to your cloud. The simplest way to do this is to sign up for an AWS account on Amazon, and get EC2 credentials (a secret key and a key ID is needed).

If you install juju in Ubuntu 11.10, or from the daily build PPA in any other release, you’ll get a skeleton environments.yaml just by running ‘juju’.

Once this is done, edit ~/.juju/environments.yaml to add your access-key: and secret-key:. Optionally, you can set them in AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in the environment.

Now, you need the “magic” bit that turns juju status changes into commands for the “gource” source code visualization tool. Its available here:

http://bazaar.launchpad.net/~clint-fewbar/juju/gource-output/view/head:/misc/status2gource.py

(wgettable here)

You’ll also need to install the ‘gource’ visualization tool. I only tried this on Ubuntu 11.10, but it is available on other releases as well.

Make sure your desired target environment is either the only one in .juju/environments.yaml, or set to be the default with ‘default: xxxx’ at the root of the file. You need ‘juju status’ to return something meaningful (after bootstrap) for status2gource.py to work.

Now, in its own terminal, run this, note that cof_orange_hex.png is part of the official Ubuntu logo packs, but I forget where I got that. You may omit that commandline argument if you like, and a generic “person” image will be used.

python -u status2gource.py | gource --highlight-dirs \
--file-idle-time 1000000 \
--log-format custom \
--default-user-image cof_orange_hex.png \
--user-friction 0.5 \
-

This will not show anything until juju bootstrap is done and ‘juju status’ shows the machine 0 running. If you already have services deployed, it should build the tree rapidly.

So next if you haven’t done it already

juju bootstrap

Once your instance starts up, you should see a gource window pop up and the first two bits, the bootstrap node and the machine 0 node, will be added.

Continue reading

According to NASA, 70% of the earth is covered by clouds. Apparently, at least 70% of our computing needs can be covered by clouds as well. That seems to be the shared belief by the rather large crowd that gathered in Boston last week for the Essex edition of the OpenStack Design Summit and subsequent OpenStack Conference.

The amount of energy and corporate investment in OpenStack is staggering when one considers that it didn’t exist 2 years ago, and didn’t really do much more than spawn VM’s and store objects until this month with the Diablo release, which added some more capabilities, but from my point of view, mostly just refined those abilities and set the stage for the future.
Continue reading

I took a little trip down to San Diego yesterday to see what these CloudCamp events are all about. There are so many, and they’re all over, I figure its a good chance to take a look at what might be the “Common man’s” view of the cloud. I spend so much time talking to people at a really deep level about what the cloud is, why we like it, why we hate it, etc. This “un-conference” was more about bringing a lot of that information, distilled for business owners and professionals who need to learn more about “this cloud thing”.
Continue reading

Have you heard of Ensemble? Are you excited about Cloud/Service Orchestration? What? Ok you’re not alone if you are scratching your head.

Ensemble is an implementation of a new idea that has been taking shape the last couple of years. Ever since Amazon hooked up a remote API to thousands of machines to provide access to their virtual infrastructure (and called it macaroni? err.. AWS), people have been dreaming up ways to take advantage of what is basically a robotic “NOC guy”. No longer do you have to pre-rack servers or call your vendor frantically to get servers sent next-day to your colo. Right?

Naturally, the system administrators that would normally be in charge of racking servers, applied their existing tools to the job, to mixed success. Config management is really good at modelling identical hosts. But with virtual hosts instantly available, this left those thinking at a higher level wanting more. Chef in particular implemented a nice set of tools and functionality to allow this high level “service” definition with their knife tools and simple ruby API.

But how easy are Chef’s cookbooks to share and use without modification? Continue reading

Ubuntu and Drizzle — Run Drizzle on your Narwhal: OReilly MySQL Conference & Expo 2011 – OReilly Conferences, April 11 – 14, 2011, Santa Clara, CA.

I gave a talk this week in Santa Clara at the MySQL Users Conference. I think it went pretty well and I got a lot of feedback from Ubuntu users about the positives of having Drizzle available in Universe.The slides are available at the link above.

Since we’re just about to 11.04 beta2, I figured its high time I start using Ubuntu Server for my personal blog.

What? Almost a year at Canonical and my blog wasn’t on Ubuntu server? Well, for over 5 years now, a personal friend has provided me with a free Xen virtual machine to run my blog on. I migrated it off of Debian then, which was sad for me, but back then I was so focused on working I didn’t have time or resources to be picky, so I said OK.

Fast forward to now, I’ve been working on Ubuntu Server and getting ribbed by my co-workers about that “crappy CentOS xen box” they’d see me logged into.

Well thats all over now. I decided to marry all the new tech I’ve been playing with lately into one glorious blog migration. Continue reading

Last week, my application to join the ranks of Core Ubuntu Developers was accepted by the Ubuntu Developer Membership Board. The process is a bit humbling, as when you dig through the roster of this team, there aren’t a lot of names, and most of them are quite familiar to anyone who has done Ubuntu Development, as they fly by in bug reports and changelogs at an almost superhuman pace, fixing, improving, and generally making Ubuntu more awesome-r.

So, a few hours ago, I made my first direct upload into Ubuntu without a sponsor. Its just a little upgrade to squid’s upstart job to make it start more reliably. I hope its the beginning of a string of uploads that will continue for a long time, and help me earn the badge that I’ve been granted.

Many thanks to all of my sponsors up to this point, and a special nod to those who were able to add a positive comment to my developer application.

Handlersocket is what all the cool kids are using these days.. I think. Basically you get a couple of new ports on your mysql server that allow SQL-free reading and writing for doing many thousands of tiny transactions per second without the overhead of parsing SQL.

Thanks to my venerable Ubuntu sponsor, Chuck Short, handlersocket is now available in Ubuntu Natty in the universe repository. apt-get install handlersocket-mysql-5.1 handlersocket-doc, then follow the instructions in /usr/share/doc/handlersocket-doc/docs-en to enable it, and you have yourself a bonified NoSQL solution for your MySQL server.

There are also client libraries for perl (libnet-handlersocket-perl) and C/C++ (libhsclient-dev .. static only as the API is in flux). Other languages are still not packaged, but the protocol is simple, and links to early implementations are listed in the README file, which should be at /usr/share/doc/handlersocket-mysql-5.1/README.

It should be on Debian unstable as well soon…
Update April 3 2011, Handlersocket is now in Debian Unstable as well

Happy hacking!