Jan
22
2010
Queues seem to be all over the place right now. Maybe its like when I wanted a VW GTi VR6 a few years back. I kept seeing them pass me on the freeway and thought “crap, everybody is getting this hot new thing and I’m missing out!”.
I think everybody at one point looked at MySQL and tought.. “that would work fine as a queue system”. For low volume stuff, it *is* fine. But then somebody grabs your little transactional, relational, reliable queue system and plugs 5 million messages per hour through it, and somewhere, a man name Heikki cries.
So then you start to look around.. and for those of us who have meager budgets and tend to use open source, there aren’t a lot of choices. › Continue reading
3 comments | tags: amqp, C, gearman, PHP, qpid, queuing, stomp | posted in Scalability
Nov
2
2009
If you’re an engineer, you hate testing. Seriously, who likes doing what those mere mortal “users” do? We’re POWER users and we don’t need to use all those silly features on all those sites. Just look at Craigslist, clearly an engineer’s dream tool.
For web apps, testing actually isn’t *that* hard. The client program (the browser) is readily available on every platform known to man, and they generally don’t do much more than store and retrieve data in clever ways. So, its not like we have to fire up a Large Hadron Collider to observe the effects of our web app. › Continue reading
no comments | tags: development, opensource, PHP, selenium, software, testing, web | posted in Engineers, PHP, Scalability
Oct
26
2009
We had a fun time this week with TokyoTyrant. Recently it has become apparent that MemcacheDB has been all but abandoned. As fantastic as the early work was by Steve Chu, the project is in disrepair. That, coupled with the less than obvious failover for its replication combined to make us seek alternatives.

› Continue reading
no comments | tags: berkeleydb, caching, Memcache, memcachedb, PHP, process, RTFM, testing, tokyotyrant | posted in Memcache, PHP
Sep
26
2009
This article about “Duct Tape Programmers” excerpts another article with interviews from great coders. I just had to share this fantastic quote from Jamie Zawinski
“Yeah,” he says, “At the end of the day, ship the fucking thing! It’s great to rewrite your code and make it cleaner and by the third time it’ll actually be pretty. But that’s not the point—you’re not here to write code; you’re here to ship products.”
I’m a big fan of “the proper amount of abstraction”, but I think its important to remember the scope of each thing we’re working on. Rock on jwz.
no comments | tags: code, goals | posted in Engineers, Geeky
Aug
23
2009
Every time I get my logwatch report and see the 20 – 40 daily brute force attempts on it, I cringe. I’ve locked it down to a point, but ultimately I prefer convenience on some level. Limiting any one IP to 2 ssh connections every 5 minutes has annoyed me as many times as it has probably saved me. Preventing root from logging in is nice too.
Ultimately though, I wanted a way to fight back against the brute forcers.. to get a step ahead of them. From seeing the success of projects like SpamHAUS and Project HoneyPot, I know that massive group collaboration works. Of course I started thinking how I’d write it in my head. Every time… for months.
Well, once I let go of my egotistical desire to write it, I found this great project, DenyHosts, which does the same thing for the brute force scanners. I just installed it, and already it has added a few IPs to hosts.deny. Go download it, run it, and stop the annoying scanners!
1 comment | tags: collaboration, opensource, python, security, ssh | posted in Technology
Jul
25
2009
While attending OSCON 2009 w/ my faithful sidekick fluffy, we constantly kept finding instances of a common theme. The leading companies and projects seem to share one attribute that might shock you.
They all have at least *some* crappy code. At some point, all of them › Continue reading
no comments | tags: code, design patterns, freebsd, linux, MySQL, postgresql | posted in Technology
Jun
8
2009
Seth Godin’s recent post about responding to discussions about things you don’t understand has got me thinking about hiring people.
When involved with a staffing decision, I look for one trait in particular above all others. If you don’t know how to say “I don’t know”, and ask for an explanation or help, then you’re not really smart. You don’t have a good process for learning. You may have a mountain of knowledge in your head, but it is surrounded by a huge, impenetrable ego shield, and so, cannot ever be added to. Its like you took the sum of what you knew, and stuffed it into a snow globe. When people shake you up.. sure.. its pretty, but thats all there is to it.
I’d rather work with people who are open to having their entire belief system about certain subjects shattered by a better idea. That doesn’t mean that you shouldn’t stick to your guns and assert your own ideas and beliefs. It just means, when challenged, be like the Zen Buddhist Aikido master and flow with the force of the attack, and when possible, use it to your advantage.
› Continue reading
no comments | tags: aikido, Engineers, hiring, Life, sethgodin, zen | posted in Engineers, Life
Jun
3
2009
This past April I was riding in a late model, 2 door rental car with an interesting trio for sure. On my right sat Patrick Galbraith, maintainer of DBD::mysql and author of the Federated storage engine. Directly in front of me manning the steering wheel (for those of you keen on spatial description, you may have noted at this point that its most likely I was seated in the back, left seat of a car which is designed to be driven on the right side of the road. EOUF [end of useless fact]), David Axmark, co-founder of MySQL. Immediately to his right sat Brian Aker, of (most recently) Drizzle fame.
› Continue reading
1 comment | tags: benchmarks, drizzle, memcachedb, MySQL, Scalability, tokyocabinet, tokyotyrant | posted in Memcache, Scalability
Jun
2
2009
Its always been a dream of mine. I’ve posted about parallel replication on Drizzle’s mailing list before. I think when faced with the problem of a big, highly concurrent master, and scaling out reads simply with lower cost slaves, this is going to be the only way to go.
So today I was really glad to see that somebody is trying out the idea. Seppo Jaakola from “Codership”, who I’ve never heard of before today, posted a link to an article on his blog about his experimentation with parallel replication slaves. The findings are pretty interesting.
› Continue reading
no comments | tags: drizzle, MySQL, parallelism, replication, Scalability | posted in MySQL, Scalability
Dec
14
2008
So I was starting to play with Memcached for session storage, and I found a fairly big problem with justing memcached in its normal caching mode as a session store. It really just boils down to caching and storing of deterministic data being very different things that only look similar on the surface.
› Continue reading
1 comment | tags: memcached, memcachedb, PHP, Scalability, sessions | posted in Scalability, Technology