Comments for FewBar.com - Make it good http://fewbar.com Technology, life, and mischief, not in that order Sat, 23 Jan 2010 20:25:18 +0000 http://wordpress.org/?v=2.9.1 hourly 1 Comment on How do you do, that voodoo, that Queues Do? by Bruce Snyder http://fewbar.com/2010/01/queue-voodoo-that-queues-do/comment-page-1/#comment-265 Bruce Snyder Sat, 23 Jan 2010 20:25:18 +0000 http://fewbar.com/?p=136#comment-265 Ah, yes, that's a very different scenario and is known by a few names including infinite failover and continuous availability. This feature is not available in ActiveMQ largely because it's a very difficult problem to solve for many reasons. Some of the commercial MOMs have solved this and that's partially why they cost so much money. But they also come with some very costly requirements to make such a feature work, too. Thanks for the clarification. Bruce Ah, yes, that’s a very different scenario and is known by a few names including infinite failover and continuous availability. This feature is not available in ActiveMQ largely because it’s a very difficult problem to solve for many reasons. Some of the commercial MOMs have solved this and that’s partially why they cost so much money. But they also come with some very costly requirements to make such a feature work, too. Thanks for the clarification.

Bruce

]]> Comment on How do you do, that voodoo, that Queues Do? by clint http://fewbar.com/2010/01/queue-voodoo-that-queues-do/comment-page-1/#comment-264 clint Sat, 23 Jan 2010 05:52:35 +0000 http://fewbar.com/?p=136#comment-264 Bruce, I wasn't very clear in the article above. What I was referring to was the procedure required to recover with a pure Master/Slave. http://activemq.apache.org/pure-master-slave.html As I said, I'm not looking for the big enterprise solution where I have a SAN or a big NFS appliance that I trust not to die (a really nice Linux or FreeBSD server running nfs doesn't count). With pure master/slave, when the master fails the slave takes over and conceivably your stomp clients could be made to use it. However, when the master comes back online, according to the link above: "# A failed master cannot be re-introduced without shutting down the the slave broker (no automatic failback) # There is no automatic synchronization between brokers. This is a manual process." And the procedure: "This is a manual process - once a master has failed, the only sure way to ensure that the toplogy is synchronized again is manually: * shutdown the slave broker (The clients do not need to be shutdown - they will wait until the topology is re-established if they are failover clients * copy the data directory from the slave over the data directory of the master broker * re-start the master and the slave " Stomp doesn't really have the "failover" bit builtin, though it can be made to do it more or less. My issue with this is that one of the big points of using a queueing system is to be able to quickly dump messages into it and forget about them, trusting that a backend process will receive it. But if I have to wait while a big backed up queue is synchronized, even if its just for 1 minute.. thats 1 minute of browser spinning, or errors due to timeouts. Bruce,

I wasn’t very clear in the article above. What I was referring to was the procedure required to recover with a pure Master/Slave.

http://activemq.apache.org/pure-master-slave.html

As I said, I’m not looking for the big enterprise solution where I have a SAN or a big NFS appliance that I trust not to die (a really nice Linux or FreeBSD server running nfs doesn’t count).

With pure master/slave, when the master fails the slave takes over and conceivably your stomp clients could be made to use it. However, when the master comes back online, according to the link above:

“# A failed master cannot be re-introduced without shutting down the the slave broker (no automatic failback)
# There is no automatic synchronization between brokers. This is a manual process.”

And the procedure:

“This is a manual process – once a master has failed, the only sure way to ensure that the toplogy is synchronized again is manually:

* shutdown the slave broker (The clients do not need to be shutdown – they will wait until the topology is re-established if they are failover clients
* copy the data directory from the slave over the data directory of the master broker
* re-start the master and the slave
”

Stomp doesn’t really have the “failover” bit builtin, though it can be made to do it more or less.

My issue with this is that one of the big points of using a queueing system is to be able to quickly dump messages into it and forget about them, trusting that a backend process will receive it.

But if I have to wait while a big backed up queue is synchronized, even if its just for 1 minute.. thats 1 minute of browser spinning, or errors due to timeouts.

]]> Comment on How do you do, that voodoo, that Queues Do? by Bruce Snyder http://fewbar.com/2010/01/queue-voodoo-that-queues-do/comment-page-1/#comment-262 Bruce Snyder Fri, 22 Jan 2010 15:34:21 +0000 http://fewbar.com/?p=136#comment-262 Actually, you don't need to shut down ActiveMQ for the master/slave to reconnect. You just need to configure the connection between them using the failover transport and it works like a charm. This feature was added in the ActiveMQ 5.3.0 release. In fact, I just tested this last week for use in a network of brokers in ActiveMQ. The write-up of it is available on my blog: http://bit.ly/6loIOu Actually, you don’t need to shut down ActiveMQ for the master/slave to reconnect. You just need to configure the connection between them using the failover transport and it works like a charm. This feature was added in the ActiveMQ 5.3.0 release. In fact, I just tested this last week for use in a network of brokers in ActiveMQ. The write-up of it is available on my blog:

http://bit.ly/6loIOu

]]> Comment on MemcacheDB fault tolerance procedures by TokyoOops | FewBar.com – Make it good http://fewbar.com/2009/03/memcachedb-fault-tolerance-procedures/comment-page-1/#comment-86 TokyoOops | FewBar.com – Make it good Tue, 27 Oct 2009 04:28:58 +0000 http://fewbar.com/?p=46#comment-86 [...] fantastic as the early work was by Steve Chu, the project is in disrepair. That, coupled with the less than obvious failover for its replication combined to make us seek alternatives. Brian Aker had mentioned to me at one time that TokyoTyrant [...] [...] fantastic as the early work was by Steve Chu, the project is in disrepair. That, coupled with the less than obvious failover for its replication combined to make us seek alternatives. Brian Aker had mentioned to me at one time that TokyoTyrant [...]

]]> Comment on SSH brute force protection – Its almost always already written by JW http://fewbar.com/2009/08/ssh-brute-force-protection-its-almost-always-already-written/comment-page-1/#comment-53 JW Tue, 25 Aug 2009 05:49:02 +0000 http://fewbar.com/?p=111#comment-53 You know you have friends you could have asked... :-P You know you have friends you could have asked… :-P

]]> Comment on Using memcachedb and memcached to make things scale by xiafan http://fewbar.com/2008/06/using-memcachedb-and-memcached-make-things-scale/comment-page-1/#comment-49 xiafan Thu, 20 Aug 2009 05:55:30 +0000 http://fewbar.com/?p=9#comment-49 can you tell me how to configure the replication of the memcachedb?I have got some methods but they don't work; can you tell me how to configure the replication of the memcachedb?I have got some methods but they don’t work;

]]> Comment on TokyoTyrant – MemcacheDB, but without the BDB? by altex http://fewbar.com/2009/06/tokyotyrant-memcachedb-but-without-the-bdb/comment-page-1/#comment-48 altex Tue, 18 Aug 2009 15:21:01 +0000 http://fewbar.com/?p=85#comment-48 Hi. I made all the tests from your zip file (i used latest brutis, it's realy buggy, i made some patches for it), and two libraries libmemcached and libmemcache. 1) -------------------------------- libmemcache sometimes made sochet read errors when 20 clients was connected, so libmemcached is the only way to use to realy test hiload access to memcache protocol. 2) ----------------------------------- >> Of course, in nosync mode, memcachedb >> is just throwing data at the disk. It is not correct. In nosync mode it just operates data in memory throwing data to disk from time to time. 3) ----------------------------------- You realy should test not just cpu usage, but also disk usage (iowait). So tokyotyrant use disk much more than memcachedb-nosync. I've played with various tests for about a week, and i think memcachedb-nosync is the best solution for me (i need hignload solution for a big dating site). Thanks Hi.

I made all the tests from your zip file (i used latest brutis, it’s realy buggy, i made some patches for it), and two libraries libmemcached and libmemcache.

1) ——————————–
libmemcache sometimes made sochet read errors when 20 clients was connected, so libmemcached is the only way to use to realy test hiload access to memcache protocol.

2) ———————————–
>> Of course, in nosync mode, memcachedb
>> is just throwing data at the disk.

It is not correct. In nosync mode it just operates data in memory throwing data to disk from time to time.

3) ———————————–
You realy should test not just cpu usage, but also disk usage (iowait). So tokyotyrant use disk much more than memcachedb-nosync.

I’ve played with various tests for about a week, and i think memcachedb-nosync is the best solution for me (i need hignload solution for a big dating site).

Thanks

]]> Comment on Memcached and Mogile Form MemcacheMegaZord! by TokyoTyrant - MemcacheDB, but without the BDB? | FewBar.com - Make it good http://fewbar.com/2008/12/memcached-and-mogile-form-memcachemegazord/comment-page-1/#comment-38 TokyoTyrant - MemcacheDB, but without the BDB? | FewBar.com - Make it good Thu, 04 Jun 2009 06:40:35 +0000 http://fewbar.com/?p=27#comment-38 [...] View’s quaint downtown, Brian, Patrick, and I were discussing memcached stuff. I mentioned my idea, and subsequent implementation of the Mogile+Memcached method for storing data more reliably in memcached. I knew in my head why we had chosen to read from all of the replica servers, not just [...] [...] View’s quaint downtown, Brian, Patrick, and I were discussing memcached stuff. I mentioned my idea, and subsequent implementation of the Mogile+Memcached method for storing data more reliably in memcached. I knew in my head why we had chosen to read from all of the replica servers, not just [...]

]]> Comment on Can more queries equal a healthier MySQL server? by clint http://fewbar.com/2008/08/innodb-concurrency-problems-on-multi-core-boxes-possibly-a-thing-of-the-past/comment-page-1/#comment-8 clint Fri, 26 Dec 2008 15:50:26 +0000 http://fewbar.com/?p=14#comment-8 This is a well known problem Kunal. If you're using MyISAM tables, then you might try multiple key caches. If you are all InnoDB.. I think the only real solution is to run Percona's new XtraDB InnoDB plugin replacement: http://www.mysqlperformanceblog.com/2008/12/16/announcing-percona-xtradb-storage-engine-a-drop-in-replacement-for-standard-innodb/ This is a well known problem Kunal. If you’re using MyISAM tables, then you might try multiple key caches. If you are all InnoDB.. I think the only real solution is to run Percona’s new XtraDB InnoDB plugin replacement:

http://www.mysqlperformanceblog.com/2008/12/16/announcing-percona-xtradb-storage-engine-a-drop-in-replacement-for-standard-innodb/

]]> Comment on The new fad: Outsourced Parachute Packing by Recent Links Tagged With "parachute" - JabberTags http://fewbar.com/2008/08/online-storage-provider-linkup-goes-belly-up/comment-page-1/#comment-7 Recent Links Tagged With "parachute" - JabberTags Sat, 13 Dec 2008 21:04:20 +0000 http://fewbar.com/?p=12#comment-7 [...] public links >> parachute National Airborne Day Saved by mightguy13 on Wed 10-12-2008 The new fad: Outsourced Parachute Packing Saved by adamliptrot on Sat 22-11-2008 Parachute Band Goes ‘Technicolor’ Saved by [...] [...] public links >> parachute National Airborne Day Saved by mightguy13 on Wed 10-12-2008 The new fad: Outsourced Parachute Packing Saved by adamliptrot on Sat 22-11-2008 Parachute Band Goes ‘Technicolor’ Saved by [...]

]]>