Netuality

There’s nothing quite like a good Single Point of Failure (SPOF) during a holiday dinner.

says John Farmer on his blog, and I couldn’t agree more. Start with a meteor strike scenario for a change, just imagine a giant rock crushing your measly SPOF-ridden infrastructure in one unlucky data center. Waiting for the black swan to appear learn to keep calm and react normally using the tips from a triple post about incidents, outages and systems maintenance:

Simple problems can easily become large complicated problems after a few bad decisions made in haste. Take a breath before continuing. This is especially important with a page at 3AM or if a panicky client is in your office. Tell the client you’ll handle the problem and run through your normal procedure.

[...]

Remember the prime directive – your job is to restore service as quickly as possible. You are not there to debug interesting problems with your service.

Recommended reading!

The Percona guys are pleading for a MySQL strongly optimized for a single type of storage engine:

We could save a lot of CPU cycles by having storage format same as processing format. We could tune Optimizer to handle Innodb specifics well. We could get rid of SQL level table locks and using Innodb internal data dictionary instead of Innodb files. We would use Innodb transactional log for replication (which could be extended a bit for this purpose). Finally backup can be done in truly hot way without nasty “FLUSH TABLE WITH READLOCK” and hoping nobody is touching “mysql” database any more. Single Storage Engine server would be also a lot easier to test and operate.

This also would not mean one has to give up flexibility completely, for example one can imagine having Innodb tables which do not log the changes, hence being faster for update operations.

Looks like Twitter data buffet is back in business. Some of the data is free. Enjoy it with moderation: too much data can make you slow.

Reddit‘s Steve Huffman gives a talk at Web Apps Miami 2010. Self-healing, separation of services, be stateless and cache like crazy, redundancy and yes, a little bit of Hadoop (Amazon’s Hadoop is Elastic Map Reduce). Read the full transcript on Carsonified:

We’ve actually been using Hadoop, Amazon’s Hadoop implementation to compute awards. If we need to do a complicated query like that, we store the data, we dump our database, or at the right time we store it in a way that will make those joins possible down the road. That being said; we’ve tried to avoid doing joins as much as possible, and when the data comes in we store it in the way we’re going to need it. That’s worked much better than trying to do it at run time.

No self-respecting geek can resist either of these incredible temptations:

• Correct something wrong on “the internets”
• Produce a little bit of graphviz candy (what a fantastic tool)

Read the rest of this entry »