PUBLIC MARKS with tags cluster & rsync

Many people use SSH to log in to remote machines, copy files around, and perform general system administration. If you want to increase your productivity with SSH, you can try a tool that lets you run commands on more than one remote machine at the same time. Parallel ssh, Cluster SSH, and ClusterIt let you specify commands in a single terminal window and send them to a collection of remote machines where they can be executed. Why you would need a utility like this when, using openSSH, you can create a file containing your commands and use a bash for loop to run it on a list of remote hosts, one at a time? One advantage of a parallel SSH utility is that commands can be run on several hosts at the same time. For a short-running task this might not matter much, but if a task needs an hour to complete and you need to run it on 20 hosts, parallel execution beats serial by a mile. Also, if you want to interactively edit the same file on multiple machines, it might be quicker to use a parallel SSH utility and edit the file on all nodes with vi rather than concoct a script to do the same edit. Many of these parallel SSH tools include support for copying to many hosts at once (a parallel version of scp) or using rsync on a collection of hosts at once. Because the parallel SSH implementations know about all the hosts in a group, some of them also offer the ability to execute a command "on one host" and will work out which host to pick using load balancing. Finally, some parallel SSH projects let you use barriers so that you can execute a collection of commands and explicitly have each node in the group wait until all the nodes have completed a stage before moving on to the next stage of processing.

A l"heure où les serveurs d"entreprises doivent stocker un volume croissant de données et assurer une haute disponibilité, il est nécessaire d"imaginer des systèmes de mirroring (miroir) autres que simplement sur des disques durs. La redondance de disques durs (RAID-1, RAID-5) permet déjà d"avoir une bonne résistance aux pannes d"un disque (ou plusieurs si on est en RAID-5). Cependant, si c"est la machine qui tombe (le processeur, l"alimentation, le contrôleur de disque,...), on n"a aucun moyen pour relancer le tout rapidement sans perte de données. Ayant été confronté à cette problématique, j"ai envisagé une solution de RAID-1 au travers du réseau. C"est-à-dire que sur deux serveurs, on a deux partitions (une par serveur) qui sont à tout moment une copie exacte l"une de l"autre. C"est un mirroring (miroir) de partitions à travers une interface réseau.

This is only ONE way to keeps nodes in synchronization for a high availability cluster. It suits our needs and may be a good starting block for others so I'm sharing it. Your mileage may vary. We used this on a web server to update ~1GB worth of documents in sync every 10 minutes. Depending on how quickly your files change, you may be able to update more data, more frequently. I would not recommend using this without a private fast ethernet channel, i.e. an extra NIC in each node connected via a crossover cable.