[KLUG Members] Transactional filesystems?

Justin Buist members@kalamazoolinux.org
Sat, 21 Dec 2002 14:10:33 -0500


I'm starting to realize that a distributed filesystem that allows multiple 
updaters just might actually have some use in the world.  After playing with
SQL Server's replicatoin features I'm pretty geeked on the idea now.  I'm sort
of wondering -why- this doesn't really exist at the FS layer to tell you the 
truth.  

Anybody ever seen PDM systems that need to keep local CAD drawings in their
repository in sync and in different regions?  Could be useful there -- I know
MatrixOne and Workamanger both have stuff to do this, but it's done with
seperate processes.  Much like how SQL Server/Sybase do their DB replication
really -- at least the same model.

With the plethora of journaling filesystems being released for Linux it might
not be too much work to hi-jack one of them and turn them into something like
this.  Here's the idea:

You mount an exported directory of which you also have a local mirror of.
When you update your local copy changes are migrated off to the master server
which them pushes the changes down to all the other subscribers to the
fileystem.  Everybody has the files locally for fast access time, but work
done on one system gets pushed over to another one.  Sorta like NFS with a
giant cache I suppose.  Tag each file with a unique identiefer (128 bit random
number) that's used to track changes globally.  Simple deletes/moves/additions
can just put the GUID with a pointer into the local transaction table and as
the daemon notices theese it can replicate changes out, reading the file right
from the disk as it does this.  For updates to a file use something along the
lines of rsync's delta algorithm to gather up the changes required and slap
that data right into the transation log.

My biggest problem with the idea is -- how do you control what -order- theese
changes get commited to the master repository?  I'm sure some DB research
would let me know how they do this.  A timestamp isn't reliable, as you can't
be sure that all the system's clock are sligned perfectly.  It's just not
feasible.

Is there any interest in something like this, or does something like it
already exist and I've just never heard of it?

Just thought I'd throw this out there and see what others think of it.

Thanks,

Justin Buist