Subject: Re: [netatalk-admins] Adrian's afpd server not letting go?

From: Duncan Sinclair <[email-masked]>
Date: Wed, 11 Mar 1998 10:35:41 +0000

a sun writes:
>it's already in my current, unavailable snaphost. that snapshot has a
>few tweaks to improve solaris support. i'll probably make it available
>after i've added in my opendir/readdir/closedir version of
>case-insensitive file comparisons.

I'll look forward to it...

>actually, i would really like some data on the standard size of
>people's directories and the machines they're running on.

Is there such a thing as a standard directory size?  I have directories
with one or two files, but others with several hundred files.

My Sun is fairly low-powered.

>currently,
>the only way of properly doing case-insensitive lookups that i can
>think of is to actually read the directories, making it an O(n)
>operation. bleah.

Isn't that what the kernel has to do in its case-sensitive lookups?

Unix directory lookup can be very slow.  This is the big problem
with news servers like INN which spend more time looking up files
than it does reading them.

>that can be sped up somewhat for future directory
>reads by maintaining a hashed list or something. however, that's only
>a win if you can be sure that the files won't do something unexpected
>under you. i don't think we can make that guarantee with
>netatalk. ideas anyone?

In order to implment persistent directory ids, you'd have to do something
like this.  My idea would be to keep a single file and directory lookup
cache.  It's kept it up to date by verifying that the container directory
hasn't been modified since a particular entry was made.  It would be
built "on-demand" as I occasionally mount a 200GB (!) store for use with
netatalk, and I wouldn't want to have to pre-index that.

In CAP this is done by having a separate dir-id server process that manages
it all for the various "aufs" (CAP's afpd) processes.  If the dir-id process
dies, then the aufs can still use the dir-id store, but it has to make sure
to lock and unlock the dir-id store every time it's used.  The server process
doesn't have to worry about that - it just keeps it locked.

Cheers,

Duncan.

---

From: Mike Mansell <[email-masked]>
Date: Wed, 11 Mar 1998 09:38:08 -0500 (EST)

> netatalk, and I wouldn't want to have to pre-index that.

Being able to avoid massive pre-indexing and directory lookups would be a
life-saver for our installations.  We plan to use netatalk in some setups
to allow Mac's access to our 100GB-1500GB NFS NearLine storage server
(nearline cuz its tape + disk cache, Network <---> Disk <---> Tape Array) 
via a UNIX box.  Because our storage server is nearline, and includes tape
drives, opening directories through netatalk can generate a whole bunch of
requests into the tape array to get the .parent and .appledouble files and
such! - it can be a real nightmare (esp. if it preempts someone trying to
get to an actual file!).  If the '.' files are in the disk cache though,
its not too bad, but still theres a lot of network chatter just opening
dir's.  If a single indexing file could hold info. for many other files,
it would at least speed up the number of files netatalk has to open and
close - but I know this won't happen overnight.  For now, I think we'll
just have to add in someway to 'lock' the .parent and .appledouble files
into the disk cache (so our tape array doesn't have a heart attack!). 

Maybe my first question should have been, "has anyone had experience with
netatalk and nearline storage?".

Mike.

--
    /\     Michael G. Mansell, BSc. (SCS)  Office:  (416) 665-0767  x228
   /\ \    Engineering Support Services    clearNET:(416) 984-7839
  /\ \ \   DataVation Inc., Toronto        Fax:     (416) 665-8285
 /  \ \ \  WWW.DataVation.COM              Email:   [email-masked]