Subject: Re: [netatalk-admins] Trash problems

From: Magnus Stenman <[email-masked]>
Date: Thu, 24 Sep 1998 11:47:30 +0200

[email-masked] wrote:
> 
> Memphisto <[email-masked]> just wrote about some problems with the trash
> under Netatalk and he asked if the trash can behaviour was documented
> anywhere. I know I've seen similar problems discussed in the past, but
> there seems to not have been any consensus as to the problem (or its
> solution).

This is actually the only thing that´s keeping me from switching from
mac servers to netatalk, the mac users get nervous over this behaviour,
and don´t trust the Linux box....

The Permissions/Trash Can/DID problem is perhaps only a small problem,
but probably keeps many from switching to netatalk..

> 
> I've seen exactly the same problems as Memphisto described, except with a
> Linux server. As he described, all clients logged in as the same user and
> sometimes a file thrown in the trash on Mac A appears in the trash of other
> Macs. Also, entire folders have been know to vanish from the shared disks
> and end up in a trash can. Thankfully, this doesn't happen very often,
> though of course this makes it harder to track down. So, what's happening
> here ? Is it a Netatalk or Mac problem i.e. does Netatalk (or any Appletalk
> server, for that matter) do any special handling of trash or is it that
> when a client connects to a shared folder, it looks for a folder called
> Network Trash Folder, and then looks in there for folders called Trash Can
> #n and then creates for itself a folder called Trash Can #m, where m = (max
> n) + 1. Does the confusion arise when >1 clients manage to both believe
> they should use Trash Can #X ?
> Does anyone know what's going on here :-( ?
> 

/magnus

---

From: PayPC System Mail Subscriber <[email-masked]>
Date: Tue, 29 Sep 1998 01:31:27 +1000

> This is actually the only thing that's keeping me from switching from
> and don't trust the Linux box....
> 

Actually.. the solution I've come up with seems to work the "best" for me and 
probably for those who don't have TOO many netatalk users.

Set the Network Trash Folder's permissions to read-only access group+others, 
owned by root (or the netatalk admin account).  pre-create the trash can 
directories, each owned by the various users...  rwx owner only, no access to 
others or group at all (in other words, mode 0700).

You could write a script to do this easily enough...  it sounds like it's a 
pain, but it's not that bad.  I have 10 users here and it took a few minutes 
(I used BBEdit, cutting and pasting, etc.  then pasted the commands into my 
shell, and I was done - 2 minutes).

Not a very "elegant" solution, but now everyone who mounts the volumes get 
their own (private) trash can, and they all work as they should, and my 
security models aren't compromised, AND I didn't need to patch netatalk 
(which would require RE-patching everytime it changed).

I have no idea what happens if more than one instance of the same user tries 
to mount the same share -- maybe netatalk's locking semantics do what they 
should?  I dunno.  We don't have that problem here.

Adrian: on another note, what sort of performance improvement areas would you 
think are best pursued.  I seem to get very high transfer rates (with 
inferior os's like Windows NT) with SAMBA - but with netatalk, I get no more 
than about 600-650k/sec with netatalk.  Would read-ahead/prediction code 
help?  Larger i/o buffers perhaps?  Or perhaps read/write overlapping 
technology

read case
---------
Request for 100k arrives
[ 16k chunk  comes in off disk] -- netatalk starts to send it down the 
wire....
by the time the 16k chunk's sent, the rest of the 84k has arrived from disk 
and can continue being sent w/o delay.

This way, the data gets sent to the client a few milliseconds sooner (at 
least).  [I'm assuming 10baset ethernet and SCSI harddrives of at least 
1.5Mb/s or greater and no fragmentation].

Write case is similar, except it begins writing the data after a certain 
fraction of the data has arrived... with systems like Linux which cache so 
much and intelligently (write-accumulation happens almost all of the time), 
perhaps this isn't much of a gain, since people who write to a Linux system 
always feel like they're writing to a RAM disk (assuming the system's been 
quiescent long enough for updated to flush its dirty buffers).

Anyhow - it seems like netatalk/AppleShare could be quite a bit faster.  When 
i use protocols like FTP, I get the ethernet limit all of the time to/from 
the same box (1,200,000 bytes/sec, measured against wall-clock).

(By the way, I do realise many of the performance issues are in the 
Appleshare client, but from the settings I've seen in the 3.7.x and 3.8 
clients (write-behind/read-ahead, large buffers, separate cache buffers, 
etc), it seems like it really should fly.

=Rob=

---

From: David Winkel <[email-masked]>
Date: Mon, 28 Sep 1998 13:39:08 -0400 (EDT)

> help?  Larger i/o buffers perhaps?  Or perhaps read/write overlapping 

I know someone who's been doing some fairly serious performance testing,
mostly to see why web servers do better file throughput on NT than they do
on unix.  The reason is that NT allows for one system call to transfer an
entire file out of a socket.  Which means that the entire transfer is done
with only one system call, and everything else is handled in the kernel. 
Unix handles all files in (usually) 8K blocks, which means that for every
8K, there's a system call.  They're not that expensive, but for large
files, they add up significantly.  But, that's most of the reason that NT
does better than unix when handling file throughput...  For those who
care.  :)
You could probably aid this by increasing the buffer that netatalk uses to
read and write files, but that takes additional memory for the buffers,
too.  I haven't looked at netatalk specifically, but if you're right, and
16k is the size of the buffer, then increasing that may help, but won't
fix the problem.

> etc), it seems like it really should fly.

 ~~~~
           [email-masked]               UofM/ITD Web Services
                      University of Michigan Webmaster

      At some point you have to accept that 99% uptime means 1% downtime.

---

From: a sun <[email-masked]>
Date: Mon, 28 Sep 1998 11:31:06 -0700 (PDT)

> Adrian: on another note, what sort of performance improvement
   > areas would you think are best pursued.  I seem to get very high
   > transfer rates (with inferior os's like Windows NT) with SAMBA -
   > but with netatalk, I get no more than about 600-650k/sec with
   > netatalk.  Would read-ahead/prediction code help?  Larger i/o
   > buffers perhaps?  Or perhaps read/write overlapping technology

well, i can usually saturate 10BaseT in at least one direction in my
current setup. having g3's really helps here. unfortunately, i only
get about 4.5MB/s max throughput w/ 100BaseT in my setup and have
annoying slowdowns in a linux -> g3 running os 8.1 direction. i
actually suspect the current mac os of being the culprit, but i need
to load linux on the g3 to be sure.

currently, there's an inherent tradeoff between interactive response
and transfer speed. the appleshare client actually doesn't request the
entire file. it asks for it in fairly small pieces and intersperses it
every so often with FPEnumerate calls. as a result, you won't get
maximal throughput do to all the chatter. on my part, i would like to
separate the "data conversion on the fly path" more from the "no
conversion necessary" path. i've already dealt with parts of it, but
it still needs some work. that would allow a sendfile() case in the
future if apple's appleshare client were ever to handle that kind of
thing. as things stand now, though, it's a bit premature.

i would also like to make the FPEnumerate call faster as well as it's
probably one of the slowest calls that exist in afpd. given how the
appleshare client works, improving it should actually improve apparent
file transfer speed. 

beyond that, i would look at the hardware/os setup a bit. afpd depends
upon the underlying os' fs structure for read-ahead/write-behind
buffering. in addition, i've noticed that non-dma capable ide drives
screw things up a lot as they fight with the ethernet card. when i do
performance testing of just file transfer performance, i usually use
something like helios lantest and compare it against a direct ftp
transfer. in most cases, the lantest throughput and the ftp throughput
match up pretty well.

-a

---

From: PayPC System Mail Subscriber <[email-masked]>
Date: Tue, 29 Sep 1998 05:22:13 +1000

David Winkel said in Re: [netatalk-admins] Trash problems at 29/Sep/1998 
(Tue) 03:39:08.

> does better than unix when handling file throughput...  For those who

Actually even without "dirty API technology" like NT's, my unix box normally 
outblasts the NT ones with less CPU usage.
Yes, serving up static content is more efficiently handled this way I 
suppose, but it's not what I'm referring to.  Alas, I noticed the 2.1.x 
kernel notes mention a new syscall basically doing this very same thing.  So 
you can get it now (2.1.x) or wait a little while longer for 2.2 to be 
released, which is expected before year's end.  Purist kernel types like 
myself really despise this kind of API boundary crossing.  "File copies" are 
really a "higher layer" function than kernel-level.  Basically, what would 
prolly do 99% of the performance win without being API child-molestors would 
be a strategy to avoid buffer copying as the data percolated down into the 
net i/o subsystem.

But we digress from my point... unix is obviously fast enough to saturate 
ethernet sending as well as receiving from my PowerMac 8500 when using FTP... 
why not with netatalk?  (Also, I have no NT file servers, so the performance 
is keyed to the UNIX machine even from my Windows clients)

> 16k is the size of the buffer, then increasing that may help, but won't

In today's world of cheap RAM, buffer space is cheap.  However, I've found 
that 8K isn't a terrible choice for most purposes.  I think wu-imapd uses 16k 
buffers, and when I changed to 64k, I observed no difference in performance.  
Turned out I saw the biggest deltas by changing CLIENTs.  Oddly enough, 
outlook express on both Macintosh and Windows is one of the fastest when 
connecting to a unix wu-imap server.  Slogs down hundreds of e-mail headers 
in mere seconds.  Fun to watch, and it really shows imap4's potential, and 
why it's so cool compared to pop3.

=Rob=