Subject: RE: Disappearing files.....[long, new case/infos]

From: Michalowski Thierry <[email-masked]>
Date: Tue, 5 Dec 2000 12:34:49 +0100

Could all of this be caused by OS9 and _not_ netatalk alone?
Let me explain:

We use Helios EtherShare at work on our main server, and it happened that
some of our users suddenly found an entire volume into their Trash.
However, these volumes are read-only by normal users: the files could not
disappear on the server because it happened to be impossible to empty the
Trash.

Only one Mac on the net believed it had trashed everything, any other Mac
normally saw all files in place (not in the Trash).
How odd, sometimes one or the other sees the files in _both_ places : right
in the volume _and_ in the Trash.

All of this is easily (hum!) solved by simply restarting the Finder on the
faulty Mac. After that, everyone sees the files at their correct location.

Sorry for the long description...Why all of this?

Simply to point out that even if netatalk is not there, the same thing may
happen with OS9.
Ok, not exactly since we never lost any file, but it may be because of
different permissions set on the U*X side.

I would be really happy to hear anything from anyone who could have
experienced anything like that.

As a side note, we used to have "AutoPurge" runnning on our Macs to work
around the "Temporary Items" non-existant purge in OS9's Finder. Since we
disabled this utility, we never met the problem again.

All of this could probably have something to do with persistent DIDs as far
as I can imagine, but I'm far from being able to explain everything alone.

Thierry Michalowski

-----Original Message-----
From: Russell Kerrison [mailto:[email-masked]]
Sent: Tuesday, December 05, 2000 1:06 AM
To: [email-masked]
Subject: Re: Disappearing files.....

on 5/12/00 3:58 AM, Netatalk at [email-masked] wrote:

> This might account for some of the file loss problems some people seem to
> experience. The window between 1 and 2 and between 2 and 3 could be
> arbitrarily large. The Mac seems to store DIDs for a long time, even
> when their resolving fails.
> 
> Solution: never reuse DIDs

I wrote a week or two ago about a fault where a great slab of folders had
vanished from the server, and have since discovered that the odd OS9 users
periodically finds entire root folders in their Trash. Thankfully they
notice before they empty their trash!

I could imagine that this might be caused by an incorrectly recycled DID?

Russell.

---

From: "lists-mail" <[email-masked]>
Date: Tue, 5 Dec 2000 18:03:06 -0000

I agree completely, I have been using Netatalk for three years daily. While 
not a great stretch compared to others I have never experienced these problems 
prior to OS 9. Interestingly enough we also experience random lost and corrupt 
files on our one NT server as well, also after upgrading to OS 9.

While the problems are not identical, I think the fact OS 8.1 and OS 8.6 in 
conjuction with Netatalk and NT not showing any problems for me, says that the 
culprit likely _is_ an OS 9 issue.

DAve.

Michalowski Thierry <[email-masked]> said:

> Let me explai

---

From: "Robert B. Bell" <[email-masked]>
Date: Tue, 05 Dec 2000 10:35:49 -0800

I have seen the disappearing files 
(whole home directories placed in the trash can by the wrong user)
under OS 7.

I am running netatalk-1.4b2+asum2.1.4

  Robert.

---

From: "Lawrence Farr" <[email-masked]>
Date: Tue, 5 Dec 2000 18:38:27 -0000

I get this on 8.6, just to spoil that! (No 9's in the
building even!)

> From: lists-mail [mailto:[email-masked]]
> Sent: 05 December 2000 18:03
> To: Michalowski Thierry; 'Russell Kerrison'; [email-masked]
> Subject: RE: Disappearing files.....[long, new case/infos]
> 
>

---

From: BWS - Offwhite <[email-masked]>
Date: Tue, 5 Dec 2000 12:46:21 -0600 (CST)

I have never experienced lost files with Netatalk, except when they are
simply hidden due to really long filenames.  The mac can only see
files with 32 or less characters in the name.  I simply shorten the names
and I can see them.

Also, of the people who have had these problems, what OS is the Netatalk
server running on?  What filesystem is on that server under Netatalk?

I have using Netatalk (netatalk-1.4b2+asun-2.1.3) for over 2 years on top
of FreeBSD with the default ufs filesystem from FreeBSD 3.0 to 4.2.  I
have not had experienced any of these problems.

Perhaps there is some bug with a specific platforms which need some
attention.  If we can isolate the potentially flawed situation the
developers could get it fixed.

As for OS9 being a problem, I am not so sure of that.  Sure it has really
screwed up networking but you would think they would have AppleShare/IP
working properly.

But then again, I keep my Macs and all the Macs in the office updated with
the Software Update program which comes with OS9.  I have seen updates for
the ethernet and various network utilities being updated since the release
of OS9.

Try running the software update on all Macs on your network and see if
this problem with lost files continues.  So far I have not read anything
about this from Apple, but I have much insentive to look as my Macs
seem to working quite well, with the required daily reboot of course.

Brennan Stehling - web developer and sys admin
projects: www.greasydaemon.com | www.onmilwaukee.com | www.sncalumni.com

This signature is just here to take up space.

---

From: Tim Carlson <[email-masked]>
Date: Tue, 5 Dec 2000 12:06:54 -0700 (MST)

> server running on?  What filesystem is on that server under Netatalk?

I would have to agree with this. I have been running recent Asun versions
of Netatalk on Solaris 7 (both UFS and NFS underneath) for quite some time
and have never heard a report of a missing file on my LAN (20 some odd
users with a web developer using netatalk full time)

Tim

Tim Carlson                                  Voice:    (505) 984-8800x255
Director of Computing: Santa Fe Institute    Fax:      (505) 982-0565
WWW: http://www.santafe.edu/~tim             Email:   [email-masked]

---

From: "Robert B. Bell" <[email-masked]>
Date: Tue, 05 Dec 2000 11:15:42 -0800

> files with 32 or less characters in the name.  I simply shorten the names

That is not a lost of files by not being seen.  
I am talking about files deleted forever (except for backup tapes) 
from the servers filesystem.
I have since disabled the Network Trash Can by making in unwritable to everyone,
and thus have not been told of the problem since then.
The server stays up for a month or two before I reboot.
Some mac users stay logged on for weeks,
some log off every night.
We had about 20-40 users.
(We currently only have about 3 users).

> server running on?  What filesystem is on that server under Netatalk?

I am using mac with different OS 7's.
The server is Linux (RedHat 6.1) on an Intel processor.
The server's filesystem is ext2.
netatalk-1.4b2+asun2.1.4

---

From: BWS - Offwhite <[email-masked]>
Date: Tue, 5 Dec 2000 14:14:43 -0600 (CST)

Why do you have to reboot your server?  In the very least you could have
Netatalk shut down and restart every week if you want to close mac
connections to it.

So far it looks like 2 Redhat server have the problem while SunOS and
FreeBSD were ok.  I actually ran Netatalk first on a SunOS box (Solaris
2.5 I think) and it worked for me as well and that was a while ago.

Perhaps other Linux users will report problems as well.  It could be ext2
or something Redhat has going on.  Do we have any highly skilled Linux
admins out there?  If so, is there anything in Linux could be conflicting
with Netatalk?

The one problem I had with Netatalk on FreeBSD was it would crash the
server, but that was more of a traffic overload than anything.  I had
multiple graphic designers saving large files at the same time onto a disk
which was too slow to handle it.  It did not help that the server had
little memory and relied on swap space on the same disk.  FreeBSD would
kernel panic when it could not pull a page from swap because the disk head
was busy writing 3 large files to disk.

I solved that by getting a server with 3 SCSI drives and putting the
system disk on it's own and the Netatalk mount points on the other fast
SCSI drives.  Perhaps a similar problem can be happening here.

Since someone mentioned they ran fsck without any problems that tells me
it may not be the filesystem but it could be the Netatalk systems.  I am
not very familiar with how Netatalk manages connections and keeps
everything in line so I will not venture to guess at a way to troubleshoot
this area.

---

From: Matthew Wilson <[email-masked]>
Date: Tue, 5 Dec 2000 12:18:41 -0800 (PST)

> and thus have not been told of the problem since then.

How does this affect the user experience?  can they still delete files?

Here's my situation:

Slackware 7.1 Linux box, 256 RAM, 500G disk.  Approx 1000 users, though
most of them are windows users.  We'll have at least a 100 mac users once
this goes full swing though.  I'm just setting up netatalk on this box
(it's been a samba server for almost two years now) and I'm getting just a
little nervous with all this talk of disappearing files :P

---

From: Joerg Henner <[email-masked]>
Date: Tue, 5 Dec 2000 21:36:25 +0100

>users with a web developer using netatalk full time)

While we running Netatalk upon an SuSE 6.3 system (but with those Binary
RPM's from Sourceforge [V 1.4.99 - 08/11]) one of our Customers claiming
about "connection losts", and also that the Server is not shown in the List
- but he can connect via IP ...

Have anyone the same Situation/Problem ? Maybe any hints around ?

Greetings,

Joerg Henner.

-- 
LinuxHaus Stuttgart                    | Tel.:  +49 (7 11) 2 85 19 05
J. Henner & A. Reyer, Datentechnik GbR | D2:    +49 (1 72) 7 35 31 09
                                       | Fax:   +49 (7 11) 5 78 06 92
Linux, Netzwerke, Consulting & Support | http://lihas.de

---

From: "Robert B. Bell" <[email-masked]>
Date: Tue, 05 Dec 2000 12:46:59 -0800

> Netatalk shut down and restart every week if you want to close mac

I reboot the server because of two main reasons.
1) a power outage in our building.
They doing major construction to and around our site.
(We had 15000 employees at this site 15 years ago,
the company sold most of the land to make a quick dollar,
and now we are down to 1500 today.
Thus major building changes.)
They seem to cut power to our building 2 or 3 times a year,
(usually planned).
2) I am lazy.
When I need to put some patches on the system,
if no one is using it, I just reboot instead
of just restarting the subsystem.

When I want to close all the MAC connections,
I just do a "kill -HUP" to the main "afpd" task
which logs them all off in 5 minutes.
Then I can restart the afpd (which I have done in the past).

  Robert.

---

From: "Robert B. Bell" <[email-masked]>
Date: Tue, 05 Dec 2000 12:53:26 -0800

> How does this affect the user experience?  can they still delete files?

With the "Network Trash Can" having unwritable permissions,
just means that the user on the MAC doesn't put the deleted server files
in his trash can, but must delete them right away if he wants to delete the
file.

I set my permissions with

chmod 600 adm:everyone "Network Trash Folder"

user adm is my Admin account which I don't use much since
I can do 98% of my Admin work on the Linux box.

I could have just done
chmod 000 adm ...

Everything else works fine.

  Robert.

---

From: BWS - Offwhite <[email-masked]>
Date: Tue, 5 Dec 2000 17:12:08 -0600 (CST)

Good idea to simply reboot the host system after doing upgrades.  It sucks
your power keeps going out.

---

From: "lists-mail" <[email-masked]>
Date: Wed, 6 Dec 2000 03:07:48 -0000

These files were not placed in the trash, and were not in the network trash 
Can either. They were gone from the filesystem.

DAve

---

From: "lists-mail" <[email-masked]>
Date: Wed, 6 Dec 2000 03:20:08 -0000

> developers could get it fixed.

My Freebsd boxes do not show this problem either, both have FBSD Filesystems 
under the netatalk shares. While they are not very busy, maybe 30 gb a year 
transfered, they also haven't needed a reboot ;^)

Currently I am seeing this problem on two Linux boxes, both Slackware 7.0, 
both running ext2 file system. Kernel 2.2.13 with I believe 
netatalk-1.4b2+asun-2.1.3 (I sent that earlier)

> screwed up networking but you would think they would have AppleShare/IP

Again, I experienced no problems until OS 9. My mileage obviously varies 8^(

> seem to working quite well, with the required daily reboot of course.

I am very leary of doing that, I've never trusted the OS companies to _know_ 
what is best for me. After going from 9.02 to 9.04 I'd never trust them to 
give me a solid update again. 9.04 was very problematic for us with many 
crashes and lost data. We reverted back to 9.02 and have stayed there.

Thanks, all good points to ponder. Anyone else seeing stable netatalk on BSD 
and losing files on ext2

---

From: BWS - Offwhite <[email-masked]>
Date: Tue, 5 Dec 2000 21:49:06 -0600 (CST)

There is still another route to consider.  These machines may have a
security hole which has been compromized.  Since nothing else seems to
explain it away this could be the case.

I do not mean to bash Linux because I know at times Linux can be a better
choice than FreeBSD, but I just do not trust Linux.  My experiences with
Caldera, SuSE and RedHat were all bad and I gave them all a fair shake.  I
actually purchased a copy of each to support the cause, then I reverted
back to FreeBSD and have bought several CD sets already.

FreeBSD may not have the nice tools for setting up a GUI interface and it
may require a bit of learning to start, but I find that in the long run my
sys admin duties are much less time consuming.  I also find it performs
much better as a server.  (your experiences may be different)

And after 2 years of trying 3 major Linux distros and several versions of
FreeBSD I have to say I would have to recommend using FreeBSD for server
applications.  But then again, I have always heard the attitude Debian
developers have tends to be closer to that of the FreeBSD developers.  If
you are to stay with Linux, Debian may be a solid choice.  They seem to
choose stability over features.  The other distros have become feature
happy, which is fine for a workstation.

As a Sys Admin, my most valued feature is stability.  And I write this on
a FreeBSD server which has been up for 287 days in a row with Netatalk
running since day 1, no problems.

Choose your OS, but choose wisely.

---

From: "lists-mail" <[email-masked]>
Date: Wed, 6 Dec 2000 05:40:32 -0000

> explain it away this could be the case.
Been there, even changed the passwords and monitored the firewall. Everything 
is cool. I also made sure the simple stuff was checked. Everyone is in the 
appropriate group, all files and directories within the shared volume have the 
correct group permissions.

I'm still not inclined to think that it could be the servers OS. The server 
does nothing except Netatalk and mirror the live webserver. DNS, Sendmail, 
ODBC, PHP, Apache. It's a very clean install that sees nothing but web 
requests, private DNS queries, and afpd.

The rest of this message may be considered 'off topic' by some.

DAve

> back to FreeBSD and have bought several CD sets already.
You're preaching to the choir here ;^) I prefer FreeBSD myself but do to the 
fact I could not get Apache and Openlink to properly compile with support for 
multithreading, I had to switch to Linux. (this netatalk is on our development 
box running the dev web server)
 
> much better as a server.  (your experiences may be different)
GUI?  I have better use for that diskspace than XFree ;^)  It's a server!! 
While there is a monitor on it I've never used it since I installed the 
netatalk package. SSH is the only way into my boxes, telnet, rsh, etc have 
been removed with great predjudice.
 
> happy, which is fine for a workstation.
I've had best luck with Slackware when I have to use Linux. Very clean with 
minimal bells and whistles, BSD filesystem.

> running since day 1, no problems.

Nodding in agreement, as I write this in Twig, on my FBSD server, through my 
FBSD fi

---

From: "Lawrence Farr" <[email-masked]>
Date: Wed, 6 Dec 2000 13:36:46 -0000

My servers are all FreeBSD. They have never needed rebooting,
but do happily throw away files with anything other than the 
ancient port of netatalk that comes with FreeBSD.

The port however is completely useless to anyone using Quark/Photoshop,
which is 100% of my users. (Well known temporary file problem).

Oh, and they are on a private network.

Lawrence Farr
EPC Direct Limited mailto:[email-masked]
T:01179666123 F:01179666111 M:07970780901