Subject: Re: Hidden Files Not Being Copied

From: "Bryn Hughes" <[email-masked]>
Date: Thu, 18 Jan 2001 09:12:26 -0800

It really doesn't sound like it would be impossible to impliment... something along the lines of:

if <filename> starts with '.' then
  if <filename> is not '.AppleDouble' then
    copy file, stripping leading period
    set invisible to true

I know that the MacOS by default WILL copy any hidden files/folders in a directory.  If you want to see nice evidence of this copy the root level of a Mac's disk to a netatalk share.  All the invisible files get copied too.  In fact you'll even get messages about copying the desktop printers, as they reside inside the invisible 'Desktop Folder' at the root level of the disk.  So in theory files comming FROM a UNIX share that start with a period should be made invisible.

I really don't see any reason to keep anything whatsoever like a period in the beginning of the name once it's on the Mac.  It's there in Unix as a flag, it just needs to be paired with the invisible flag on the Mac side.

Actually, that would probably solve a few of the stranger things that happen with invisible folders/files being visible.  Is it possible to have netatalk perform this translataion both ways?  I.E. files from a Mac with the invisible flag get a period added to their name on the Unix side, and files beginning with a period get an invisible flag when copied to the Mac side.  The ideal solution would be to have netatalk seamlessly doing this conversion even with something as simple as a directory listing.  That way it would be completely transparent to everyone.

-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Bryn Hughes
Macintosh Tech Support
Information and Computing Services
Vancouver Community College

email:[email-masked]
ph: (604) 443-8702
fax: (604) 443-8353
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

>>> Bill Moran <[email-masked]> - 1/18/01 7:33 AM >>>
Basil Hussain wrote:
> 
> Hi all,
> 
> > Really, when you think about it, shouldn't Netatalk translate dotfiles
> > to "invisible" files on the Mac? Mangle the filename with an underscore
> > or something. Just a thought/suggestion.
> 
> The above suggestion wouldn't really hold water with any kind of
> out-of-the-norm usage, because what if you actually have non-hidden files
> who's names start with underscores? The same applies to almost any character
> you might think of as well.

I can't see how that would cause a problem. The underscore (or whatever
character) is not there to indicate that the file is "invisible", there
would be some sort of flag for that at the filesystem level, like the
hidden/system/readonly attributes on the old DOS fs. The Mac must have
some sort of attributes to indicate that the file is "invisible" (OK, I
admit, I don't KNOW, but how else would it be done) The mangle character
(underscore in this example) would simply make the filename valid for
the Mac. The Mac wouldn't reverse the translation (If you created a file
from the Mac that started with an underscore it wouldn't necessarily be
invisible) However, on the UNIX box, the file would be created starting
with a dot so that it retained its invisible status if it were created
invisible from the Mac. Somewhat the same way long filenames are mangled
to fit old 8.3 notation. It's a translation scheme really, actually you
could do away with the _ altogether, just send the name minus the dot.
You could get filename contention problems, but this to could be dealt
with somehow.

> Could it not be implemented so, that when a folder duplication or copy
> action is perfomed, Netatalk recognises this somehow and checks the contents
> of the folder for hidden 'dot' files/directories (other than .AppleDouble,
> obviously) and sends a message to the client Mac saying if there are?
> Something along the lines of: "The folder you just copied contained <x>
> hidden files and <y> hidden directories. If you wanted to copy these too,
> you must do it manually.".
> 
> Although (as pondered in my previous message), if the file copy actions are
> initiated client-side, then this couldn't happen, could it? Or could it?

I'm not terribly familiar with Macs, how do you set a file to be
invisible?
The popup option would be a nice feature, but it seems to me that this
could be implemented in a better manner ... something to ponder on. My
problem is that I'm simply not that familiar with Macs.

-Bill

---

From: Ray Zimmerman <[email-masked]>
Date: Thu, 18 Jan 2001 12:57:37 -0500

I think this is an excellent idea ... probably the best we can do ... 
unfortunately changing the invisible flag from MacOS of my .htaccess 
file, for example, would change the name on the *nix side (and now my 
web server would no longer see it). As a Mac user, changing the state 
of the invisible flag should not have any effects beyond changing the 
visibility of the file, but I guess you pretty much have to throw out 
those kind of expectations when you start working in a mixed OS 
environment.

So, I think it's a good idea.

	Ray

At 9:12 AM -0800 1/18/01, Bryn Hughes wrote:
>I know that the MacOS by default WILL copy any hidden files/folders 
>in a directory.  If you want to see nice evidence of this copy the 
>root level of a Mac's disk to a netatalk share.  All the invisible 
>files get copied too.  In fact you'll even get messages about 
>copying the desktop printers, as they reside inside the invisible 
>'Desktop Folder' at the root level of the disk.  So in theory files 
>comming FROM a UNIX share that start with a period should be made 
>invisible.
>
>I really don't see any reason to keep anything whatsoever like a 
>period in the beginning of the name once it's on the Mac.  It's 
>there in Unix as a flag, it just needs to be paired with the 
>invisible flag on the Mac side.
>
>Actually, that would probably solve a few of the stranger things 
>that happen with invisible folders/files being visible.  Is it 
>possible to have netatalk perform this translataion both ways?  I.E. 
>files from a Mac with the invisible flag get a period added to their 
>name on the Unix side, and files beginning with a period get an 
>invisible flag when copied to the Mac side.  The ideal solution 
>would be to have netatalk seamlessly doing this conversion even with 
>something as simple as a directory listing.  That way it would be 
>completely transparent to everyone.

---

From: Peter Westlake <[email-masked]>
Date: Thu, 18 Jan 2001 18:20:46 +0000

>I know that the MacOS by default WILL copy any hidden files/folders in a directory.  If you want to see nice evidence of this copy the root level of a Mac's disk to a netatalk share.  All the invisi…

I think changing the name is a really bad idea. If you try to duplicate
a folder, you should get a duplicate. There's no reason not to have the
dot on a Mac. If you really are duplicating, or if you are copying to
another Unix directory, then you should get a true copy when you look
at it from the Unix side. The dot prefix is only treated specially by
"ls" and a few other programs - it doesn't really mean "invisible". To
most programs, it is just part of the file's name. I suggest that if
you are doing an actual Duplicate operation (command-D), then the dotted
files should be copied. If you are copying the files to the Mac, it should
be optional whether they are ignored, copied, or copied and made invisible.

Peter.

---

From: Ray Zimmerman <[email-masked]>
Date: Thu, 18 Jan 2001 13:56:08 -0500

>be optional whether they are ignored, copied, or copied and made invisible.

I think that if a dotfile on Unix is going to translate to an 
invisible on the Mac side, then it has to strip the dot and the 
translation has to go both ways, otherwise you run into trouble.

In the scenarios you mentioned, it wouldn't be a problem. If you 
duplicate or copy a Unix directory with dotfiles, the Mac would see 
them as invisible non-dotfiles which netatalk would translate back to 
dotfiles when writing the copy.

An example of a problem if it doesn't strip the dot would go 
something like this. You copy an invisible file called 'foo' from Mac 
to a netatalk volume. On the *nix side you would have a file called 
'.foo'. Copy that back to the Mac and you have an invisible file 
called '.foo'. Copy that back to your netatalk volume and now you 
have a file called '..foo'.

	Ray

---

From: Danny Sauer <[email-masked]>
Date: Thu, 18 Jan 2001 13:17:52 -0600

> have a file called '..foo'.

What's wrong with "if it has a dot and is MacInvisible, leave alone, else,
add a dot" combined with "leave the filename alone otherwise"?   That's
*my* preference...  I don't like filenames being screwed with in any but
the most absolutely neccesary cases.

--Danny, not following this thread closely enough to know if that's already
	been suggested

---

From: Bill Moran <[email-masked]>
Date: Thu, 18 Jan 2001 17:02:49 -0500

Well, it seems to me after much discussoin, that the basic config that
we wanted already exists. In the long run, there may be ways to refine
it, but I'm pretty sure that the options are manageable as they
currently stand (my opinion anyway)
(As a side note, I feel really stupid discussing this as I was: once the
config options were pointed out to me I remembered reading about them
...)

I still want to know why there are directories that can't be deleted:
and I still suspect it has something to do with the .Apple* files. Has
anyone else witnessed this behaviour? Is it something to do with FreeBSD
only?

-Bill

---

From: Carsten Neumann <[email-masked]>
Date: Fri, 19 Jan 2001 06:03:24 +0100

On Thu, 18 Jan 2001, Bill Moran wrote:
[snip]
> anyone else witnessed this behaviour? Is it something to do with FreeBSD

Do you mean directories which contain dotfiles?
This also applies linux.

Netatalk doesn't see them but can tell that this directory is not empty.

But I tried just that scenario:
1. created an empty folder (mac)
2. created (touched) a dotfile within that (u*x)
3. dropped it into the trash (mac)
4. tried to empty the trash (mac)
   error: ...contains items that are in use...
5. put away (cmd-y) (mac)
   error: The folder <folder> could not be put away, because it was created on the desktop.

The last looks strange!

Regards

	Carsten

---

From: Bill Moran <[email-masked]>
Date: Fri, 19 Jan 2001 00:53:09 -0500

> Do you mean directories which contain dotfiles?

No, no dotfiles (except for the .Apple* files that Netatalk creates)

The sequence may be slightly different than I described. Here was
exactly what I was doing ...
1. Use WinNT to copy a folder tree full of stuff onto the FreeBSD server
via Samba
2. Sit down at the FreeBSD box and verify that the files/directories had
ownership/perms that it should be deletable by Netatalk users.
3. Attempt to delete the folder tree from a Mac
Result: all files are deleted, directories are not. Mac give a "can not
delete because in use" error.
On the server, the .AppleDouble files remain (these were created by
Netatalk when the Mac browsed the tree) but no other files exist ... "rm
-r *" deletes the tree without error. It can also be deleted from the
WinNT station.

I also duplicated the test with Irix/NFS with the same results (I.E.
create tree with Irix, Mac deletes files but not folders)

Here is the behaviour in a nutshell:
NT/Samba created, Irix/NFS deletes without error
Irix/NFS created, NT/Samba deletes without error
FreeBSD created, NT/Samba deletes without error
FreeBSD created, Irix/NFS deletes without error
Mac/Netatalk created, NT/Samba deletes without error
Mac/Netatalk created, Irix/NFS deletes without error
NT/Samba created, Mac/Netatalk cannot delete
Irix/NFS created, Mac/Netatalk cannot delete
FreeBSD created, Mac/Netatalk cannot delete
Mac/Netatalk created, Mac/Netatalk deletes without error

> Netatalk doesn't see them but can tell that this directory is not empty.

I understand, but considering Netatalk CREATED the .AppleDouble files,
should it delete them? I even dug into the source code, and as far as I
can tell, it should ALWAYS delete .AppleDouble files before removing a
directory.

> 2. created (touched) a dotfile within that (u*x)

An .AppleDouble dotfile, or another dotfile? When I run this scenerio,
there are no dotfiles in the directories other than the .AppleDouble
files.

>    error: The folder <folder> could not be put away, because it was created on the desktop.

The sequence you used here is different from what I described. If I
create a folder tree from the Macintosh, moving it to the trash succeeds
without error. I only have problems if the tree was NOT originally
created by the Mac.

The search continues ...

-Bill

---

From: "Heath Henderson" <[email-masked]>
Date: Fri, 19 Jan 2001 06:06:26 -0600

Hello all, I am writing this in hopes that someone knows of my problem.  It
isn't really a problem but more of an irritation.  I just upgraded our DNS
server RH 6.2 from Kernel 2.2.12 to 2.2.16.  All went well but now After a
reboot I get Reverse Lookup failure errors.  They hit our Proxy server and
die, but I am not sure why they are being displayed.  They look like the
following

IP_MasQ:Reverse ICMP:failed checksum from xxx.xxx.xxx.xx

(Where xxx.xxx.xxx.xx is our Proxy server)

I am not sure how it was set up before, but I am sure that it is possible to
turn this display off so as I do not see this when I am in the middle of
working on things on this box.  I am not sure why it does this, but I need
to turn off the constant display of this.

Any help would be great.

Thanks

Heath Henderson

---

From: Duncan Sinclair <[email-masked]>
Date: Fri, 19 Jan 2001 14:34:55 +0000

>-r *" deletes the tree without error. It can also be deleted from the

Do any of the files have a name longer than 32 characters???

These files also are never seem by a Mac, and so cannot be deleted,
and a directory containing them will not be deleteable.

Duncan.

---

From: Bill Moran <[email-masked]>
Date: Fri, 19 Jan 2001 09:56:28 -0500

> and a directory containing them will not be deleteable.

Nope, already been checked. In my tests I created directory trees
/test/test/test with files called test.txt in them. Same behaviour
though.

Thanks for the reply.

My original question still stands: Can anyone duplicate this behaviour?
Has anyone tried and got different results than me?

-Bill

---

From: Mike Johnson <[email-masked]>
Date: 19 Jan 2001 09:02:13 -0800

Hello,

I can't say that I've seen that problem before, but you can modify what
messages are sent to the console.

Save a backup copy of /etc/syslog.conf and open syslog.conf. Look for
(probably) the first line that's something like this:

kern.*          /dev/console

change it to something like

kern.*   /var/log/kernel

or wherever you store your log files. If there is anything else in that
file that says /dev/console, you might want to send that info to a text
file, too.

If you still want to get kernel error messages, then add this line.

kern.err   /dev/console

I can't say what priority of error the one you are describing is, but
hopefully the .err will filter the less important info. Check out 'man
syslog.conf', too.

- Mike Johnson

---

From: Basil Hussain <[email-masked]>
Date: Fri, 19 Jan 2001 17:50:26 +0000

Hi,

> change it to something like

Wow, thanks! I've been wanting to find out what settings to modify to do
exactly this for ages! And now your advice to someone else turns up out of
the blue...

This is gonna be a real boon. I've been trying to keep track of kernel-level
networking errors ("Suspect short first fragment" errors, if anyone can tell
me what they mean) on a couple of my servers that I suspect are being
generated by some evil port-scanning technique, but as they only got logged
to the console I miss most of them.

Anyway, thanks again!

Regards,

------------------------------------------------
Basil Hussain ([email-masked])