Subject: Re: Hidden Files Not Being Copied

From: Jason Quigley <[email-masked]>
Date: Thu, 18 Jan 2001 12:39:12 +0100

On Thursday, January 18, 2001, at 09:40 AM, Basil Hussain wrote:

>  
> In my experience, MacOS is quite dumb when it comes to some things, and 
> thinking about the way it probably works leads me to imagine that when you 
> initiate a duplication of a directory (or just a straight copy) MacOS 
> recursively traverses the directory structure and tells the underlying 
> filesystem - in this case, Netatalk - to individually copy each file. But, 
> as at the MacOS end it doesn't even *know* about any hidden files, it won't 
> tell the filesystem to copy them. So basically, if you can't see it, you 
> can't copy it. 
>

MacOS is not dumb at all about this. What it's doing makes perfect sense! Files beginning with a dot on MacOS are normally considered to be disk driver files. You can't see these files and copying them to your hard drive could cause problems later.

>  
> But, as I said, I might be wrong. Really, the base of this problem really 
> stems from the clash between MacOS having no notion of hidden files and UNIX 
> systems making prolific use of them. Ah, well... 
>  

MacOS does have a notion of hidden files. They are simply called invisible files. Based on what you're saying, it could be said that the problem stems from UNIX systems having no notion of forked files. If they did, then a Mac file server on UNIX would be much easier to implement.

Cheers,
Jason.

---

From: Basil Hussain <[email-masked]>
Date: Thu, 18 Jan 2001 13:41:56 +0000

Hi,

> MacOS is not dumb at all about this. What it's doing makes perfect sense!
> Files beginning with a dot on MacOS are normally considered to be disk driver
> files. You can't see these files and copying them to your hard drive could
> cause problems later.

Ah, I see. I didn't know about 'disk driver' files. So what you're saying is
that MacOS knows 'dot' are there, but it doesn't copy them because it thinks
they're something else.

> MacOS does have a notion of hidden files. They are simply called invisible
> files. Based on what you're saying, it could be said that the problem stems
> from UNIX systems having no notion of forked files. If they did, then a Mac
> file server on UNIX would be much easier to implement.

Yes, you could put it that way as well... :) I suppose an implementation of
forked files is why Windows NT4/2000 has always had AppleTalk file sharing,
as the NTFS filesystem also has 'streams', as MS call them.

So anyway, I'll just have to continue to copy my .htaccess and .htpasswd
files in a command prompt on the Netatalk server, as this is really neither
a Netatalk or MacOS problem.

Regards,

------------------------------------------------
Basil Hussain ([email-masked])

---

From: Bill Moran <[email-masked]>
Date: Thu, 18 Jan 2001 09:43:59 -0500

> a Netatalk or MacOS problem.

Really, when you think about it, shouldn't Netatalk translate dotfiles
to "invisible" files on the Mac? Mangle the filename with an underscore
or something. Just a thought/suggestion.

-Bill

---

From: Basil Hussain <[email-masked]>
Date: Thu, 18 Jan 2001 15:04:08 +0000

Hi all,

> or something. Just a thought/suggestion.

This has got me thinking.

The above suggestion wouldn't really hold water with any kind of
out-of-the-norm usage, because what if you actually have non-hidden files
who's names start with underscores? The same applies to almost any character
you might think of as well.

However, I just remembered seeing somewhere someone mention something about
sending 'popup' messages to Mac clients using Netatalk - like in a similar
manner to Winpopup on Windows.

Could it not be implemented so, that when a folder duplication or copy
action is perfomed, Netatalk recognises this somehow and checks the contents
of the folder for hidden 'dot' files/directories (other than .AppleDouble,
obviously) and sends a message to the client Mac saying if there are?
Something along the lines of: "The folder you just copied contained <x>
hidden files and <y> hidden directories. If you wanted to copy these too,
you must do it manually.".

Although (as pondered in my previous message), if the file copy actions are
initiated client-side, then this couldn't happen, could it? Or could it?

Anyway, this is just an idea. Any comments?

Regards,

---

From: Bill Moran <[email-masked]>
Date: Thu, 18 Jan 2001 10:30:47 -0500

> you might think of as well.

I can't see how that would cause a problem. The underscore (or whatever
character) is not there to indicate that the file is "invisible", there
would be some sort of flag for that at the filesystem level, like the
hidden/system/readonly attributes on the old DOS fs. The Mac must have
some sort of attributes to indicate that the file is "invisible" (OK, I
admit, I don't KNOW, but how else would it be done) The mangle character
(underscore in this example) would simply make the filename valid for
the Mac. The Mac wouldn't reverse the translation (If you created a file
from the Mac that started with an underscore it wouldn't necessarily be
invisible) However, on the UNIX box, the file would be created starting
with a dot so that it retained its invisible status if it were created
invisible from the Mac. Somewhat the same way long filenames are mangled
to fit old 8.3 notation. It's a translation scheme really, actually you
could do away with the _ altogether, just send the name minus the dot.
You could get filename contention problems, but this to could be dealt
with somehow.

> initiated client-side, then this couldn't happen, could it? Or could it?

I'm not terribly familiar with Macs, how do you set a file to be
invisible?
The popup option would be a nice feature, but it seems to me that this
could be implemented in a better manner ... something to ponder on. My
problem is that I'm simply not that familiar with Macs.

-Bill

---

From: Ray Zimmerman <[email-masked]>
Date: Thu, 18 Jan 2001 12:35:50 -0500

>I'm not terribly familiar with Macs, how do you set a file to be

Each file has a set of attributes stored with it indicating whether 
it is visible, the file type, creator app, whether it is locked, etc.

I should also mention that on MacOS a filename starting with a . is 
NOT an illegal name. Or to get rid of the double negative ... it's 
JUST FINE. I have .htaccess files on my Mac and they've never behaved 
any differently than files with other names.

The problem is that in *nix the . means it's invisible, whereas on 
the Mac it just means that you wanted to call your file .<something> 
... netatalk has to try to do what you want and I think it seems to 
be doing pretty close to the right thing, though it does seem that it 
could take .files (with the exception of .AppleDouble) and treat them 
as .files with the invisible bit set (I see no need to change the 
name). This way they would be "seen" by the MacOS (but not by the Mac 
user) and hence copied along with the enclosing folder.

Thoughts?

	Ray

---

From: Bill Moran <[email-masked]>
Date: Thu, 18 Jan 2001 12:58:17 -0500

> it is visible, the file type, creator app, whether it is locked, etc.

Is this part of the resource fork?
If so, how is this resource fork represented through Netatalk? The
invisible attribute could be stored there as well.
Another thought: Why are dotfiles ignored to begin with? Why not just
hide the .AppleDouble and .AppleDesktop files but show other dotfiles?

Getting philosophical,
Bill

---

From: Ray Zimmerman <[email-masked]>
Date: Thu, 18 Jan 2001 13:01:15 -0500

At 12:35 PM -0500 1/18/01, Ray Zimmerman (yes, I know that's me) wrote:
>be doing pretty close to the right thing, though it does seem that 
>it could take .files (with the exception of .AppleDouble) and treat 
>them as .files with the invisible bit set (I see no need to change 
>the name).

After thinking through it a bit more I think you would need to strip 
the '.' for the Mac side as suggested by Bryn Hughes in order to make 
it 2-way transparent.

	Ray

---

From: Ray Zimmerman <[email-masked]>
Date: Thu, 18 Jan 2001 13:13:59 -0500

>Is this part of the resource fork?

I'm not 100% sure where it is stored, but I think it's part of the 
file system definition (wherever file create and mod dates are 
stored). It's definitely not in the resource fork though, because 
files with no resource fork have this info.

>invisible attribute could be stored there as well.

I believe it is. IIRC the AppleDouble format stores the resource fork 
as well as these extra file attributes not included in other files 
systems. If you create files on the Mac and copy them to netatalk all 
of these attributes stay intact. If the file starts with a . that 
gets encoded when copied to a netatalk volume.

>hide the .AppleDouble and .AppleDesktop files but show other dotfiles?

The only way to get a dotfile is to create it on the *nix side I 
believe, in which case it is intended to be invisible. So I still 
think it should be made available to the Mac side, but as a file with 
the invisible bit set.

Next best would be just to make them available as visible files.

	Ray