Subject: RE: Please help: long filenames not showing up

From: Michalowski Thierry <[email-masked]>
Date: Mon, 21 Aug 2000 10:22:42 +0200

No, I think this is not handled in samba, because they mimic the Windows
behaviour, which does not preserve the short name along time.

<example>
You can check this by creating a file named "myfilebbbbbbbbbbb" . Windows
will map it to "myfile~1" .
Then, create in the same directory a file named "myfileaaaaaaaaaa" . Windows
will map it to "myfile~1" and map the first file to "myfile~2" .
</example>

This was just to suggest two things: 
If we are doing such a great thing with netatalk, we should avoid this
crappy behaviour - thus ignoring samba code on the point.

Second thing is: we also should try to preserve something as horrible as a
filename _extension_ if we do not want to break the PC-Mac exchange feature.

Both things do lead to a clever algorithm to do the mapping, and I humbly
suggest we should wait for persistent IDs to be implemented for both files
and directories under netatalk to link that "short file names" things to it.
IMHO.

Just my 0.02 EUR

Thierry

-----Original Message-----
From: Mike Fedyk [mailto:[email-masked]]
Sent: Monday, August 21, 2000 9:36 AM
To: Steve Freitas
Cc: Ryan McBeth; Netatalk
Subject: Re: Please help: long filenames not showing up

How would we deal with aliases in this case?

I'm relatively new to the mac world.  Do aliases use strait paths ie
zone:comp:share:dir:dir:file?  Or some number?  How do we make sure
that if new files are added with the same begining 31 chars that the
first short name wouldn't change?

Sorry, this is probably already handled in samba...

Mike

Steve Freitas wrote:
> 
> >Has anyone looked at the code that creates the short names in samba?
> >Maybe something like that could be done.
> 
> Yes, it would be a fantastic mod if Netatalk dynamically shortened names.
> I hate dealing with the user arguments:
> 
> Jane: "I copied it to your directory."
> 
> John: "No you didn't."
> 
> Jane: "Yes I did."
> 
> John: "Look, it's not even there. See?"
> 
> Jane: "But I copied it... It must be a virus!"
> 
> Bleah.
> 
> Steve

---

From: Matthew Geier <[email-masked]>
Date: Tue, 22 Aug 2000 09:30:50 +1000

> Michalowski Thierry wrote:
> 
> No, I think this is not handled in samba, because they mimic the
> Windows behaviour, which does not preserve the short name along time.
> 
> <example>
> You can check this by creating a file named "myfilebbbbbbbbbbb" .
> Windows will map it to "myfile~1" .
> Then, create in the same directory a file named "myfileaaaaaaaaaa" .
> Windows will map it to "myfile~1" and map the first file to "myfile~2"
> .
> 
> </example>

 On a Samba share yes, on a windows machines, in most case no, the short
name will stay the same as BOTH long and short names are stored in the
filesystem. Fat32 stores the long name as a 'comment', and I understand
NTFS does it the otherway, the short name is stored as a 'comment'. With
the both names stored in the filesystem, it can keep some consistancy,
Samba in an effort to avoid having to syncronise metadata files (ala
NetAtalk's .AppleDouble
directories) builds the short names on the fly and just lives with the
occasional clash. The trade off is that a win32 API program will always
use the long name, and older 8.3 API programs are dying out, so the
occasional clash caused by unexpected short name changes is
just 'lived with', to avoid extra complexity needed for only a very
small subset of programs.

 32+ filenames in NetAtalk would need a better solution, you can't just
take the Samba approach and say 'well programs that use that API are
dying out', this solution that works most of the time will do.

 I've already hit the problem, with a user moving between Mac and PC.
They couldnt see they long named files from the Mac. I've since found a
reverse problem, some windows versions at least take great exception to
files ending in just . (Users Imac died, now using a Windows 98 laptop,
it refused to open files/folders with named ending in .)

-- 
Matthew Geier			[email-masked]
Arts IT Unit			+61 2 9351 4713
Sydney University

---

From: Mike Fedyk <[email-masked]>
Date: Mon, 21 Aug 2000 23:26:05 -0700

> dying out', this solution that works most of the time will do.

Can we store something more in the .AppleDouble for this purpose? 
Maybe we could even have symlinks in there with the appropriately
short name point to the actual file contained in the .AppleDouble's
parent dir?  Just make a nightly cron job to deal with the dangling
symlinks from windows delete/renames and you're set.

Thinking of that.  What happens if a pc user deletes a mac file that
had a resource fork in the .AppleDouble?  Does afpd clean it up on the
fly? or do they just sit there wasting away?

> it refused to open files/folders with named ending in .)

I just had this happen for the today, for the first time.  It was on a
win98 box.  Haven't tested it from one of our NT workstations.  

I started the transition from an unstable ASIP 6.1 the day before.  So
far, this has been the only problem. :)

Mike

---

From: Rasca Gmelch <[email-masked]>
Date: Tue, 22 Aug 2000 10:46:19 +0200

hi mike,

> fly? or do they just sit there wasting away?

at least for linux/unix:

my latest version of filemanager for linux (gtk+ based) supports
copying, renamed and removing of AppleDouble files as well.
it's not perfect until now but it should work.

the filemanager itself looks like a macos finder and has support for
the most common finder key shortcuts (alt+n, alt+o, alt+backspace, ..).
might be interessting for some people .. don't know.

see http://rasca.home.pages.de/xap/

ps: changing attibutes (chown, chmod) of the corresponding AppleDouble
files will be supported in the next release.

does someone has a compact code for parsing .AppleDouble files?
the netatalk code look a little bit confusing to me.. or could
someone point me to the right places in the netatalk sources?

cu
 rasca
-- 
Triad Berlin Projektgesellschaft mbH * Marburger Str. 3 * D-10789 Berlin
Fon +49 (0)30 219098-69 * Fax +49 (0)30 219098-61 * http://www.triad.de/
.____________/\ Triad Berlin: Culture Media Environment /\_____________.

---

From: Harry Zink/Netatalk List <[email-masked]>
Date: Tue, 22 Aug 2000 02:01:51 -0700

...under pre-39, I have an installation that was storing a healthy amount of
data on a netatalk volume, and they lost roughly 5.5 gigs of data today.

Let's ignore for the moment that the user's insistence to not have or use a
backup system (to save money) finally paid off in a big way (sarcasm) - the
circumstances of this data loss seem rather strange, though interesting.

The Linux undelete utilities were apparently able to recover a healthy
portion of the 5.5 gigs (sieving through to identify the data now).
Apparently, the data loss occured after a crash/freeze on a client Mac,
while the Mac was writing to the server.

A sub-directry that used to contain 15-17 other directories now was left
with just 4 directories.

There seems to be an issue with stability after a client crash/recovery, or
so it would seem. What were the circumstances of others on here that have
lost data?

Maybe we can find similarities that could lead to a solution.

Harry

---

From: Harry Zink/Netatalk List <[email-masked]>
Date: Tue, 22 Aug 2000 02:08:12 -0700

I was thinking about the resource fork situation on the Mac, and the
apparent need for AppleDouble directories on netatalk - or is there?

While I understand that there is a need to store the resource information
somewhere, why not find a way to store said information in the actual file,
instead of storing it in a separate directory.

Thus, each file could have a header which provides pointers to data and
resource 'fork', and the location they are stored - yet the file itself is
stored in the fs as a single file.

Thus, you'd have:

 +--------+-----------------+---------------------+
<| header | xx data fork xx | ** resource fork ** |>
 +--------+-----------------+---------------------+

Location of data/resource fork could be specified by either byte offset, or
actual location on the disk (the latter would make it easier to update
changing sizes of either).

The big benefit of it all is that if another network system, like Samba,
deletes a file, no other strangglers of that file need to be cleaned up -
furthermore, there is no need to manage lots of extra files in lots of
duplicated locations - which should make responsiveness and administration
far easier.

Of course, I'm sure I overlooked something - so, what are the drawbacks to
this suggestion?

Harry
-- 
CAN'T SEE MAC IN CHOOSER UNDER RedHat 6.2: If you get errors starting
AppleTalk under RedHat 6.2 and Mandrake 7.1, you need to manually add "alias
net-pf-5 appletalk" to your /etc/conf.modules file.

---

From: Rasca Gmelch <[email-masked]>
Date: Tue, 22 Aug 2000 11:18:15 +0200

hi,

> stored in the fs as a single file.

and then you can't access the files in the normal way under the
unix/linux system - i mean the data fork! :-(

cu
 rasca

---

From: Franz Dürr <[email-masked]>
Date: Tue, 22 Aug 2000 11:16:41 +0200

> symlinks from windows delete/renames and you're set.
Mike,

the problem with storing short names permanently (say, in the AppleDouble
directory) is to keep it in sync with changes to file names applied by
non-atalk-clients. I think, users would not accept it, if you tell them:
'Hey, just wait a day and you'll see that file'. And watching the
directories permanently would be a waste of resources.
To do it cleanly, you would have to apply hooks  in the kernel's create,
rename and unlink, so what you end up with, is basicly a kernel mode afpd
(perhaps not a bad idea anyway?). If you want to avoid that, creating short
names 'on the fly' is IMHO the second worst solution. To distingish files
that have the same short file name in a persistent manner, you could include
the inode-numer in the file name (problem: Different short file name after
file copy into a different directory) or a hash based on the creation date
(same problem, if you don't preserve the creation date when copying) or a
hash / checksum íd based on the complete name (problem when two names result
in the same id, persistency lost in this case). That id would have to be
appended in any case, regardless whether the short name is unique or not.
The need to encode a 32+ file name will probably be relatively rare, so I
think, it can be accepted if one non persistent file name turns up once in a
year or so.
Admittedly no perfect solution, but probably fast and easy to implement.

Just my Euro 0.0223 (= $0.02)

---

From: [email-masked]
Date: Tue, 22 Aug 2000 10:55:11 -0400

> From:    Harry Zink/Netatalk List <[email-masked]>
> To:      "[email-masked]" <[email-masked]>

> instead of storing it in a separate directory.

AKA: AppleSingle.

:wes

---

From: Sean Phelan <[email-masked]>
Date: Tue, 22 Aug 2000 10:23:02 -0500

>while the Mac was writing to the server.

What is this Linux undelete utilities that you speak of?  Could you tell me
some more information or where I can find some information?

Thanks in advance

Sean Phelan
Information Systems
Chicago Reader

---

From: Matthew Keller <[email-masked]>
Date: Tue, 22 Aug 2000 11:38:09 -0400

> instead of storing it in a separate directory.

	Then there is a bunch of "garbage" at the beginning (or end) of each
file, which will prevent any other service from burning on reading them
(Linux/UNIX, Windows, etc.)

-- 

 Matthew Keller
 WebMaster & Lead Programmer/Analyst
 Distributed Computing/Telemedia
 Information Services Division
 State University of New York at Potsdam
 
 Website: http://mattwork.potsdam.edu/
 PGP: http://mattwork.potsdam.edu/crypto/
 Webcam:  http://webcam.mattwork.potsdam.edu:85/

---

From: Bob Rogers <[email-masked]>
Date: Tue, 22 Aug 2000 12:04:36 -0400 (EDT)

From: Harry Zink/Netatalk List <[email-masked]>
   Date: Tue, 22 Aug 2000 02:01:51 -0700

   ...under pre-39, I have an installation that was storing a healthy amount of
   data on a netatalk volume, and they lost roughly 5.5 gigs of data today . . .

   A sub-directry that used to contain 15-17 other directories now was left
   with just 4 directories . . .

Did you try fsck on the partition?  Maybe you could find more of those
missing directories that way.

					-- Bob Rogers

---

From: Harry Zink/Netatalk List <[email-masked]>
Date: Tue, 22 Aug 2000 09:47:16 -0700

> some more information or where I can find some information?

Go to www.freshmeat.net

type 'undelete' in the search field.

You'll be presented with several. Pick the one right for you (I used
'recover').

Harry

---

From: Harry Zink/Netatalk List <[email-masked]>
Date: Tue, 22 Aug 2000 09:54:19 -0700

> unix/linux system - i mean the data fork! :-(

Okay, so such a header would interfere with a standard *nix reading these
files. While this is irrelevant for application files, since they are of no
use natively anyways, this could be a slight issue with data files (again,
if you are using the device uniquely and solely as an appltealk file server,
I don't see much need to ever need to 'use' the files in the traditional
way. Only way I can see is to natively copy a file, in which the single file
format again would be of benefit.

Now, I understand that the header, thus, is bad - how about just placing
this information at the very end of the file? That way, no garbage at the
end; the fs knows where the end of the file is, netatalk could just look
there, in a designated place, and grab the same information.

Any drawbacks to that?

It seems to me, regardless of how outdated the applesingle format might be,
there are great benefits to storing everything in a single file,
particularly when multiple filesystems are involved.

Just curious, but what situations could arise that make it necessary to be
able to have other services 'burning and reading' these files? Only
situation I can see is SAMBA accessing them, in which case it will just
serve them as single files - recognition of data files could be achieved
either by an update to SAMBA, or by using the aforementioned garbage at end
suggestion.

Either way, SAMBA currently is able to recognize netatalk files and volumes
(and retain filenames), so some form of conversion seems to already take
place, does't it?

Harry

---

From: Harry Zink/Netatalk List <[email-masked]>
Date: Tue, 22 Aug 2000 09:55:20 -0700

> missing directories that way.

Yeah, certainly did - made no difference.

Now, in addition, these disappearances happened in RAID drives - did anyone
else have their problems on RAID drives?

Harry

---

From: Harry Zink/Netatalk List <[email-masked]>
Date: Tue, 22 Aug 2000 10:26:42 -0700

Is anyone else getting Will Davis' Out Of Office replies everytime you post
to the list?

Anything that can be done?

Harry

---

From: Marc Miller <[email-masked]>
Date: Tue, 22 Aug 2000 14:04:54 -0700 (PDT)

I am getting those.  There's nothing we can do except to set up mail
filters.  

> Anything that can be done?

============================================================
	/\/\arc ._|. /\/\iller ([email-masked])
	Computer Room Consultant
	Information Technology/Lab Management
============================================================
I can be contacted through the Communication Center link from
http://www.mother.com/~mjmiller/

---

From: Rasca Gmelch <[email-masked]>
Date: Wed, 23 Aug 2000 19:15:25 +0200

hi harry,

> format again would be of benefit.

we use netatalk in a mixured network (linux,nt,macos).. so the file
itself doesn't have more than the data fork it self.

> there, in a designated place, and grab the same information.

i don't see more drawbacks to stay on the current solution with
the .AppleDouble files.. i would improve the server / daemon
software and wouldn't change the current concept. here we
would loose time to find a stable solution.. imho.

cu
 rasca

---

From: Mike Fedyk <[email-masked]>
Date: Sat, 26 Aug 2000 20:27:01 -0700

> directories permanently would be a waste of resources.

It seems that I wasn't clear in my assumptions you are referencing.  I
wanted afpd to find files with names 32+ long and create a short
name.  That short name would be stored in the .AppleDouble.  

If we come across a new file that is 32+ we just add it to the list. 
There would be no need to keep the mac users from seeing the file
until after a cronjob was run.

We can have a nightly cronjob that finds references to non existing
files that were removed by a samba/ftp/whatever user, and removes
them.  

That brings me to another question.  How does afpd deal with mac files
with a resource fork that are removed by a non-macos user?  Do they
stay orphaned, or does afpd clean them up during the next listing of
that dir?

BTW, with the current solution (or should I say non-solution), you
would have to have a cronjob that runs fairly often to rename the
offending files.  That is, if you don't want the admin to have to deal
with it every time it comes up.  So, most any solution in afpd would
require a lot fewer resources than what we have currently.

> (perhaps not a bad idea anyway?). 

Wouldn't that be in the libc, and not the kernel?  Just like httpd in
the kernel, I don't think afpd should go there.

> If you want to avoid that, creating short

What if we just rename the files?  That way, everyone will see the
same name, and won't be confused by differing names.  This can be
turned off with a config option if someone doesn't like remaming. 
Then we would revert back to storing the short name in .AppleDouble...

If two files have the same short name, doesn't macos refuse to copy
the second file into the same directory?  This would force the user to
deal with the naming conflict, just like normal.

Mike