Subject: netatalk search volume

From: Jeff Masud <[email-masked]>
Date: Tue, 31 Jul 2001 19:01:00 -0700 (PDT)

I'm having a couple of issues with our netatalk server, maybe someone can
give ideas.  

1. We are unable to search a 100GB volume, it gos and goes and goes... for
about 5 mins and the returns an error.  Is there any way to index the
server like how Watson index the local drive for faster searching?  

2. When I go to use Retrospect to backup the file server remotely,
Retrospect 4.3 will index the server display a list but as it starts to
run will return an error "Too many files open (-43)".   The server is
running a RAID but it's really import we can do DLT backups to archive old
projects.

thanks for any suggestions.

Jeff

---

From: Thomas Kaiser <[email-masked]>
Date: Wed, 01 Aug 2001 08:54:23 +0200

> I'm having a couple of issues with our netatalk server, maybe someone can

Please supply Netatalk version and the os you use.
 
> server like how Watson index the local drive for faster searching?

Since AFP 2.1 (if I remember correctly) there exists a feature called
FPCatSearch. If a Mac mounts a servers volume it gets the information
whether this volume will support FPCatSearch or not.

If it supports FPCatSearch then the clients search criteria will be passed
to the afp server and will be answered by him from information stored in a
volume database.

AFAIK, Netatalk doesn't support FPCatSearch at the moment, so your clients
finder have to run through the whole volume on its own, to search for files
matching your search criteria.
 
> run will return an error "Too many files open (-43)".

Under which MacOS version are you running Retrospect? There are some issues
with versions before MacOS 9.x and running out of file handles. Check it
with the ListOpenFiles Utility: <http://www.alsoft.com/AskAl/download.html>

I don't know if it has something to do with the way, RetroSpect tries to
check what to backup. With Helios EtherShare you can support a switch
"nofakeoffspring" when trying to backup from a Mac. This prevents RetroSpect
from giving up before backupping when your volume is full of folders.

There were some problems reported on volumes with a lot of files with custom
icons.

> The server is running a RAID but it's really import we can do DLT backups to
> archive old projects.

... and maybe to backup recent projects too? ;-)

Regards,

Thomas

---

From: Harald Wagener <[email-masked]>
Date: Wed, 01 Aug 2001 11:00:27 +0200

> > server like how Watson index the local drive for faster searching?

[snip]
> matching your search criteria.

A workaround is not to search the entire volume, but only a subfolder.
If Your data is sufficiently organized, this might be an option.

Regards,
	Harald

-- 
Harald Wagener      |                                Systemadministrator 
FCB/Wilkens GmbH    |                              Tel.:+49-40-2881-1252 
An der Alster 42    |                              Fax.:+49-40-2881-1263 
20099 Hamburg       |                         http://www.fcb-wilkens.com

---

From: [email-masked]
Date: Wed, 1 Aug 2001 05:22:12 -0700 (PDT)

I was able to resolve my volume searching.  

I had a hunch that it could be file naming convention.  I wrote a shell
script to go through all 159,000 files and check if there was any
macintosh extended chars or control chars and correct the file names.

After running this script I was able to search the volume.  It was damn
slow with so many files, but a volume search did work properly.

Jeff

---

From: [email-masked]
Date: Wed, 1 Aug 2001 05:27:04 -0700 (PDT)

> There were some problems reported on volumes with a lot of files with custom

I'm running Retrospect 4.3 on OS 9.1, it's a clean install with only
Retrospect on the machine.  I noticed something odd about this issue.  If
I unmount the file server and remount.

1. run retrospect do backup
2. out of 3 test it backed up 1014, 1017 and 1011 files then gives "too
many files open" error
3. run backup again
4. immediately gives "too many files open"
5. quit retrospect
6. unmount netatalk volume
7. remount netatalk volume
8. rerun retrospect and backup
9. backs up 1017 file and gives too many files error.  

So to me it looks like it's not completely a OS 9x issue it has something
to do with the netatalk volume that is mounted on the desktop.

> ... and maybe to backup recent projects too? ;-)

Yeeees that would be nice too.  

Jeff

---

From: Thomas Kaiser <[email-masked]>
Date: Wed, 01 Aug 2001 15:34:38 +0200

On Wed, 1 Aug 2001 05:22:12 -0700 (PDT), [email-masked] wrote:

> I had a hunch that it could be file naming convention.

You know that with MacRoman encoding there exists only one illegal char in a
filename suitable for afp?

It's the colon ":"

Maybe your encoding table setup (code pages) is wrong?

But I found that Netatalk treats ASCII 127 different than EtherShare (to
which it behaves normally absolutely identical comparing the filename's
encoding on the server)

Regards,

Thomas

---

From: [email-masked]
Date: Wed, 1 Aug 2001 05:40:59 -0700 (PDT)

Yeah you are probably right.  I did remove any string in a filename that
had ":hh" a colin with two hex characters afterwards. -j

---

From: Chip Mefford <[email-masked]>
Date: Wed, 1 Aug 2001 09:40:26 -0400

or so it seems. 

   I've upgraded from 1.4b2 to 1.5pre2 when I upgraded 
   my RH6 box to a RH7.1 box. 

   1.4b2 under RH 6 was quite controllable and worked well,
I've been using it for quite some time. 

   Since the upgrade, my uses now seem to connect 
   to ANY interface, although my atalkd.conf is as follows 

   ------include-------------- 
   eth1 -phase 2 -net 0-65534 -addr 65280.181 
   -----end include------------ 

And my afpd.conf is

   -----include--------------- 
   kiosk -port 12000 -address 192.168.0.4 -noguest
   ----end include------------

Kiosk of course, is the name of the 
interface eth1 at 192.168.0.4

   This set up worked and kept my Netatalk connections 
   on eth1 under 1.4b2, however under 1.5pre2, atalk
connections are allowed at all interfaces, and in some 
   case are going around the corner though the router to 
   get at eth0, which is not a bad thing in my case, just 
   really slow. 

   What have I done wrong? the .rpmnew file looks just 
   like the orginal default atalk.conf and non of this 
   seems very well documented, if there are differences 
   in this file relative to 1.4 

   Any input is a help, 

   thanks in advance 

   chipper

---

From: Thomas Kaiser <[email-masked]>
Date: Wed, 01 Aug 2001 15:56:52 +0200

On Wed, 1 Aug 2001 05:40:59 -0700 (PDT), [email-masked] wrote:

[quotings rearranged :-/ ]

> had ":hh" a colin with two hex characters afterwards. -j

":hh" is _not_ a hex value ("ff" aka 255 would be the highest ;-)

Interesting though. Netatalk normally displays these characters in its
MacRoman representation. In my eyes it is a violation of afp specs,
displaying characters that can confuse an afp client. Would be better if
Netatalk replaces these characters with another char (eg. ":c4") to map
these illegal chars to 'friendlier' ones.

Regards,

Thomas

---

From: Chris Murley <[email-masked]>
Date: Wed, 1 Aug 2001 10:25:01 -0500 (CDT)

-Chris

-----------------------
Christopher Murley
Application Index
http://www.appindex.net
http://www.appindex.org
-----------------------

---

From: Chip Mefford <[email-masked]>
Date: Wed, 1 Aug 2001 09:40:26 -0400

or so it seems. 

   -----end include------------ 

And my afpd.conf is

   ----end include------------

   

   Any input is a help, 

   thanks in advance 

   chipper

---

From: Thomas Kaiser <[email-masked]>
Date: Wed, 01 Aug 2001 19:35:33 +0200

[problems caused by displaying the path separator as part of filenames]
> I did remove any string in a filename that had ":hh" a colin with two hex
> characters afterwards. -j

Jeff didn't told us which netatalk version he's using (maybe 1.4-something?)

Assuming netatalks behaviour didn't change with actual versions (can anybody
confirm this please!), I would suggest trying to change netatalks displaying
behaviour in cases where it founds an colon in a filename.

I tried it with Ethershare 2.6 (also with AppleDouble-Encoding, which means
storing undisplayable MacRoman characters as a colon followed by the hex
representation of the char's ASCII value on the UNIX side).

ES maps legal hex values to MacRoman characters, eg. if a UNIX filename
contains ":41" (decimal 65), the macs will se it as "A". If the hex value is
out of range (eg greater than ff) then ES will map the colon to a pipe
symbol ("|" --> es will map the UNIX name ":hh" to "|hh" when viewed from
mac)

This prevents a violation of afp specs and would be a big improvement in
situations where files and folders are created from the UNIX side without
taking care of afp specs (normally you cannot store a file with a colon in
its filename on an afp server from a mac)

If somebody can confirm that Netatalk 1.5pre6 does also display a UNIX file
or folder with ":hh" in its name as ":hh" to an afp client, then it's time
for a feature request on the netatalk-devel list :-)

Regards,

Thomas

---

From: [email-masked]
Date: Wed, 1 Aug 2001 14:14:00 -0700 (PDT)

> Jeff didn't told us which netatalk version he's using (maybe 1.4-something?)

FYI, I'm running 1.5pre6 on a Linux Mandrake 7.2 with Kernel
2.2.17-21mdksecure.

Jeff