Problems copying many files

Fri Jan 29 15:32:04 GMT 1999

Warning! Warning!

Never, ever, ever set the current limit to anything larger than 1024, at
least not on Solaris systems, and probably not on *any* Unix system you
cannot recompile completely, from scratch, after having updated
/usr/include/sys/select.h.

The reason is that the file descriptor set manipukation macros defined
in <sys/select.h> hardcode a maximum number of file descriptors and that
maximum is 1024.

Now, any applications which use select() could fail if run with a
current fd limit larger than 1024. This goes for RPC applications using
the select() polling interface as well (Solaris offers a select() and a
poll() interface for RPC applications). Granted, if applications using
select() are coded correctly then they wouldn't fail, or at least not
mysteriously.

But I have seen spectacular failures with rlim_fd_cur > 1024.

The max fd limit can be set to anything you want though.

I think code that uses select() should allow users to configure it such
that FD_SETSIZE is defined to an appropriately large number. 1024 is a
really small number nowadays, 16384 seems more appropriate to me.

You have been warned.

Nico

On Fri, Jan 29, 1999 at 11:03:21AM +1100, jandeep kang wrote:
> This message is in MIME format. Since your mail reader does not understand
> this format, some or all of this message may not be legible.
> 
> ------ =_NextPart_001_01BE4B1A.EB96B320
> Content-Type: text/plain
> 
> i have found the reason for the 1014 limit. it is the default OS limit
> of 1K files open by any process. under solaris 2.5 and above you can use
> 'ulimit -n' to get the current limit value and use 'ulimit -n
> needed_value' to set the new limit. alternatively you can add the
> following lines to your /etc.system file
> set rlim_fd_cur = needed_current_limit
> set rlim_fd_max = needed_max_limit
> 
> if you put the above in /etc/system then the users will have a default
> of 'needed_current_limit' open files and could set their own limit upto
> 'needed_max_limit' value. a superuser can go even beyond the maximum
> limit without setting it.
> after we set this thing right, it works fine now.
> thanks, Jandeep
> 
> ------ =_NextPart_001_01BE4B1A.EB96B320
> Content-Type: text/html
> Content-Transfer-Encoding: quoted-printable
> 
> <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
> <HTML>
> <HEAD>
> <META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
> charset=3DUS-ASCII">
> <META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
> 5.5.1960.3">
> <TITLE>Re: Problems copying many files</TITLE>
> </HEAD>
> <BODY>
> 
> <P><FONT SIZE=3D2>i have found the reason for the 1014 limit. it is the =
> default OS limit of 1K files open by any process. under solaris 2.5 and =
> above you can use 'ulimit -n' to get the current limit value and use =
> 'ulimit -n needed_value' to set the new limit. alternatively you can =
> add the following lines to your /etc.system file</FONT></P>
> 
> <P><FONT SIZE=3D2>set rlim_fd_cur =3D needed_current_limit</FONT>
> <BR><FONT SIZE=3D2>set rlim_fd_max =3D needed_max_limit</FONT>
> </P>
> 
> <P><FONT SIZE=3D2>if you put the above in /etc/system then the users =
> will have a default of 'needed_current_limit' open files and could set =
> their own limit upto 'needed_max_limit' value. a superuser can go even =
> beyond the maximum limit without setting it.</FONT></P>
> 
> <P><FONT SIZE=3D2>after we set this thing right, it works fine =
> now.</FONT>
> <BR><FONT SIZE=3D2>thanks, Jandeep</FONT>
> </P>
> 
> </BODY>
> </HTML>
> ------ =_NextPart_001_01BE4B1A.EB96B320--

Nico
--
Nicolas Williams	(x5220, Stamford, CT)
Stamford SysAdmin