problem syncing files on RAM in embedded device

[John Van Essen]

On Fri, 10 Jun 2005, Vijay Ram.C <cvijay at banyannetworks.com> wrote:
> 
> i am working on an networking product which has  a 32 MB flash memory
> and a 64MB RAM.
> i need to sync a file named appl.gz which is of size 3.1MB from a unit
> with ip 192.168.10.1 to a unit with ip 192.168.10.2.
> the rsync is called from a C code which does a fork-execve operation.
> the command line options being passed to rsync is as below:
> 
> command: rsync --stats -aI --temp-dir=/tmp/  192.168.10.1::path/appl.gz
> /config/appl.gz
> 
> in the above command tmp/ is a part of the filesystem on the RAM space
> while /config/ is mounted onto a flash space.
> the above operation is done in two phases. one is a direct function call
> which does the above operation and the second  is  in a
> while loop which keeps doing the operation every one hour. For  first
> phase an error is thrown as below.
> 
> write failed on appl.gz: Success
> unexpected EOF in read_timeout

I understand your rationale for using an old version, but you are not
likely to get much help on such an old version.  For example, the above
bogus "Success" message was fixed in 2.6.0.  When a write() call returned
with less than the requested bytes being written (a partially successful
write), the write_file routine returned immediately instead of adjusting
the buffer pointer and length and retrying until it got a -1 returned or
successfully wrote the entire buffer.  See write_file() in fileio.c if
you want to try to port the fix to your version.

> after giving the above error the, syncing of files terminates.
>  From the second phase onwards,  the syncing of the same file with the
> same options as given above works well.

That's odd.  Is it actually ever rsyncing a new version of the file?
Since you are using -I, only the file size will be checked.

> On experimenting, the same code worked well when done with the
> --temp-dir=/config/ i.e, temporary redirected to flash ie, where the
> destination is to be present.
> The platform being used in the product is monta vista linux and. i have
> compiled the source code of version 2.4.8 and ported on the device.
> there is also enough space on the flash  as well as the RAM for the temp
> file to be made.
> I read in some old mails in the mailing list which said that this error

The "unexpected EOF in read_timeout" error?  It may be related to the
previous bogus error.

> comes when there is no data on the socket. But on certain experimentation,
> i tried syncing another file of size less that 512KB instead of the
> above mentioned file. At that time everything went well and the syncing
> was successfull. The main reason for me going with this old version of
> rsync is the space taken by the rsync binary.
> Can anybody help me figure out the reason for the failure on the first
> go.Is there any extra care that should be taken which am missing when
> redirecting temporary file to RAM space??

None that I'm aware of.  There is an extra whole-file copy that rsync has
to do every time since an intra-partition rename won't work.  Are you using
temp space out of concern that there won't be room on the destination for
the file and it's temporarily-created replacement?  Otherwise, creating the
temp file on the destination is more efficient since only a simple rename
is needed.

    John