[distcc] Re: lzo problem under HPUX
Martin Pool
mbp at samba.org
Fri Feb 20 04:44:13 GMT 2004
On 10 Feb 2004, Joe Buehler <jbuehler at hekimian.com> wrote:
> Martin Pool wrote:
>
> >Turning off mmap works in samba, or in distcc?
>
> Turning off mmap() in distcc makes it work.
>
> >I think it is probably the same bug.
>
> I agree.
>
> >I wonder if it would be safe to munmap before we ftruncate? That
> >might be less likely to cause any problems in interaction between mmap
> >and regular IO.
>
> I am going to look at the code -- I assume there is some sort of
> optimization that resulted in the current code that modifies the file
> after it is mapped.
Could you try this patch please?
Do you see the problem if you don't use LZO?
Index: head/src/compress.c
===================================================================
--- head.orig/src/compress.c 2003-09-22 13:15:28.000000000 +1000
+++ head/src/compress.c 2004-02-20 15:37:30.000000000 +1100
@@ -2,7 +2,7 @@
*
* distcc -- A simple distributed compiler system
*
- * Copyright (C) 2003 by Martin Pool <mbp at samba.org>
+ * Copyright (C) 2003, 2004 by Martin Pool <mbp at sourcefrog.net>
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as
@@ -76,7 +76,15 @@
* arbitrary, but basically means that debug objects and source will be
* mmaped, and errors and stripped output will not be. This has the
* incidental benefit that we won't try to mmap terminals too often.
- **/
+ *
+ * It might be nice to unify this code with that in pump.c, which deals with
+ * uncompressed files. There are some parallels between the routines.
+ * However the details are rather different, because with compressed files we
+ * do not know ahead of time how big the expanded form will be. This affects
+ * sending, where we need to make a large-enough temporary buffer to compress
+ * into. It also affects receipt, where we need to allow extra space for data
+ * coming in. So for the moment they remain separate.
+ */
/**
@@ -283,15 +291,7 @@
(long) in_len, (long) out_len,
(int) (out_len ? 100*in_len / out_len : 0));
- if (is_mmapped) {
- /* we over-allocated the file before; now shrink it down */
- if (ftruncate(out_fd, out_len) == -1) {
- rs_log_error("ftruncate fd%d failed: %s", out_fd,
- strerror(errno));
- ret = EXIT_IO_ERROR;
- goto out;
- }
- } else {
+ if (!is_mmapped) {
ret = dcc_writex(out_fd, out_buf, out_len);
}
@@ -299,10 +299,20 @@
free(in_buf);
if (is_mmapped) {
+ /* NOTE: We must ftruncate *after* unmapping, or HPUX's inconsistent
+ * page/buffer cache can be corrupted. */
if (munmap(out_buf, out_size) == -1) {
rs_log_error("munmap (output) failed: %s",
strerror(errno));
- ret = ret ? ret : EXIT_IO_ERROR;
+ ret = EXIT_IO_ERROR;
+ /* try to truncate anyhow.. */
+ }
+
+ /* we over-allocated the file before; now shrink it down */
+ if (ftruncate(out_fd, out_len) == -1) {
+ rs_log_error("ftruncate fd%d failed: %s", out_fd,
+ strerror(errno));
+ ret = EXIT_IO_ERROR;
}
} else if (out_buf != NULL) {
free(out_buf);
--
Martin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://lists.samba.org/archive/distcc/attachments/20040220/860daae3/attachment.bin
More information about the distcc
mailing list