100% CPU load

[ronnie sahlberg]

thanks,

merged.


On Wed, Jan 13, 2010 at 3:32 AM, Stefan (metze) Metzmacher
<metze at samba.org> wrote:
> Rusty Russell schrieb:
> Hi Ronnie,
>
>>> I've found another problem with ctdbd spinning
>>> with 100% CPU load reading 0 bytes from the log child.
>
> Here're the fixes for the spinning.
>
> The problem were the dup2() call in the wrong direction, which implicit
> closed the stdout.
>
> git pull git://git.samba.org/metze/ctdb/wip.git master-for-ronnie
>
> metze
>
>>>
>>> ctdb_log_handler()
>>>
>>> gets n = 0 from read and log is log_state.
>>>
>>> I think it's wrong that we don't test for n <= 0 and not error out.
>>>
>>> I don't understand the code enough to fix it for the log == log_state.
>>
>> This is connected to stdout and stderr, so it should *never* be closed.
>> Hence Ronnie's patch description:
>>
>> commit bcf494b81f4277dc75f05faccf0c446bd15f6e2b
>> Author: Ronnie Sahlberg <ronniesahlberg at gmail.com>
>> Date:   Tue Dec 15 19:04:52 2009 +1100
>>
>>     This is a dodgy patch.
>>
>>     I saw once where the master ctdbd logging structure was talloc freed
>>     which caused issues.
>>     So only free the structure if it is NOT the master structure.
>>
>>     This needs to be looked into in more detail.
>>
>> I've added a hack to log closing of stdout/stderr, and I don't hit
> anything in
>> normal running (the parent closes them to set up logging, that's it):
>>
>> commit 6bcf19397e33cc7e4c63bdd3688003ae3a3add5c
>> Author: Rusty Russell <rusty at rustcorp.com.au>
>> Date:   Mon Dec 21 15:34:28 2009 +1030
>>
>>     patch who-is-closing-stderr.patch
>>
>> diff --git a/lib/replace/replace.c b/lib/replace/replace.c
>> index cec158b..3b8dac8 100644
>> --- a/lib/replace/replace.c
>> +++ b/lib/replace/replace.c
>> @@ -621,3 +621,25 @@ int rep_socketpair(int d, int type, int protocol,
> int sv[2])
>>       return pipe(sv);
>>  }
>>  #endif
>> +
>> +#undef close
>> +int _close(int fd)
>> +{
>> +     if (fd == 0 || fd == 1 || fd == 2) {
>> +             static int errorfd = -1;
>> +             char msg[100];
>> +             if (errorfd == -1) {
>> +                     int lowfd = open("/tmp/close-errors.log",
>> +                                      O_WRONLY|O_APPEND|O_CREAT, 0600);
>> +                     if (lowfd < 0)
>> +                             abort();
>> +                     if (dup2(lowfd, 1000) < 0)
>> +                             abort();
>> +                     close(lowfd);
>> +                     errorfd = 1000;
>> +             }
>> +             sprintf(msg, "Warning: %i closing fd %i!\n", getpid(), fd);
>> +             write(errorfd, msg, strlen(msg));
>> +     }
>> +     return close(fd);
>> +}
>> diff --git a/lib/replace/replace.h b/lib/replace/replace.h
>> index f8a89a7..ad471c4 100644
>> --- a/lib/replace/replace.h
>> +++ b/lib/replace/replace.h
>> @@ -546,4 +546,6 @@ typedef int bool;
>>  #define QSORT_CAST (int (*)(const void *, const void *))
>>  #endif
>>
>> +#define close(x) _close(x)
>> +
>>  #endif /* _LIBREPLACE_REPLACE_H */
>
>
>