[syslog-ng] Re: syslog-ng Digest, Vol 6, Issue 10

[Roberto Nibali]

> Erm. No.
> 
> On a Linux 2.6 kernel, in include/linux/threads.h:

Well, not all 2.6 kernels. Although 2.6.5 SuSE spits following:

/*
  * This controls the default maximum pid allocated to a process
  */
#define PID_MAX_DEFAULT 0x8000

/*
  * A maximum of 4 million PIDs should be enough for a while:
  */
#define PID_MAX_LIMIT (4*1024*1024)

A simple stupid test would wrap the pids at 32k:

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>
#include <errno.h>

#define SECS_TO_SLEEP 10

int main(int argc, char *argv[]) {
         unsigned long i;
         unsigned long loop;
         pid_t pid;

         loop = 0;
         if (argc > 1) {
                 loop = strtoul(argv[1], NULL, 10);
         } else {
                 printf("%s NUM_OF_FORKS\n", argv[0]);
         }
         for (i = 0; i < loop; i++) {
                 pid = fork();
                 if (pid < 0) {
                         perror("fork");
                         exit(EXIT_FAILURE);
                 } else if (pid > 0) {
                         /* child */
                         fprintf(stderr, "forked pid: %d\n", pid);
                         usleep(SECS_TO_SLEEP * 1024 * 1024);
                         exit(EXIT_SUCCESS);
                 }
                 i++;
         }
         exit(EXIT_SUCCESS);
}

> On 32-bit kernels, there's some code in the /proc that borks up if the
> PID is over 64K (it invents an inode number based on the pid, and the left-shift
> overflows).  Running on a 64 bit system, the default is 4M, and can be raised
> *much* higher if needed.

I knew that you could have 4Mio threads but I somehow wasn't aware that 
you can have 4Mio forked processes. Since the Linux world changed to 
NPTL we do not necessarily call clone() anymore when doing a 
pthread_create() and thus we don't really fork() anymore, as with 
linux_threads-0.10.

On my system, despite starting threaded applications, I get the tgrp == 
pid (ps -AL for example), since my glibc uses linux_threads and thus LWPs:

ratz at webphish:~> /lib/libc.so.6
GNU C Library stable release version 2.3.3 (20040325), by Roland McGrath 
et al.
Copyright (C) 2004 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Configured for i586-suse-linux.
Compiled by GNU CC version 3.3.3 (SuSE Linux).
Compiled on a Linux 2.6.4 system on 2004-04-05.
Available extensions:
         GNU libio by Per Bothner
         crypt add-on version 2.1 by Michael Glad and others
         linuxthreads-0.10 by Xavier Leroy
         GNU Libidn by Simon Josefsson
         NoVersion patch for broken glibc 2.0 binaries
         BIND-8.2.3-T5B
         libthread_db work sponsored by Alpha Processor Inc
         NIS(YP)/NIS+ NSS modules 0.19 by Thorsten Kukuk
Report bugs using the `glibcbug' script to <bugs at gnu.org>.

> And it isn't just Linux either - seen on an AIX box:
> 
> # cut -f2 -d'[' /var/adm/seclog | cut -f1 -d']' | sort -nr | uniq | head
> 191446
> 191208
> 191188
> 191186
> 191184
> 191170
> 191124
> 191122
> 191108
> 191104

Mhh, these must be PIDs I reckon, since IIRC AIX can only handle 512 
threads per process, right?

> Yep, PIDs pushing the 200K mark (AIX plays some funky games when generating
> the 'next PID' to reduce cache-line aliasing issues when dereferencing entries
> in some structures indexed by PID).

Thank you, Valdis, of course. Since I haven't actively remarked high 
PIDs in my current development field I was still under the distinct 
impression that PIDs are 16bit; I probably just overlooked the number of 
digits in the logfiles all those years. That about concludes my ability 
to comment on Jason's patch ;).

Best regards,
Roberto Nibali, ratz
-- 
echo 
'[q]sa[ln0=aln256%Pln256/snlbx]sb3135071790101768542287578439snlbxq' | dc