[syslog-ng]Syslog-ng's memory footprint growing constantly on DEC alpha
Wim-Jan Hilgenbos
wimjan@xs4all.nl
Mon, 29 Oct 2001 14:17:20 +0100
On Wed, Oct 24, 2001 at 12:28:15PM +0200, Balazs Scheidler wrote:
> On Tue, Oct 23, 2001 at 07:14:26PM +0200, Wim-Jan Hilgenbos wrote:
> > Hi,
> >
> > I have quite a lot of trouble with syslog-ng. More specific with the
> > memory footage. It grows within a day to a size of over 130M after which
> > it stops logging except for its own internal messages and then a short
> > time later it dies.
> >
> > I have tried all kinds of combinations of compilers/compiler options.
> > motivated in part by the unaligned messages and the former emails on
> > this list.
> >
> > I'm able to get rid of the analigned access messages by using the native
> > compiler, but I can't get a stable memory-size.
> >
> > My idea is that log messages are only partially freed. When I look
> > at the core I see a lot of 'tails of logmessages' varying in length
> > from 2 to 15 chars.
> >
> > The whole application becomes more stable and grows slower when I use
> > the '-taso' option of the native compiler, which directs the loader
> > to load the modules in 31-bit address space.
>
> hmm.. interesting. I _hopefully_ found the unaligned problem, so you might
> give a try to gcc as well.
>
> to fix the problem you should change the first line in libol/acconfig.h:
>
> from
>
> #define DEBUG_ALLOC 0
>
> to
>
> #undef DEBUG_ALLOC
>
> then rerun autoheader & configure. and check that olconfig.h contains #undef
> DEBUG_ALLOC instead of #define DEBUG_ALLOC 0.
>
> btw: the previous leak problem turned out to be an OS issue (the NIS+ one on
> Linux), so this might be one again. Try to turn off DNS usage (use_dns(no)
> in your global options), and try to get gnu malloc lib for your platform.
>
> --
> Bazsi
> PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
>
> _______________________________________________
> syslog-ng maillist - syslog-ng@lists.balabit.hu
> https://lists.balabit.hu/mailman/listinfo/syslog-ng
Hi,
It looks like the unalignment error is fixed and with it the memoryleak.
I've got 1.5.12 running in two development environments. I've started one
last thursday and the other last friday and they seem to be holding. No
memory growth, no crashes. Logging and forwarding seem to be
working fine. I going to install the new binary in some test enviroments today
and probably take them in production next week. I anything funny shows up by
then "I'll be back" ;-)
Thnx for your support.
Things I have changed/patched to get this far.
Taking vanilla libol-0.3.1 and syslog-ng-1.5.12
* Untarred the archives somewhere.
* Its not possible to use GNU-malloc
since glibc is not ported to Alpha's True64 Unix
* GNU-gcc works fine, but if True64 Unix cc is used use the compilerflags:
-assume_aligned_objects -no_misalign -taso -call_shared
Formally only one of -assume_aligned_objects and -no_misalign should have
to be specified, but better save than sorry.
* changed the firstline of libol-0.3.1/src/acconfig.h
from #define DEBUG_ALLOC 0
to #undef DEBUG_ALLOC
ran autoheader and configure
* Changed the CFLAGS option in libol-0.3.1/src/Makefile
CFLAGS= -Wall -g
* Changed the CFLAGS option in libol-0.3.1/src/Makefile
CFLAGS= -Wall -g -O2
* removed syslog-ng-1.5.12/src/cfg-lex.c and syslog-ng-1.5.12/src/cfg-grammar.c
to force recreation by flex and bison. If I don't do this I get linking
problems. I also added the path to the flex-lib by hand.
LIBS=/usr/gmsdev/lib/libol.a -L/usr/gmsdev/lib -lfl -lxnet
* Patched syslog-ng-1.5.12/src/affile.c with the patch of Oct 22
* added an empty statement at line 157 of syslog-ng-1.5.12/src/center.c
below the label 'next_connection'
True64 cc gives an error on a label without any further statements.
* patched syslog-ng-1.5.12/src/main.c with the patch below to get a pid even
when running in the foreground. I have a wrapperscript that restarts syslog-ng
whenever it crashes. It just starts the program in the foreground in a infinite
while loop.
To enable starting and stopping of syslog-ng from outside this script I need
the pid of syslog and the script.
(The patch is from the 1.5.11 but works without modifications in 1.5.12)
-------------- snip snip --------------
*** /home/hilgenbo/sources/syslog-ng-1.5.11/src/main.c Sun Aug 26 17:06:25 2001
--- main.c Fri Oct 19 13:41:42 2001
***************
*** 213,218 ****
--- 213,232 ----
return 0;
}
+ int save_pid()
+ {
+ int fd;
+
+ fd = open(pidfilename, O_CREAT | O_WRONLY | O_NOCTTY | O_TRUNC, 0600);
+ if (fd != -1) {
+ struct ol_string *pid_s = c_format("%i\n", getpid());
+ write(fd, pid_s->data, pid_s->length);
+ ol_string_free(pid_s);
+ close(fd);
+ }
+ }
+
+
int go_background()
{
pid_t pid;
***************
*** 225,243 ****
}
pid = fork();
if (pid == 0) {
- int fd;
close(wakeup_pipe[0]);
! fd = open(pidfilename, O_CREAT | O_WRONLY | O_NOCTTY | O_TRUNC, 0600);
! if (fd != -1) {
! struct ol_string *pid_s = c_format("%i\n", getpid());
! write(fd, pid_s->data, pid_s->length);
! ol_string_free(pid_s);
! close(fd);
! }
return wakeup_pipe[1];
! }
! else if (pid == -1) {
werror("Cannot fork(), (%z)\n", strerror(errno));
exit(1);
}
--- 239,249 ----
}
pid = fork();
if (pid == 0) {
close(wakeup_pipe[0]);
! save_pid();
return wakeup_pipe[1];
! } else if (pid == -1) {
werror("Cannot fork(), (%z)\n", strerror(errno));
exit(1);
}
***************
*** 352,360 ****
{ "foreground", no_argument, NULL, 'F' },
{ "help", no_argument, NULL, 'h' },
{ "version", no_argument, NULL, 'V' },
! { "chroot", required_argument, NULL, 'C' },
! { "user", required_argument, NULL, 'u' },
! { "group", required_argument, NULL, 'g' },
#ifdef YYDEBUG
{ "yydebug", no_argument, NULL, 'y' },
#endif
--- 358,366 ----
{ "foreground", no_argument, NULL, 'F' },
{ "help", no_argument, NULL, 'h' },
{ "version", no_argument, NULL, 'V' },
! { "chroot", required_argument, NULL, 'C' },
! { "user", required_argument, NULL, 'u' },
! { "group", required_argument, NULL, 'g' },
#ifdef YYDEBUG
{ "yydebug", no_argument, NULL, 'y' },
#endif
***************
*** 427,433 ****
--- 433,442 ----
}
if (do_fork) {
wakeup_fd = go_background();
+ } else {
+ save_pid();
}
+
if (!CONFIG_INIT(backend->configuration, NULL)) {
char res = 1;
-------------- snip snip --------------
Thnx again,
Wim-Jan