January 2009 - syslog-ng - lists.balabit.hu

are nested quotes possible in parser
by Michael Hocke 14 Jan '09

14 Jan '09

Hi there, my Avocent console servers are sending console port output via syslog to my syslog server running syslog-ng 3.0.1. Console output looks like this: Jan 13 00:18:39 sysl@cyc2 Buffering: S39.gwa [Jan 13 00:14:04.379: % EARL_NETFLOW-SP-4-TCAM_THRLD: Netflow TCAM threshold exceeded, TCAM Utilization [97%]] Jan 13 00:18:53 sysl@cyc2 Buffering: S42.sw2f [2009 Jan 13 00:18:53 EST -05:00 %ETHC-5-PORTTOSTP:Port 3/27 joined bridge port 3/27] Jan 13 00:18:53 sysl@cyc2 Buffering: S42.sw2f [2009 Jan 13 00:18:53 EST -05:00 %DTP-7-PORTLINKDOWN:Port 3/27 Link down] Jan 13 00:18:53 sysl@cyc2 Buffering: S42.sw2f [2009 Jan 13 00:18:53 EST -05:00 %ETHC-5-PORTFROMSTP:Port 3/27 left bridge port 3/27] The goal is to store the console output within square brackets into separate files named after the server that created this output. The first line of the example above should go into the file "gwa" while the others go into "sw2f". This is what I have so far: > source s_udp { udp (); }; > > # --- parse console server output > # separate port description from message > parser p_console_output { > csv-parser (columns ("CONSOLE.SOURCE", "CONSOLE.MSG") > delimiters (" ") > quote-pairs ("[]") > template ("${MSGONLY}")); > }; > > # extract port label from port description > parser p_console_source { > csv-parser (columns ("CONSOLE.PORT", "CONSOLE.LABEL") > delimiters (".") > template ("${CONSOLE.SOURCE}")); > }; > > # --- destination of console output > destination d_console_output { > file ("/usr/local/var/log/remote/${HOST_FROM}/console/$ > {CONSOLE.LABEL}" > template ("${CONSOLE.MSG}\n")); > }; > > # --- filter console output > filter f_console_output { > facility (local7) and host ("^sysl(a)cyc.*"); > }; > > # --- log console output > log { > source (s_udp); > filter (f_console_output); > parser (p_console_output); > parser (p_console_source); > destination (d_console_output); > }; This works just fine with the last three lines of my example data above. The problem I am having is that if the console output (the text between square brackets) contains its own square brackets the message will cut off right after the first occurrence of the closing bracket. The first line of my example data will look like this: Jan 13 00:14:04.379: %EARL_NETFLOW-SP-4-TCAM_THRLD: Netflow TCAM threshold exceeded, TCAM Utilization [97% instead of Jan 13 00:14:04.379: %EARL_NETFLOW-SP-4-TCAM_THRLD: Netflow TCAM threshold exceeded, TCAM Utilization [97%] I could probably get around this by using a rewrite rule using PCRE but considering the amount of data that needs to be looked at this solution is going to be very expensive. Is there a way to make syslog-ng aware of nested quotes? If not, is there something in the pipeline to support this in future releases? If not, I will be willing to take a shot and implement this. Any pointers or suggestions are welcome. Thanks. - Michael

2 1

include branch merged
by Balazs Scheidler 14 Jan '09

14 Jan '09

Hi, After the release of syslog-ng OSE 3.0.1, I merged include file support to mainline. It is already pushed to the git repository, but also should be in tomorrow's snapshot. The syntax is: include "file-or-directory-to-be-included"; If you specify a directory, syslog-ng is going to include the files in alphabetical order. The same rules apply to as to run-parts(8). -- Bazsi

1 0

Getting named pipes to work
by Mike Rykowski 13 Jan '09

13 Jan '09

Hello, I'm new to syslog-ng and trying to get named pipes to work. I'm running version 2.0.2. I get the following messages in the log on start up: Jan 12 10:09:02 netlog-e0 netlog syslog-ng[11791]: syslog-ng starting up; version='2.0.2' Jan 12 10:09:02 netlog-e0 netlog syslog-ng[11791]: Error opening file for writing; filename='/var/log/netlog/sec-pipe', error='No such file or directory (2)' In my conf file I have: destination dst_sec { pipe("/var/log/netlog/sec-pipe")}; On the host I have the pipe setup as: prw-r--r-- 1 syslog-ng syslog-ng 0 Jan 12 10:05 sec-pipe The syslog-ng process is run with user 'syslog-ng'. So I added the owner and group options for the pipe command and set them to syslog-ng and that resulted in the same message. Any ideas? -- Mike Rykowski NU-IT Telecommunications and Network Services

4 7

syslog-ng graceful shutdown
by Rosenblum, Jason 13 Jan '09

13 Jan '09

I have recently compiled syslog-ng open source edition. I noticed that the install does not create a startup/shutdown script in /etc/init.d. What is the proper way to shutdown or reload syslog-ng gracefully? Thanks, Jason

2 1

Re: [syslog-ng] Fw: Error encountered while building syslog-ng-2.1.3 on RHEL3
by Chanchal Verma 12 Jan '09

12 Jan '09

Hi Balazs, Thanks for your expert advise. I have successfully compiled syslog-ng-2.1.3 with --enable-static-linking option. I were using old version of gcc (3.2.3) which causing compilation (make) fail. I upgraded my gcc to 4.1.2 to get rid of compilation errors. Though it is working fine till now, I have faced some error messages on standard video output while doing 'make'. I am not sure if this will impact new build syslog-ng application in terms of created problem in logging features. I am here pasting the error output: ************************************************* macros.c: In function âresult_appendâ: macros.c:137: warning: pointer targets in passing argument 2 of âg_string_append_lenâ differ in signedness macros.c: In function âlog_macro_expandâ: macros.c:149: warning: pointer targets in passing argument 2 of âresult_appendâ differ in signedness macros.c:218: warning: pointer targets in passing argument 2 of âresult_appendâ differ in signedness macros.c:238: warning: pointer targets in passing argument 2 of âresult_appendâ differ in signedness macros.c:246: warning: pointer targets in passing argument 2 of âresult_appendâ differ in signedness macros.c:268: warning: pointer targets in passing argument 2 of âresult_appendâ differ in signedness macros.c:413: warning: pointer targets in passing argument 2 of âresult_appendâ differ in signedness macros.c:432: warning: pointer targets in passing argument 2 of âresult_appendâ differ in signedness macros.c:450: warning: pointer targets in passing argument 2 of âresult_appendâ differ in signedness filter.c: In function âfilter_netmask_evalâ: filter.c:452: warning: âaddr.s_addrâ may be used uninitialized in this function logmsg.c: In function âlog_msg_parseâ: logmsg.c:151: warning: pointer targets in assignment differ in signedness logmsg.c:216: warning: pointer targets in passing argument 2 of âg_string_assign_lenâ differ in signedness logmsg.c:224: warning: pointer targets in assignment differ in signedness logmsg.c:242: warning: pointer targets in passing argument 1 of âstrlenâ differ in signedness logmsg.c:283: warning: pointer targets in passing argument 2 of âg_string_assign_lenâ differ in signedness logmsg.c:309: warning: pointer targets in passing argument 2 of âg_string_assign_lenâ differ in signedness logmsg.c:335: warning: pointer targets in passing argument 2 of âg_string_assign_lenâ differ in signedness logmsg.c:394: warning: pointer targets in assignment differ in signedness logmsg.c:398: warning: pointer targets in assignment differ in signedness logmsg.c:429: warning: pointer targets in assignment differ in signedness logmsg.c:463: warning: pointer targets in assignment differ in signedness logmsg.c:474: warning: pointer targets in assignment differ in signedness logmsg.c:486: warning: pointer targets in assignment differ in signedness logmsg.c:499: warning: pointer targets in assignment differ in signedness logmsg.c:520: warning: pointer targets in assignment differ in signedness logmsg.c:525: warning: pointer targets in assignment differ in signedness logmsg.c:530: warning: pointer targets in passing argument 2 of âg_string_assign_lenâ differ in signedness main.o(.text+0x133): In function `setup_creds': /home/soetest1/syslog-ng-2.1.3/src/main.c:324: Using 'initgroups' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking libsyslog-ng.a(misc.o)(.text+0xc4): In function `resolve_group': /home/soetest1/syslog-ng-2.1.3/src/misc.c:283: Using 'getgrnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking libsyslog-ng.a(misc.o)(.text+0x144): In function `resolve_user': /home/soetest1/syslog-ng-2.1.3/src/misc.c:258: Using 'getpwnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/local/lib/libglib-2.0.a(gutils.o)(.text+0xbd8): In function `g_get_any_init_do': /home/soetest1/glib-2.16.0/glib/gutils.c:1646: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/local/lib/libglib-2.0.a(gutils.o)(.text+0xbc8):/home/soetest1/glib-2.16.0/glib/gutils.c:1645: Using 'setpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/local/lib/libglib-2.0.a(gutils.o)(.text+0xbe3):/home/soetest1/glib-2.16.0/glib/gutils.c:1647: Using 'endpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/local/lib/libglib-2.0.a(gutils.o)(.text+0x9d1):/home/soetest1/glib-2.16.0/glib/gutils.c:1589: Using 'getpwnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/local/lib/libglib-2.0.a(gutils.o)(.text+0xa10):/home/soetest1/glib-2.16.0/glib/gutils.c:1595: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking libsyslog-ng.a(afinet.o)(.text+0xcbb): In function `afinet_resolve_name': /home/soetest1/syslog-ng-2.1.3/src/afinet.c:103: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking libsyslog-ng.a(misc.o)(.text+0x50a): In function `resolve_hostname': /home/soetest1/syslog-ng-2.1.3/src/misc.c:166: Using 'gethostbyaddr' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking libsyslog-ng.a(misc.o)(.text+0x464): In function `getlonghostname': /home/soetest1/syslog-ng-2.1.3/src/misc.c:71: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/local/lib/libnet.a(libnet_resolve.o)(.text+0x4dc): In function `libnet_name2addr6': : Using 'gethostbyname2' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking libsyslog-ng.a(afinet.o)(.text+0x6bb): In function `afinet_set_port': /home/soetest1/syslog-ng-2.1.3/src/afinet.c:58: Using 'getservbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking loggen.o(.text+0x6cc): In function `main': /home/soetest1/syslog-ng-2.1.3/tests/loggen/loggen.c:236: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking ************************************************ Grateful if anybody put their comment on this. With Regards, Chanchal Verma Balazs Scheidler <bazsi(a)balabit.hu> 01/12/2009 08:44 PM To Chanchal Verma/GIS/CSC@CSC cc Subject Re: Fw: [syslog-ng] Error encountered while building syslog-ng-2.1.3 on RHEL3 add -lnsl to the end of your linking command. On Mon, 2009-01-12 at 19:37 +0530, Chanchal Verma wrote: > > Hi Bazsi, > > Could you please help me is resolving error message coming to > syslog-ng, when I am trying to build is with --enable-static-linking? > > Please refer to mail tail for detail. > > > With Regards, > Chanchal Verma > > ----- Forwarded by Chanchal Verma/GIS/CSC on 01/12/2009 07:35 PM > ----- > Chanchal Verma/GIS/CSC@CSC > Sent by: > syslog-ng-bounces(a)lists.balabit.hu > > 01/09/2009 07:25 PM > Please respond to > "Syslog-ng users' and developers' > mailing list" > <syslog-ng(a)lists.balabit.hu> > > > > > To > Kostyantyn > Gushtin > <kgushtin(a)n-ix.com.ua>, syslog-ng(a)lists.balabit.hu > cc > > Subject > Re: [syslog-ng] > Error encountered > while building > syslog-ng-2.1.3 > on > RHEL3 > > > > > > > > > > Hi Kostyantyn, > > I have checked that all required static libraries (.a) are available > at /usr/lib location. I also created linked at /usr/local/lib for all > requried libraries. It is giving same error messages. > Following libraries are checked available at /usr/lib: > > libnsl.a > librt.a > libfl.a > libevtlog.a (Created linked > from /opt/soe/local/syslog-ng-2.1.3/eventlog/lib/libevtlog.a) > libnet.a > libwrap.a > > Here below is error messages: > > ----------------------------------------------------------- > Here below is error messge: > > Compilation option: > [champ@ankit-lin syslog-ng-2.1.3]# ./configure > --enable-static-linking --prefix=/opt/soe/local/syslog-ng-2.1.3 > > Make works fine uptill (Pasted last few lines): > .. > .. > if gcc -DHAVE_CONFIG_H -I. -I. -I.. -I/usr/local/include/glib-2.0 > -I/usr/local/lib/glib-2.0/include -I/opt/soe/local/ > syslog-ng-2.1.3/eventlog/include/eventlog -D_BSD_SOURCE > -D__BSD_SOURCE -D__FAVOR_BSD -DHAVE_NET_ETHERNET_H -D_GNU_SOUR > CE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g -O2 -Wall -MT > globals.o -MD -MP -MF ".deps/globals.Tpo" \ > -c -o globals.o `test -f 'globals.c' || echo './'`globals.c; \ > then mv -f ".deps/globals.Tpo" ".deps/globals.Po"; \ > else rm -f ".deps/globals.Tpo"; exit 1; \ > fi > rm -f libsyslog-ng.a > ar cru libsyslog-ng.a misc.o utils.o messages.o syslog-names.o cfg.o > cfg-lex.o cfg-grammar.o macros.o filter.o logmsg.o l > ogpipe.o logsource.o driver.o sgroup.o dgroup.o center.o templates.o > logreader.o logwriter.o affile.o afstreams.o afuser. > o afprog.o afsocket.o afunix.o afinet.o afinter.o children.o stats.o > gsockaddr.o fdwrite.o fdread.o memtrace.o dnscache.o > apphook.o serialize.o logqueue.o afsql.o alarms.o globals.o > ranlib libsyslog-ng.a > if gcc -DHAVE_CONFIG_H -I. -I. -I.. -I/usr/local/include/glib-2.0 > -I/usr/local/lib/glib-2.0/include -I/opt/soe/local/ > syslog-ng-2.1.3/eventlog/include/eventlog -D_BSD_SOURCE > -D__BSD_SOURCE -D__FAVOR_BSD -DHAVE_NET_ETHERNET_H -D_GNU_SOUR > CE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g -O2 -Wall -MT main.o > -MD -MP -MF ".deps/main.Tpo" \ > -c -o main.o `test -f 'main.c' || echo './'`main.c; \ > then mv -f ".deps/main.Tpo" ".deps/main.Po"; \ > else rm -f ".deps/main.Tpo"; exit 1; \ > fi > gcc -g -O2 -Wall -o syslog-ng main.o libsyslog-ng.a -static -lnsl > -lrt -lfl -L/usr/local/lib -lglib-2.0 -L/opt/soe > /local/syslog-ng-2.1.3/eventlog/lib -levtlog -lnet -lwrap > make[2]: Leaving directory `/home/soetest1/syslog-ng-2.1.3/src' > make[1]: Leaving directory `/home/soetest1/syslog-ng-2.1.3' > > Showing error message on standard output device: > > main.o(.text+0x5ec): In function `setup_creds': > /home/soetest1/syslog-ng-2.1.3/src/main.c:324: Using 'initgroups' in > statically linked applications requires at runtime t > he shared libraries from the glibc version used for linking > libsyslog-ng.a(misc.o)(.text+0x44d): In function `resolve_group': > /home/soetest1/syslog-ng-2.1.3/src/misc.c:283: Using 'getgrnam' in > statically linked applications requires at runtime the > shared libraries from the glibc version used for linking > /usr/lib/gcc-lib/i386-redhat-linux/3.2.3/../../../libwrap.a(options.o)(.text+0x53c): In function `group_option': > : Using 'endgrent' in statically linked applications requires at > runtime the shared libraries from the glibc version used > for linking > libsyslog-ng.a(misc.o)(.text+0x3ed): In function `resolve_user': > /home/soetest1/syslog-ng-2.1.3/src/misc.c:258: Using 'getpwnam' in > statically linked applications requires at runtime the > shared libraries from the glibc version used for linking > /usr/local/lib/libglib-2.0.a(gutils.o)(.text+0xbd8): In function > `g_get_any_init_do': > /home/soetest1/glib-2.16.0/glib/gutils.c:1646: Using 'getpwuid' in > statically linked applications requires at runtime the > shared libraries from the glibc version used for linking > /usr/local/lib/libglib-2.0.a(gutils.o)(.text > +0xbc8):/home/soetest1/glib-2.16.0/glib/gutils.c:1645: Using > 'setpwent' in st > atically linked applications requires at runtime the shared libraries > from the glibc version used for linking > /usr/local/lib/libglib-2.0.a(gutils.o)(.text > +0xbe3):/home/soetest1/glib-2.16.0/glib/gutils.c:1647: Using > 'endpwent' in st > atically linked applications requires at runtime the shared libraries > from the glibc version used for linking > /usr/local/lib/libglib-2.0.a(gutils.o)(.text > +0x9d1):/home/soetest1/glib-2.16.0/glib/gutils.c:1589: Using > 'getpwnam_r' in > statically linked applications requires at runtime the shared > libraries from the glibc version used for linking > /usr/local/lib/libglib-2.0.a(gutils.o)(.text > +0xa10):/home/soetest1/glib-2.16.0/glib/gutils.c:1595: Using > 'getpwuid_r' in > statically linked applications requires at runtime the shared > libraries from the glibc version used for linking > libsyslog-ng.a(afinet.o)(.text+0x163): In function > `afinet_resolve_name': > /home/soetest1/syslog-ng-2.1.3/src/afinet.c:103: Using 'getaddrinfo' > in statically linked applications requires at runtim > e the shared libraries from the glibc version used for linking > libsyslog-ng.a(misc.o)(.text+0x1d1): In function `resolve_hostname': > /home/soetest1/syslog-ng-2.1.3/src/misc.c:166: Using 'gethostbyaddr' > in statically linked applications requires at runtim > e the shared libraries from the glibc version used for linking > libsyslog-ng.a(misc.o)(.text+0xbd): In function `getlonghostname': > /home/soetest1/syslog-ng-2.1.3/src/misc.c:71: Using 'gethostbyname' in > statically linked applications requires at runtime > the shared libraries from the glibc version used for linking > /usr/lib/gcc-lib/i386-redhat-linux/3.2.3/../../../libnet.a(libnet_resolve.o)(.text+0x4dc): In function `libnet_name2addr6 > ': > : Using 'gethostbyname2' in statically linked applications requires at > runtime the shared libraries from the glibc versio > n used for linking > libsyslog-ng.a(afinet.o)(.text+0xb2): In function `afinet_set_port': > /home/soetest1/syslog-ng-2.1.3/src/afinet.c:58: Using 'getservbyname' > in statically linked applications requires at runti > me the shared libraries from the glibc version used for linking > /usr/lib/gcc-lib/i386-redhat-linux/3.2.3/../../../libwrap.a(hosts_access.o)(.text+0x862): In function `host_match': > : undefined reference to `yp_get_default_domain' > collect2: ld returned 1 exit status > make[2]: *** [syslog-ng] Error 1 > make[1]: *** [all-recursive] Error 1 > make: *** [all] Error 2 > ---------------------------------------------------- > > With Regards, > Chanchal Verma > > > Kostyantyn Gushtin > <kgushtin(a)n-ix.com.ua> > > 01/09/2009 06:08 PM > > > > To > Chanchal > Verma/GIS/CSC@CSC > cc > > Subject > Re: [syslog-ng] > Error encountered > while building > syslog-ng-2.1.3 > on RHEL3 > > > > > > > > > > > Chanchal Verma wrote: > > Hi Kostyantyn, > > Yes my compilation with --enable-dynamic-linking option works fine. I > am facing problem while doing static compilation. I have followed the > given steps but still getting error while doing 'make'. > I am not sure how to use 'glib' option for compilation, I have > libglib-2.0.a in '/usr/lib'. > Now you really have problems with glibc :) . glib is just abstraction > (on unix systems it use glibc). > gcc -g -O2 -Wall -o syslog-ng main.o libsyslog-ng.a -static -lnsl > -lrt -lfl -L/usr/local/lib -lglib-2.0 -L/opt/soe > /local/syslog-ng-2.1.3/eventlog/lib -levtlog -lnet -lwrap > make[2]: Leaving directory `/home/soetest1/syslog-ng-2.1.3/src' > make[1]: Leaving directory `/home/soetest1/syslog-ng-2.1.3' > > From this command line you can retrieve info which static libraries > you need (some of them are glibc libraries, some not ). > So you must have thous libraries : libnsl.a (I saw linker error > message that it can't find this library), librt.a, libfl.a, > libevtlog.a, libnet.a, libwrap.a . Maybe you need to install > glibc-devel package and/or others that contains such libraries. > > -- > Kostyantyn Gushtin > Software engineer > N-iX LLC > Solutions for Your success > http://www.n-ix.com________________________________________________________… > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > -- Bazsi

2 1

Re: [syslog-ng] Error encountered while building syslog-ng-2.1.3 on RHEL3
by Chanchal Verma 12 Jan '09

12 Jan '09

Hi Kostyantyn, I have checked that all required static libraries (.a) are available at /usr/lib location. I also created linked at /usr/local/lib for all requried libraries. It is giving same error messages. Following libraries are checked available at /usr/lib: libnsl.a librt.a libfl.a libevtlog.a (Created linked from /opt/soe/local/syslog-ng-2.1.3/eventlog/lib/libevtlog.a) libnet.a libwrap.a Here below is error messages: ----------------------------------------------------------- Here below is error messge: Compilation option: [champ@ankit-lin syslog-ng-2.1.3]# ./configure --enable-static-linking --prefix=/opt/soe/local/syslog-ng-2.1.3 Make works fine uptill (Pasted last few lines): .. .. if gcc -DHAVE_CONFIG_H -I. -I. -I.. -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/opt/soe/local/ syslog-ng-2.1.3/eventlog/include/eventlog -D_BSD_SOURCE -D__BSD_SOURCE -D__FAVOR_BSD -DHAVE_NET_ETHERNET_H -D_GNU_SOUR CE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g -O2 -Wall -MT globals.o -MD -MP -MF ".deps/globals.Tpo" \ -c -o globals.o `test -f 'globals.c' || echo './'`globals.c; \ then mv -f ".deps/globals.Tpo" ".deps/globals.Po"; \ else rm -f ".deps/globals.Tpo"; exit 1; \ fi rm -f libsyslog-ng.a ar cru libsyslog-ng.a misc.o utils.o messages.o syslog-names.o cfg.o cfg-lex.o cfg-grammar.o macros.o filter.o logmsg.o l ogpipe.o logsource.o driver.o sgroup.o dgroup.o center.o templates.o logreader.o logwriter.o affile.o afstreams.o afuser. o afprog.o afsocket.o afunix.o afinet.o afinter.o children.o stats.o gsockaddr.o fdwrite.o fdread.o memtrace.o dnscache.o apphook.o serialize.o logqueue.o afsql.o alarms.o globals.o ranlib libsyslog-ng.a if gcc -DHAVE_CONFIG_H -I. -I. -I.. -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/opt/soe/local/ syslog-ng-2.1.3/eventlog/include/eventlog -D_BSD_SOURCE -D__BSD_SOURCE -D__FAVOR_BSD -DHAVE_NET_ETHERNET_H -D_GNU_SOUR CE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g -O2 -Wall -MT main.o -MD -MP -MF ".deps/main.Tpo" \ -c -o main.o `test -f 'main.c' || echo './'`main.c; \ then mv -f ".deps/main.Tpo" ".deps/main.Po"; \ else rm -f ".deps/main.Tpo"; exit 1; \ fi gcc -g -O2 -Wall -o syslog-ng main.o libsyslog-ng.a -static -lnsl -lrt -lfl -L/usr/local/lib -lglib-2.0 -L/opt/soe /local/syslog-ng-2.1.3/eventlog/lib -levtlog -lnet -lwrap make[2]: Leaving directory `/home/soetest1/syslog-ng-2.1.3/src' make[1]: Leaving directory `/home/soetest1/syslog-ng-2.1.3' Showing error message on standard output device: main.o(.text+0x5ec): In function `setup_creds': /home/soetest1/syslog-ng-2.1.3/src/main.c:324: Using 'initgroups' in statically linked applications requires at runtime t he shared libraries from the glibc version used for linking libsyslog-ng.a(misc.o)(.text+0x44d): In function `resolve_group': /home/soetest1/syslog-ng-2.1.3/src/misc.c:283: Using 'getgrnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/lib/gcc-lib/i386-redhat-linux/3.2.3/../../../libwrap.a(options.o)(.text+0x53c): In function `group_option': : Using 'endgrent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking libsyslog-ng.a(misc.o)(.text+0x3ed): In function `resolve_user': /home/soetest1/syslog-ng-2.1.3/src/misc.c:258: Using 'getpwnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/local/lib/libglib-2.0.a(gutils.o)(.text+0xbd8): In function `g_get_any_init_do': /home/soetest1/glib-2.16.0/glib/gutils.c:1646: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/local/lib/libglib-2.0.a(gutils.o)(.text+0xbc8):/home/soetest1/glib-2.16.0/glib/gutils.c:1645: Using 'setpwent' in st atically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/local/lib/libglib-2.0.a(gutils.o)(.text+0xbe3):/home/soetest1/glib-2.16.0/glib/gutils.c:1647: Using 'endpwent' in st atically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/local/lib/libglib-2.0.a(gutils.o)(.text+0x9d1):/home/soetest1/glib-2.16.0/glib/gutils.c:1589: Using 'getpwnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/local/lib/libglib-2.0.a(gutils.o)(.text+0xa10):/home/soetest1/glib-2.16.0/glib/gutils.c:1595: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking libsyslog-ng.a(afinet.o)(.text+0x163): In function `afinet_resolve_name': /home/soetest1/syslog-ng-2.1.3/src/afinet.c:103: Using 'getaddrinfo' in statically linked applications requires at runtim e the shared libraries from the glibc version used for linking libsyslog-ng.a(misc.o)(.text+0x1d1): In function `resolve_hostname': /home/soetest1/syslog-ng-2.1.3/src/misc.c:166: Using 'gethostbyaddr' in statically linked applications requires at runtim e the shared libraries from the glibc version used for linking libsyslog-ng.a(misc.o)(.text+0xbd): In function `getlonghostname': /home/soetest1/syslog-ng-2.1.3/src/misc.c:71: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/lib/gcc-lib/i386-redhat-linux/3.2.3/../../../libnet.a(libnet_resolve.o)(.text+0x4dc): In function `libnet_name2addr6 ': : Using 'gethostbyname2' in statically linked applications requires at runtime the shared libraries from the glibc versio n used for linking libsyslog-ng.a(afinet.o)(.text+0xb2): In function `afinet_set_port': /home/soetest1/syslog-ng-2.1.3/src/afinet.c:58: Using 'getservbyname' in statically linked applications requires at runti me the shared libraries from the glibc version used for linking /usr/lib/gcc-lib/i386-redhat-linux/3.2.3/../../../libwrap.a(hosts_access.o)(.text+0x862): In function `host_match': : undefined reference to `yp_get_default_domain' collect2: ld returned 1 exit status make[2]: *** [syslog-ng] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 ---------------------------------------------------- With Regards, Chanchal Verma Kostyantyn Gushtin <kgushtin(a)n-ix.com.ua> 01/09/2009 06:08 PM To Chanchal Verma/GIS/CSC@CSC cc Subject Re: [syslog-ng] Error encountered while building syslog-ng-2.1.3 on RHEL3 Chanchal Verma wrote: Hi Kostyantyn, Yes my compilation with --enable-dynamic-linking option works fine. I am facing problem while doing static compilation. I have followed the given steps but still getting error while doing 'make'. I am not sure how to use 'glib' option for compilation, I have libglib-2.0.a in '/usr/lib'. Now you really have problems with glibc :) . glib is just abstraction (on unix systems it use glibc). gcc -g -O2 -Wall -o syslog-ng main.o libsyslog-ng.a -static -lnsl -lrt -lfl -L/usr/local/lib -lglib-2.0 -L/opt/soe /local/syslog-ng-2.1.3/eventlog/lib -levtlog -lnet -lwrap make[2]: Leaving directory `/home/soetest1/syslog-ng-2.1.3/src' make[1]: Leaving directory `/home/soetest1/syslog-ng-2.1.3' >From this command line you can retrieve info which static libraries you need (some of them are glibc libraries, some not ). So you must have thous libraries : libnsl.a (I saw linker error message that it can't find this library), librt.a, libfl.a, libevtlog.a, libnet.a, libwrap.a . Maybe you need to install glibc-devel package and/or others that contains such libraries. -- Kostyantyn Gushtin Software engineer Solutions for Your success http://www.n-ix.com

2 1

syslog-ng 3.0.1 has been released
by devel＠balabit.hu 12 Jan '09

12 Jan '09

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.0.1 SUMMARY : new stable release DATE : Jan 12, 2009 ------------------------------------------------------------------------------ DESCRIPTION: A new major version of syslog-ng Open Source Edition 3.0 has been released. The news and highlights of this version are summarized below. CHANGES: 3.0.1 Wed, 24 Dec 2008 12:16:08 +0100 NEWS: Communication: * Version 3.0 of syslog-ng supports the new syslog protocol standards developed by the Internet Engineering Task Force (IETF). The details of these protocols can be found at http://www.ietf.org/internet-drafts/draft-ietf-syslog-protocol-23.txt and http://www.ietf.org/internet-drafts/draft-ietf-syslog-transport-tls-11.txt. * The capability to encrypt log connections using TLS has been added. Configuration: * Log statements can be embedded into each other, making it possible to design complex log paths. That way for example the results of a filter can be sent to multiple destinations, or it can be sent to a destination unchanged, and partly rewritten for the other destination. Log files: * The encoding of source files can be set for proper character conversion (internally syslog-ng represents every message in UTF-8). Log messages: * The syslog-ng application assigns a unique message identification number to every log message, making it easy to detect if any messages are lost. * The syslog-ng application can read, process, and rewrite structured messages (e.g., Apache webserver logs) using templates and regular expressions. Both messages with fixed field sizes and fields separated with delimiters (e.g., comma-separated values) are supported. DOWNLOAD: This is a source code only release, no binary packages are available at this time. Those will be provided at a later date. The documentation of syslog-ng OSE 3.0, including the installation and user guides is available at: http://www.balabit.com/support/documentation/?product=syslog-ng Best Regards, BalaBit IT Security DOWNLOAD: You can download the latest source and binary packages of syslog-ng from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system…

1 0

Re: [syslog-ng] Host/IP Macros in relay chains
by Paul Robert Marino 10 Jan '09

10 Jan '09

I have one thing to add to this. I know this probably goes without saying but the reverse lookup should not be reliant on DNS. It should use the the systems native name resolution. I've often seen application programed to use only DNS for reverse resolution and in many of the secure environments I've worked in hosts files are used on loggers (also bastions, and firewalls) and DNS support is removed via the nsswhich.conf to make them impervious to DNS spoofing. DNS reliance is often a deal breaker on these hosts. -----Original Message----- From: Balazs Scheidler <bazsi(a)balabit.hu> Subj: Re: [syslog-ng] Host/IP Macros in relay chains Date: Fri Jan 9, 2009 1:09 pm Size: 1K To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> On Fri, 2009-01-09 at 10:46 +0000, Pennington, Philip wrote: > Sandor, > > > Thanks for your comments and useful suggestions. > > The requirement is somewhat complicated in that at a point along the > chain, I need to have the originating hostname for host filtering > purposes, whereas at the end of the chain, I need syslog-ng to present > the IP. That's why I began talking about reverse name resolution on the > last relay. well, with syslog-ng 3.0 and parse/rewrite you could probably encode all the needed information into the message payload and the change it back at the endpoints. see my blog about parse/rewrite capabilities: http://bazsi.blogs.balabit.com/2008/10/syslog-ng-message-parsing.html or the what's new document: http://www.balabit.com/dl/guides/syslog-ng-v3.0-guide-whatsnew-en.pdf the open source version of syslog-ng 3.0 is already released, although the official announcement is still due. -- Bazsi ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

2 1

Host/IP Macros in relay chains
by Pennington, Philip 09 Jan '09

09 Jan '09

Question on Host/IP Macros in relay chains. Is there a way to present the original sender IP on a final relay in a chain of several relays ? With hostname, the FULLHOST and HOST macros are capable of doing this (with tuning of keep_hostname() and chain_hostnames() ) Their corresponding FULLHOST_FROM and HOST_FROM marcos exhibit the same behaviours as the SOURCEIP macro in the sense that they only provide the NAME/IP of the previous relay . I have read Michael Gehrmann's post " https://lists.balabit.hu/pipermail/syslog-ng/2004-November/006695.html " which discusses the compile time option "--enable-spoof-source" feature which spoofs the sourceip using UDP but this doesn't really help as I need to use tcp and retain the relay's source (firewalls in the chain). Presumably this setting would also be required on all members of the relay chain. I guess I am really looking for an alternative way to achieve similar effect to Michael's solution using the first (original) entry in the chain (rather than spoofing the last entry in the chain at all points through the chain) however to force an IP from the original sender's entry in the chain (Reverse/PTR DNS resolution perhaps ??). What HOST_FROM is-to HOST, SOURCEIP is-to what I want. Does anyone know whether this is currently possible (or pipeline development) ? Maybe in the premium version (although I can't see it documented there either)??? Thanks, Philip Excerpts from the syslog-ng-v2.0 admin guide (section 9.5 Macros) FULLHOST: The full FQDN of the host name chain (without trimming chained hosts), including the domain name. To use this macro, make sure that the keep_hostname() option is enabled. FULLHOST_FROM: FQDN of the host that sent the message to syslog-ng as resolved by syslogng using DNS. If the message traverses several hosts, this is the last host in the chain. To use this macro, make sure that the keep_hostname() option is enabled. HOST: The name of the source host where the message originates from. If the message traverses several hosts and the chain_hostnames() option is on, the first host in the chain is used. To use this macro, make sure that the keep_hostname() option is enabled. HOST_FROM: Name of the host that sent the message to syslog-ng, as resolved by syslogng using DNS. If the message traverses several hosts, this is the last host in the chain. To use this macro, make sure that the keep_hostname() option is enabled. SOURCEIP: IP address of the host that sent the message to syslog-ng. (I.e. the IP address of the host in the FULLHOST_FROM macro.) Please note that when a message traverses several relays, this macro contains the IP of the last relay. ============================================================================== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html ==============================================================================

3 3

syslog-ng loses hostname information on some syslog logs sent via UDP
by D S, Manu (STSD) 09 Jan '09

09 Jan '09

Hi, I am running syslog-ng on a HP-UX server listening on UDP port 514. It is receiving logs from syslogd running on another server. For some messages syslog-ng does not log the hostname information found in the UDP packet. Rather, it mistakes some data in UDP as the hostname information. Here is the complete information. syslog-ng 2.0.9 on HP-UX. Syslogd on node01 sends logs to syslog-ng on node02. The logs in node02 are, Jan 9 11:55:11 node01 root: testing1 Jan 9 11:55:32 above message repeats 5 times Jan 9 11:55:32 node01 root: testing4 Notice that hostname is missing in the second message. tcpdump on UDP port 514 for the above logs 11:57:26.183996 00:30:6e:4b:26:37 (oui Unknown) > 00:30:6e:4a:32:44 (oui Unknown), ethertype IPv4 (0x0800), length 76: (tos 0x0, ttl 64, id 39220, offset 0, flags [DF], proto UDP (17), length 62) node01.xxx.com.57403 > node02.xxx.com.syslog: [udp sum ok] SYSLOG, length: 34 Facility user (1), Severity notice (5) Msg: Jan 9 11:55:11 root: testing1 0x0000: 3c31 333e 4a61 6e20 2039 2031 313a 3535 0x0010: 3a31 3120 726f 6f74 3a20 7465 7374 696e 0x0020: 6731 0x0000: 0030 6e4a 3244 0030 6e4b 2637 0800 4500 .0nJ2D.0nK&7..E. 0x0010: 003e 9934 4000 4011 3c2c 10b5 a1f0 10b5 .>.4@.@.<,...... 0x0020: a1f4 e03b 0202 002a a973 3c31 333e 4a61 ...;...*.s<13>Ja 0x0030: 6e20 2039 2031 313a 3535 3a31 3120 726f n..9.11:55:11.ro 0x0040: 6f74 3a20 7465 7374 696e 6731 ot:.testing1 11:57:26.185727 00:30:6e:4b:26:37 (oui Unknown) > 00:30:6e:4a:32:44 (oui Unknown), ethertype IPv4 (0x0800), length 92: (tos 0x0, ttl 64, id 39221, offset 0, flags [DF], proto UDP (17), length 78) node01.xxx.com.57403 > node02.xxx.com.syslog: [udp sum ok] SYSLOG, length: 50 Facility user (1), Severity notice (5) Msg: Jan 9 11:55:32 above message repeats 5 times 0x0000: 3c31 333e 4a61 6e20 2039 2031 313a 3535 0x0010: 3a33 3220 2061 626f 7665 206d 6573 7361 0x0020: 6765 2072 6570 6561 7473 2035 2074 696d 0x0030: 6573 0x0000: 0030 6e4a 3244 0030 6e4b 2637 0800 4500 .0nJ2D.0nK&7..E. 0x0010: 004e 9935 4000 4011 3c1b 10b5 a1f0 10b5 .N.5@.@.<....... 0x0020: a1f4 e03b 0202 003a b3b0 3c31 333e 4a61 ...;...:..<13>Ja 0x0030: 6e20 2039 2031 313a 3535 3a33 3220 2061 n..9.11:55:32..a 0x0040: 626f 7665 206d 6573 7361 6765 2072 6570 bove.message.rep 0x0050: 6561 7473 2035 2074 696d 6573 eats.5.times 11:57:26.186879 00:30:6e:4b:26:37 (oui Unknown) > 00:30:6e:4a:32:44 (oui Unknown), ethertype IPv4 (0x0800), length 76: (tos 0x0, ttl 64, id 39222, offset 0, flags [DF], proto UDP (17), length 62) node01.xxx.com.57403 > node02.xxx.com.syslog: [udp sum ok] SYSLOG, length: 34 Facility user (1), Severity notice (5) Msg: Jan 9 11:55:32 root: testing4 0x0000: 3c31 333e 4a61 6e20 2039 2031 313a 3535 0x0010: 3a33 3220 726f 6f74 3a20 7465 7374 696e 0x0020: 6734 0x0000: 0030 6e4a 3244 0030 6e4b 2637 0800 4500 .0nJ2D.0nK&7..E. 0x0010: 003e 9936 4000 4011 3c2a 10b5 a1f0 10b5 .>.6@.@.<*...... 0x0020: a1f4 e03b 0202 002a a86e 3c31 333e 4a61 ...;...*.n<13>Ja 0x0030: 6e20 2039 2031 313a 3535 3a33 3220 726f n..9.11:55:32.ro 0x0040: 6f74 3a20 7465 7374 696e 6734 ot:.testing4 When I change keep_hostname(yes) to keep_hostname(no) and add the chain_hostnames(yes) option I get the following logged. Jan 9 11:55:22 node01/node01 root: testing3 Jan 9 11:57:13 above/node01 message repeats 6 times Jan 9 11:57:13 node01/node01 root: testing8 I would say, syslog-ng is confusing 'above' as the hostname before rewriting hostname. The tcpdump for these logs are 11:59:06.362374 00:30:6e:4b:26:37 (oui Unknown) > 00:30:6e:4a:32:44 (oui Unknown), ethertype IPv4 (0x0800), length 76: (tos 0x0, ttl 64, id 39223, offset 0, flags [DF], proto UDP (17), length 62) node01.xxx.com.57403 > node02.xxx.com.syslog: [udp sum ok] SYSLOG, length: 34 Facility user (1), Severity notice (5) Msg: Jan 9 11:55:22 root: testing3 0x0000: 3c31 333e 4a61 6e20 2039 2031 313a 3535 0x0010: 3a32 3220 726f 6f74 3a20 7465 7374 696e 0x0020: 6733 0x0000: 0030 6e4a 3244 0030 6e4b 2637 0800 4500 .0nJ2D.0nK&7..E. 0x0010: 003e 9937 4000 4011 3c29 10b5 a1f0 10b5 .>.7@.@.<)...... 0x0020: a1f4 e03b 0202 002a a870 3c31 333e 4a61 ...;...*.p<13>Ja 0x0030: 6e20 2039 2031 313a 3535 3a32 3220 726f n..9.11:55:22.ro 0x0040: 6f74 3a20 7465 7374 696e 6733 ot:.testing3 11:59:06.364052 00:30:6e:4b:26:37 (oui Unknown) > 00:30:6e:4a:32:44 (oui Unknown), ethertype IPv4 (0x0800), length 92: (tos 0x0, ttl 64, id 39224, offset 0, flags [DF], proto UDP (17), length 78) node01.xxx.com.57403 > node02.xxx.com.syslog: [udp sum ok] SYSLOG, length: 50 Facility user (1), Severity notice (5) Msg: Jan 9 11:57:13 above message repeats 6 times 0x0000: 3c31 333e 4a61 6e20 2039 2031 313a 3537 0x0010: 3a31 3320 2061 626f 7665 206d 6573 7361 0x0020: 6765 2072 6570 6561 7473 2036 2074 696d 0x0030: 6573 0x0000: 0030 6e4a 3244 0030 6e4b 2637 0800 4500 .0nJ2D.0nK&7..E. 0x0010: 004e 9938 4000 4011 3c18 10b5 a1f0 10b5 .N.8@.@.<....... 0x0020: a1f4 e03b 0202 003a b2af 3c31 333e 4a61 ...;...:..<13>Ja 0x0030: 6e20 2039 2031 313a 3537 3a31 3320 2061 n..9.11:57:13..a 0x0040: 626f 7665 206d 6573 7361 6765 2072 6570 bove.message.rep 0x0050: 6561 7473 2036 2074 696d 6573 eats.6.times 11:59:06.364302 00:30:6e:4b:26:37 (oui Unknown) > 00:30:6e:4a:32:44 (oui Unknown), ethertype IPv4 (0x0800), length 76: (tos 0x0, ttl 64, id 39225, offset 0, flags [DF], proto UDP (17), length 62) node01.xxx.com.57403 > node02.xxx.com.syslog: [udp sum ok] SYSLOG, length: 34 Facility user (1), Severity notice (5) Msg: Jan 9 11:57:13 root: testing8 0x0000: 3c31 333e 4a61 6e20 2039 2031 313a 3537 0x0010: 3a31 3320 726f 6f74 3a20 7465 7374 696e 0x0020: 6738 0x0000: 0030 6e4a 3244 0030 6e4b 2637 0800 4500 .0nJ2D.0nK&7..E. 0x0010: 003e 9939 4000 4011 3c27 10b5 a1f0 10b5 .>.9@.@.<'...... 0x0020: a1f4 e03b 0202 002a a76a 3c31 333e 4a61 ...;...*.j<13>Ja 0x0030: 6e20 2039 2031 313a 3537 3a31 3320 726f n..9.11:57:13.ro 0x0040: 6f74 3a20 7465 7374 696e 6738 ot:.testing8 Is this a bug on how syslogd sends the message or is it a syslog-ng logging problem? Thanks, Manu P.S: Apologies for the long mail

3 2