January 2009 - syslog-ng - lists.balabit.hu

Re: [syslog-ng] importing old syslog-ng messages
by Daniel L. Spells Sr. 08 Jan '09

08 Jan '09

Joe, I tried catting the whole file to a named pipe, and echoing one line at a time. Both methods do the same thing. Daniel On Mon, January 5, 2009 04:18, Fegan, Joe wrote: > Then I would guess that whatever method you are using to send the old log > files to syslog-ng is stripping the newline characters, making it look > like one gigantic message. > > -----Original Message----- > From: syslog-ng-bounces(a)lists.balabit.hu > [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Daniel L. Spells > Sr. > Sent: 29 December 2008 15:59 > To: Syslog-ng users' and developers' mailing list > Subject: Re: [syslog-ng] importing old syslog-ng messages > > > I have pushed that number to 90K, and get the same message, except the > "log_msg_size=XXXX" changes to the what ever value the XXXX is set to in > the log, and nothing is written the the database. > > > On Mon, December 29, 2008 05:36, Balazs Scheidler wrote: > >> On Tue, 2008-12-16 at 11:44 -0600, Daniel L. Spells Sr. wrote: >> >> >>> I am getting: >>> >>> >>> >>> "syslog-ng[18859]: Message length overflow, line is split, >>> log_msg_size=4096" >>> >>> When I try to cat the log files to the named pipe. It repeats until >>> it completes the lines in the log file. >>> >>> This is on a base install on a Suse box. >>> >>> >>> >>> >> >> the message above says that you have messages over 4096 bytes in >> length. increase log_msg_size(). >> >> -- >> Bazsi >> >> >> >> _______________________________________________________________________ >> __ >> _____ >> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >> Documentation: >> http://www.balabit.com/support/documentation/?product=syslog-ng >> FAQ: http://www.campin.net/syslog-ng/faq.html >> >> >> >> > > > _________________________________________________________________________ > _____ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > > > _________________________________________________________________________ > _____ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > > >

2 1

Syslog-NG Openssh RHEL 4
by Christopher Bland 08 Jan '09

08 Jan '09

Hi guys, Having a bit of trouble logging sshd messages to my host. Sshd logs fine using syslog but I get nothing using Syslog-NG. I tried changing the SyslogFacility to DAEMON in sshd_config and still nothing. I have the following lines in my config which should have caught something: options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_local { file ("/proc/kmsg" log_prefix("kernel: ")); unix-dgram("/dev/log"); # unix-stream("/dev/log"); udp(ip(0.0.0.0) port(514)); tcp(ip(0.0.0.0) port(514)); internal(); }; destination secure { file("/var/log/secure"); }; destination messages { file("/var/log/messages"); }; filter f_daemon { facility(daemon); }; filter f_authpriv { facility(auth, authpriv); }; log { source(s_local); filter(f_authpriv); destination(secure); }; log { source(s_local); filter(f_daemon); destination(messages); }; log { source(s_local); destination(loghost); }; My sshd config looks like this: Protocol 2 SyslogFacility AUTHPRIV PasswordAuthentication yes ChallengeResponseAuthentication no GSSAPIAuthentication yes GSSAPICleanupCredentials yes UsePAM yes AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT AcceptEnv LC_IDENTIFICATION LC_ALL Subsystem sftp /usr/libexec/openssh/sftp-server Any help is apprciated -Chris

2 1

Message length
by G. Rayot 08 Jan '09

08 Jan '09

Hi! Using logger command, I'm trying to send long messages to syslog-ng. I'm aware that messages cannot be too long (UDP limitation). The problem is that logger doesn't just cut my log, it also remove some part of the log replacing it with the syslog header. Needless to say, this information is crucial to me. Is there a way to change this or must I change my log generation creating shorter logs ? Thanks. G.

3 3

bug in http://www.balabit.com/downloads/files/syslog-ng/sources/3.0.1/source/syslog-ng_3.0.1.tar.gz
by Joseph Pietras 08 Jan '09

08 Jan '09

There is, as far as I know, only a 1 character bug! (and it is missing a "!") The file is src/misc.c $ diff -c misc.c misc.c.orig *** misc.c 2008-12-30 12:12:36.000000000 -0600 --- misc.c.orig 2008-12-30 12:24:00.000000000 -0600 *************** *** 274,280 **** struct passwd *pw; *uid = 0; ! if (!*user) return FALSE; pw = getpwnam(user); --- 274,280 ---- struct passwd *pw; *uid = 0; ! if (*user) return FALSE; pw = getpwnam(user); Hope this helps, hope it gets to someone who can fix it, thanks for syslog-ng. -joseph This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error please notify the originator of the message. This footer also confirms that this e-mail message has been scanned for the presence of computer viruses. Any views expressed in this message are those of the individual sender, except where the sender specifies and with authority, states them to be the views of Iowa Student Loan.

2 1

Error encountered while building syslog-ng-2.1.3 on RHEL3
by Chanchal Verma 08 Jan '09

08 Jan '09

Hi, I am facing problem while builing syslog-ng-2.1.3 on RHEL3 operating system. Configuration goes fine but it shows error while doing 'make'. Here is details of configuration option: champ@localhost syslog-ng-2.1.3]# ./configure --enable-static-linking --prefix=/opt/soe/local/syslog-ng-2.1.3 EVTLOG_CFLAGS="-I/opt/soe/local/syslog-ng-2.1.3/eventlog/include/eventlog" EVTLOG_LIBS="-L/opt/soe/local/syslog-ng-2.1.3/eventlog/lib" checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for gawk... gawk checking whether make sets $(MAKE)... yes . . . checking for EVTLOG... yes checking for LIBNET... yes checking whether to enable Sun STREAMS support... no checking whether to enable Sun door support... no checking whether to enable TCP wrapper support... yes checking whether to enable spoof source support... yes checking whether to enable IPv6 support... yes checking whether to enable SQL support... no checking for pkg-config... (cached) /usr/local/bin/pkg-config checking pkg-config is at least version 0.14... yes checking for GLIB - version >= 2.4.0... yes (version 2.16.0) checking for static GLib libraries... yes checking sanity checking Glib headers... yes configure: creating ./config.status config.status: creating dist.conf config.status: creating Makefile config.status: creating syslog-ng.spec.bb config.status: creating src/Makefile config.status: creating doc/Makefile config.status: creating doc/docvars.xml config.status: creating contrib/Makefile config.status: creating tests/Makefile config.status: creating solbuild/Makefile config.status: creating tgzbuild/Makefile config.status: creating tests/unit/Makefile config.status: creating tests/functional/Makefile config.status: creating tests/loggen/Makefile config.status: creating config.h config.status: config.h is unchanged config.status: executing depfiles commands . . **************************************************************** While doing 'make' following error encountered: champ@localhost syslog-ng-2.1.3]#make (I have pasted last few lines of error message) . . libsyslog-ng.a(afinet.o)(.text+0x78a):/home/soetest1/syslog-ng-2.1.3/src/afinet.c:342: undefined reference to `evt_tag_str' libsyslog-ng.a(afinet.o)(.text+0xae7): In function `afinet_dd_queue': /home/soetest1/syslog-ng-2.1.3/src/afinet.c:522: undefined reference to `evt_tag_str' libsyslog-ng.a(stats.o)(.text+0xd3): In function `stats_register_counter': /home/soetest1/syslog-ng-2.1.3/src/stats.c:97: undefined reference to `evt_tag_str' libsyslog-ng.a(stats.o)(.text+0x1aa):/home/soetest1/syslog-ng-2.1.3/src/stats.c:140: more undefined references to `evt_tag_str' follow libsyslog-ng.a(stats.o)(.text+0x3a8): In function `stats_generate_log': /home/soetest1/syslog-ng-2.1.3/src/stats.c:230: undefined reference to `evt_tag_printf' libsyslog-ng.a(stats.o)(.text+0x3b2):/home/soetest1/syslog-ng-2.1.3/src/stats.c:230: undefined reference to `evt_rec_add_tag' libsyslog-ng.a(fdwrite.o)(.text+0xad): In function `fd_write_write_method': /home/soetest1/syslog-ng-2.1.3/src/fdwrite.c:49: undefined reference to `evt_tag_int' libsyslog-ng.a(fdwrite.o)(.text+0xbe):/home/soetest1/syslog-ng-2.1.3/src/fdwrite.c:49: undefined reference to `evt_tag_int' libsyslog-ng.a(fdwrite.o)(.text+0x16e): In function `fd_write_free_method': /home/soetest1/syslog-ng-2.1.3/src/fdwrite.c:80: undefined reference to `evt_tag_int' libsyslog-ng.a(fdread.o)(.text+0x9d): In function `fd_do_read': /home/soetest1/syslog-ng-2.1.3/src/fdread.c:49: undefined reference to `evt_tag_int' libsyslog-ng.a(fdread.o)(.text+0xae):/home/soetest1/syslog-ng-2.1.3/src/fdread.c:49: undefined reference to `evt_tag_int' libsyslog-ng.a(fdread.o)(.text+0x1be):/home/soetest1/syslog-ng-2.1.3/src/fdread.c:103: more undefined references to `evt_tag_int' follow libsyslog-ng.a(dnscache.o)(.text+0x2db): In function `dns_cache_check_hosts': /home/soetest1/syslog-ng-2.1.3/src/dnscache.c:221: undefined reference to `evt_tag_errno' libsyslog-ng.a(dnscache.o)(.text+0x2f0):/home/soetest1/syslog-ng-2.1.3/src/dnscache.c:221: undefined reference to `evt_tag_str' libsyslog-ng.a(apphook.o)(.text+0x47): In function `register_application_hook': /home/soetest1/syslog-ng-2.1.3/src/apphook.c:38: undefined reference to `evt_tag_int' libsyslog-ng.a(apphook.o)(.text+0x5c):/home/soetest1/syslog-ng-2.1.3/src/apphook.c:38: undefined reference to `evt_tag_int' libsyslog-ng.a(apphook.o)(.text+0x114): In function `run_application_hook': /home/soetest1/syslog-ng-2.1.3/src/apphook.c:56: undefined reference to `evt_tag_int' libsyslog-ng.a(serialize.o)(.text+0x43): In function `serialize_archive_read_bytes': /home/soetest1/syslog-ng-2.1.3/src/serialize.c:33: undefined reference to `evt_tag_str' libsyslog-ng.a(serialize.o)(.text+0xcb): In function `serialize_archive_write_bytes': /home/soetest1/syslog-ng-2.1.3/src/serialize.c:49: undefined reference to `evt_tag_str' libsyslog-ng.a(logreader.o)(.text+0xce): In function `log_reader_fd_check': /home/soetest1/syslog-ng-2.1.3/src/logreader.c:104: undefined reference to `evt_tag_errno' libsyslog-ng.a(logreader.o)(.text+0x19a):/home/soetest1/syslog-ng-2.1.3/src/logreader.c:112: undefined reference to `evt_tag_errno' libsyslog-ng.a(logreader.o)(.text+0x30d): In function `log_reader_handle_line': /home/soetest1/syslog-ng-2.1.3/src/logreader.c:195: undefined reference to `evt_tag_printf' libsyslog-ng.a(logreader.o)(.text+0x6cf): In function `log_reader_fetch_log': /home/soetest1/syslog-ng-2.1.3/src/logreader.c:450: undefined reference to `evt_tag_int' libsyslog-ng.a(logreader.o)(.text+0x6e6):/home/soetest1/syslog-ng-2.1.3/src/logreader.c:450: undefined reference to `evt_tag_int' libsyslog-ng.a(logreader.o)(.text+0x6f7):/home/soetest1/syslog-ng-2.1.3/src/logreader.c:450: undefined reference to `evt_tag_int' libsyslog-ng.a(logreader.o)(.text+0x774):/home/soetest1/syslog-ng-2.1.3/src/logreader.c:440: undefined reference to `evt_tag_int' libsyslog-ng.a(logreader.o)(.text+0x7c2):/home/soetest1/syslog-ng-2.1.3/src/logreader.c:425: undefined reference to `evt_tag_errno' libsyslog-ng.a(logreader.o)(.text+0x7d3):/home/soetest1/syslog-ng-2.1.3/src/logreader.c:425: undefined reference to `evt_tag_int' libsyslog-ng.a(logreader.o)(.text+0x8a5): In function `log_reader_init': /home/soetest1/syslog-ng-2.1.3/src/logreader.c:479: undefined reference to `evt_tag_int' libsyslog-ng.a(logreader.o)(.text+0x8ba):/home/soetest1/syslog-ng-2.1.3/src/logreader.c:479: undefined reference to `evt_tag_int' /usr/lib/gcc-lib/i386-redhat-linux/3.2.3/../../../libwrap.a(hosts_access.o)(.text+0x862): In function `host_match': : undefined reference to `yp_get_default_domain' collect2: ld returned 1 exit status make[2]: *** [syslog-ng] Error 1 make[2]: Leaving directory `/home/soetest1/syslog-ng-2.1.3/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/soetest1/syslog-ng-2.1.3' make: *** [all] Error 2 I am building it on - RHEL 3.0 - Operating system. Any help will be highly appreciated. With Regards, Chanchal Verma

2 1

Extra characters at beginning of line
by Florian Hines 07 Jan '09

07 Jan '09

Hi Everyone, I'm running into an issue where syslog-ng is adding extra characters to beginning of every line. Specifically, "<134>" is getting inserted right before the time stamp: <134>Jan 7 13:06:17 host1 kernel: device eth0 entered promiscuous mode This syslog-ng server is sending traffic to a remote Splunk instance (using TCP, not UDP), at first I though it was Splunk adding the characters but when I did a tcpdump on syslog-ng's outbound connection I found that they where already present. In addition to sending this traffic to Splunk the syslog-ng instance also log's local to a file. The <134> doesn't show up in the local file. Anyone have any ideas where this is coming from ? Thanks! Florian

2 2

Issues with SYSLOG-NG 2.0.10 and relaying via a SSH tunnel
by Dean Thompson 07 Jan '09

07 Jan '09

Hi Everybody, I was wondering whether somebody might be able to help me with a problem which appears to have been discussed before but is still continuing to be a problem for an environment that I am working within. The issue relates to one host forwarding SYSLOG messages to another host via a SSH tunnel. Scenario: ========= <Device> --SYSLOG MESSAGE--> Machine A <--SSH Tunnel--> Machine B Items to note include Machine B setting up a reverse SSH tunnel with Machine A. Machine A has a local port of 5514 that it uses to forward syslog messages to which sents them to machine B. This connectivity is working without any problems. The issue that I am currently experiencing is on Machine B when looking at the logs delivered via Machine A. It would appear that Machine B thinks that all the logs are coming from the IP address of 127.0.0.1. I have checked the configuration of syslog-ng on machine A and machine B and they look as if they are configured correctly. Syslog-NG 2.0.10 on Machine A is configured as follows: ======================================================= options { log_fifo_size(1000); long_hostnames(off); use_dns(no); use_fqdn(no); keep_hostname(yes); check_hostname(yes); log_msg_size(16384); }; source s_network { udp(ip(0.0.0.0) port(514)); tcp(ip(0.0.0.0) port(514) max_connections(1000)); }; destination d_syslogmgr { tcp("127.0.0.1" port(5514)); }; [...] Syslog-NG 2.0.10 on Machine B is configured as follows: ======================================================= options { log_fifo_size(1000); long_hostnames(off); use_dns(no); use_fqdn(no); keep_hostname(yes); check_hostname(yes); log_msg_size(16384); }; [...] Output that is being observed: ============================== Jan 7 12:32:34 127.0.0.1 172.XX.YY.1 3317162: 3339552: Jan 7 12:37:09.608: %LINK-3-UPDOWN: Interface FastEthernet0/7, changed state to up Jan 7 12:32:34 127.0.0.1 172.XX.YY.1 3317163: 3339553: Jan 7 12:37:10.457: %LINK-3-UPDOWN: Interface FastEthernet0/7, changed state to down Jan 7 12:32:34 127.0.0.1 172.XX.YY.1 23:37:07 accept 172.XX.YY.242 > >eth-s2p2c3 src: 192.168.17.48; dst: 192.168.19.167; proto: tcp; rule: 22; product: VPN-1 & FireWall-1; service: 9999; s_port: 3310; Jan 7 12:32:34 127.0.0.1 172.XX.YY.1 23:37:07 accept 172.XX.YY.242 > >eth-s2p4c1 src: XXX.YYY.235.195; dst: 192.168.108.1; proto: udp; rule: 5; product: VPN-1 & FireWall-1; service: 53; s_port: 5707; Jan 7 12:32:34 127.0.0.1 172.XX.YY.1 23:37:07 accept 172.XX.YY.242 > >eth-s2p4c1 src: XXX.YYY.235.197; dst: 192.168.108.1; proto: udp; rule: 5; product: VPN-1 & FireWall-1; service: 53; s_port: 11639; Jan 7 12:32:34 127.0.0.1 172.XX.YY.1 23:37:08 accept 172.XX.YY.242 > >eth-s2p4c1 src: XXX.YYY.235.196; dst: 192.168.108.1; proto: udp; rule: 5; product: VPN-1 & FireWall-1; service: 53; s_port: 1325; Jan 7 12:32:34 127.0.0.1 172.XX.YY.1 8023816: Jan 7 23:37:09.028: %SYS-CFC8-2-MMAP: mmap failed for size 24576 bytes Caller PC 0x72B12BCC errno 12 : ios-base : (PID=20499, TID=23) : -Traceback=(c6lc2-sp-4-dso-b.so+0x1F9AD0) ([19:0]+0x204790) ([19:0]+0x20CEDC) ([16:-2]np.+0x84BD4) ([21:0]+0x855B8) ([9:-3]1-dso-b+0x1FE904) ([9:-3]0-dso-bn+0xD354C) ([20:0]+0xD2658) ([20:0]+0xD29D0) ([9:-3]2-dso-b+0x4E4684) ([19:0]+0x4E465C) Ideally, I would prefer that the 127.0.0.1 was actually the 172.XX.YY.1 IP address, but it appears that there is a re-write of IP addresses going on. I have placed a packet sniffer on the line and I can see that the packet which is actually sent to MACHINE B is correct (including the correct IP addresses). To me it would appear that the issue is in the processing of the message at MACHINE B, but the issue may lie elsewhere. Is there anything obvious with the configurations which would be contributing to this problem, or is my basic understanding of how SYSLOG-NG works when wanting to keep hostnames and IP addresses flawed ? Regards, Dean

3 2

Solaris Client Not sending to Central Log Host
by Zeek Anow 05 Jan '09

05 Jan '09

We are running Solaris 9 on various Sun servers. I need to be able to send logs from the clients to the Central loghost using UDP. Below is a partial Client config file. Note that I don't have a Filter for this. If that is the problem, what kind of filter would I need to send all logs? Any help or suggestions would be appreciated. # GLOBAL OPTIONS ########################################################### options { keep_hostname(yes); chain_hostnames(no); create_dirs(yes); use_time_recvd(yes); use_dns(no); use_fqdn(no); }; # Source source local { sun-streams("/dev/log" door("/etc/.syslog_door")); internal(); }; destination d_centralhost { udp("10.5.2.3") port(514); }; log { source(local); destination (d_centralhost); }; ################################################################## Thanks, Zeek

3 4

underscore issue
by Stefan Varga 05 Jan '09

05 Jan '09

Hi all, I'm using syslog-ng 2.0.9 as a central logging server with destination like this: destination r_messages { file ("/logpool/logs/$DAY$MONTH$YEAR/$HOST_$DAY$MONTH$YEAR$_$HOUR"); }; But I get this: #ls /logpool/logs/05012009 0501200910 HOST is not expanded and _ before $HOUR is missing . What is special about underscore ? While using destination r_messages { file ("/logpool/logs/$DAY$MONTH$YEAR/$HOST-$DAY$MONTH$YEAR$-$HOUR"); }; everything is fine: # ls /logpool/logs/05012009/ 10.96.47.233-05012009-10 I would prefer to use an underscore if I could. Thank you, Stefan

2 3

importing old syslog-ng messages
by Daniel L. Spells Sr. 05 Jan '09

05 Jan '09

Sirs, Is there a way to import /var/log/messages, created by syslog-ng, into syslog-ng to be placed into a database? Daniel

5 11