December 2002 - syslog-ng - lists.balabit.hu

Urgent: Mangled messages in pipe() destinations in version 1.5.19
by Andreas Schulze 20 Mar '03

20 Mar '03

Hello Balazs, in version 1.5.19 (and in other versions between >1.5.7 and <=1.5.19, too) we observing obscure message mangling in pipe() destinations. Imagine the config below and the scenario: syslog-ng --write(2)--> pipe() --read(2)--> application All four log paths below, writes lines to the pipe '/tmp/pipe'. With 1.5.7 we read from the pipe exactly that, what syslog-ng (using the config below) writes to it. But with all newer versions (espec. 1.5.19) we read (with the same config): message_1 message_2 messagmessage_4 !!! ... message_n if we try to let syslog-ng write: message_1 message_2 message_3 message_4 ... message_n (The numbers and terms are only examples, because of the different sources and high message load.) Seems, that write(2) to fifo skips some message ends now. (espec. the terminating newline) > # example config: > # > source sys { sun-streams ("/dev/log" door("/var/run/syslog_door")); }; > source net { udp (ip(w.x.y.z) port(n)); }; > destination d_cisco { pipe("/tmp/pipe" template("@cisco: | $MSG\n")); }; > destination d_icmp { pipe("/tmp/pipe" template("@icmpd: | $MSG\n")); }; > log { source(net); filter(fa); destination(da); destination(d_cisco); }; > log { source(net); filter(fb); destination(db); destination(d_cisco); }; > log { source(sys); filter(f1); destination(d1); destination(d_icmp); }; > log { source(sys); filter(f2); destination(d2); destination(d_icmp); }; We are running a config similar to this above, with 1.5.7 since a lot of months very successfully in production ennvironment with very high message load. Since 1.5.1? the message mangling occurs. But now, some of your newer features becomes mission critical for us. Have you any idea, what happens? Versions <= 1.5.7 works well. Thanks a lot. -- Best regards --Andreas Schulze [phone: +49.5246.80.1275, fax: +49.5246.80.2275] | I believe, it was Dennis Ritchie who said something like: | "C is rarely the best language for a given task, | but it's often the second-best". | The implication being that: "[...]" | | sh# cat>$$.c<<EOT | main(l,a,n,d)char**a;{for(d=atoi(a[1])/10*80-atoi(a[2])/5-596;n="@NK\ | ACLCCGZAAQBEAADAFaISADJABBA^SNLGAQABDAXIMBAACTBATAHDBANZcEMMCCCCAAhE\ | IJFAEAAABAfHJETBdFLDAANEfDNBPHdBcBBBEA_AL H E L L O, W O R L D! " | [l++-3];)for(;n-->64;)putchar(!d+++33^l&1);} | EOT | gcc -o$$ $$.c;clear;./$$ 52 8;rm -f $$*

6 19

replacing part of prog name with hostname
by Nate Campi 06 Jan '03

06 Jan '03

I have syslog-ng 1.5.24 on solaris 8, reading from /etc/.syslog_door and I have a log entry like this: Dec 31 13:48:15 larry 6.0[8704]: [ID 702911 local0.warning] [0] Can't stat file in FlushFile [news/PointCast]: No such file or directory ...but the program name was sent from the app was: "ctlds 6.0[8704]:" and syslog-ng replaced the first part of the messed up program name with the host's name. In the next version, can syslog-ng "learn" that it don't get a hostname from solaris ever and that the entire text coming in is actually the log message? I'm losing information this way. I wonder how many other apps split up the program name and lose data - most people would never know as end users. -- Nate Campi http://www.campin.net An Emacs reference mug is what I want. It would hold ten gallons of coffee. -- Steve VanDevender And, no doubt, have a lid that could only be removed with an obscure finger combination requiring both hands. (Ctrl-Alt-Meta-X gimme-the-damn-coffee) -- William Beegle

5 14

Over zealous syslog-ng problem
by Aaron Jackson 06 Jan '03

06 Jan '03

My Setup: I have syslog-ng running on several Solaris 8 machines. Each machine writes log messages to their local disks and also forwards the messages to a central log server via a UPD connection, also a Solaris 8 machine. The central log server stores everything into a mysql database via a perl script I wrote. My Problem: I am running the UNIX version of Cisco Secure on one of the Solaris boxes. A couple of times it has blown up. When this happens, it generates millions of log messages in a very short period. The problem is that syslog-ng logs most of these messages (I also get the mangled message problem during these heavy loads). The most recent episode generated 1,930,974 messages that made it into the mysql database and 49,573 mangled messages on the central log server, but 16,040,886 messages were written to disk on the local machine (see below). My Questions: Is there any way to throttle syslog-ng, or make syslog-ng not accept all log messages when an app goes crazy? I want to log some of these messages, so I know when to restart the service, but I don't want all 16 million. Also, it seems that around 15 million log messages didn't make it to my central server. Where were they lost? Is this a problem with the UDP transport? Aaron # cat local0.log | grep -c 'ERROR - error on accept' 16040886 jackson@auth:/tmp {5} cat sql_errors | grep -c 'INSERT INTO' 49573 mysql> delete from logs where host='acs' and facility='local0' and priority='err' and msg like '%ERROR - error on accept%'; Query OK, 1943387 rows affected (1 hour 40.16 sec)

5 10

(no subject)
by Noam Meltzer 31 Dec '02

31 Dec '02

Hi! I am using Linux running Syslog-ng as my logserver, while the logs for each host i have are sent into different directory (and in the directory, i create files for each facility.priority). My clients platforms (the servers being logged) are: 1. Linux (runnig classic syslogd... can't remember which) 2. HPUX11i (running the native syslogd) 3. Solaris7&8 The mechanism works flowlessly, except for solaris. What happens? If I have a one line log, it works just fine. Syslog-ng identifies the host it comes from, and divide the data to files according to the rules I mentioned above. When I have a longer logs which takes two lines or more, starting from the 2nd line, syslog-ng doesn't know the the message was originated from, and creates for funny like: "?DLT" "?corrupt" "?See", etc. (The ? is actually 1 character or more, which the terminal can't recognize) The word you see, is actually the first word of the actual message. If I look at the operation of the native on solaris, local messages are logged just fine. I did a little experiment and tried sending logs from one native solaris syslog to another. It worked just fine (except the fact i can't into hostnames...) I looked at the big log "/var/adm/messages" (of two hosts) and saw that the "bad" logs, were logged by the native logger just fine, with stating the originating host as it should be in the beginning of the line. I tried simulating this using the command "logger" with no sucess. I suspect that sends the data of each packet, not to lines as syslog-ng expects and this cause its mechanism to fail recognizing the originating host succesfully. Did any1 encountered this problem? How did you solve it? Noam tsnoam(a)excite.com _______________________________________________ Join Excite! - http://www.excite.com The most personalized portal on the Web!

4 3

Re: Bad hostname from solaris (added a subject now)
by Noam Meltzer 31 Dec '02

31 Dec '02

Hi! It is possible to use syslog-ng on solaris, but not preferable. I don't like the idea of replacing something core from the system. I'm trying to find a solution for the solaris syslogd, not to replace it with a different product. As I said: When its one native solaris syslog sending to another, it works flowlessly. Noam >Can you replace the Solaris syslog with syslog-ng? >Aaron >On Monday, December 30, 2002, at 05:32 PM, Noam Meltzer wrote: >>Hi! >>I am using Linux running Syslog-ng as my logserver, while the logs for >>each host i have are sent into different directory (and in the directory, i create files for each facility.priority). >>My clients platforms (the servers being logged) are: >>1. Linux (runnig classic syslogd... can't remember which) >>2. HPUX11i (running the native syslogd) >>3. Solaris7&8 >>The mechanism works flowlessly, except for solaris. >>What happens? If I have a one line log, it works just fine. Syslog-ng >>identifies the host it comes from, and divide the data to files according to the rules I mentioned above. >>When I have a longer logs which takes two lines or more, starting from >>the 2nd line, syslog-ng doesn't know the the message was originated from, and creates for funny like: >>"?DLT" "?corrupt" "?See", etc. >>(The ? is actually 1 character or more, which the terminal can't recognize) >>The word you see, is actually the first word of the actual message. >>If I look at the operation of the native on solaris, local messages are logged just fine. >>I did a little experiment and tried sending logs from one native solaris syslog to another. It worked just fine (except the fact i can't into hostnames...) >>I looked at the big log "/var/adm/messages" (of two hosts) and saw that the "bad" logs, were logged by the native logger just fine, with stating the originating host as it should be in the beginning of the line. >>I tried simulating this using the command "logger" with no sucess. >>I suspect that sends the data of each packet, not to lines as syslog-ng expects and this cause its mechanism to fail recognizing the originating host succesfully. >>Did any1 encountered this problem? How did you solve it? >>Noam >>tsnoam(a)excite.com _______________________________________________ Join Excite! - http://www.excite.com The most personalized portal on the Web!

1 0

syslog-ng doesn't work with minilogd
by Daniel Tschan 30 Dec '02

30 Dec '02

Hi I installed syslog-ng 1.5.24 on Red Hat Linux 7.3 and 8.0 and discovered that it doesn't work together with minilogd. For a description of minilogd please see: <http://www.cygnus.com/support/wpapers/redhat/initscripts/what.html>. Minilogd terminates if syslog-ng is started like expected, but the messages buffered by minilogd are lost. Any idea what's the cause and how to fix it? Daniel

3 2

Syslog and Checkpoint-NG
by Jason Howell 29 Dec '02

29 Dec '02

Has anyone heard of any problems getting Checkpoint-NG to syslog to a remote server? I am running CP-NG on RH7.3 and I can get it to syslog local but it won't send the logs to a remote server. J

1 0

Can Not Compile on Solaris 2.8
by Stephen Bargon 27 Dec '02

27 Dec '02

I've looked to Google and found references to the problem. I'm afraid I'm more a router / network person then Unix programmer and need more information. Where do I make the link to yacc and put the -ll or -ly switches? -----Original Message----- From: Aaron Jackson [mailto:Aaron.Jackson@dc.gov] Sent: Thursday, December 26, 2002 4:30 PM To: syslog-ng(a)lists.balabit.hu Cc: sbargon(a)amdocs.com Subject: Re: [syslog-ng]Can Not Compile on Solaris 2.8 Search the mailing list archive and google. This problem has been thoroughly discussed and solved. Aaron On Thursday, December 26, 2002, at 05:09 PM, Stephen Bargon wrote: I'm recieving the following error(s) when trying run make on syslog-ng-1.5.23. Any suggestions or help would be appreciated. # make Making all in src make all-recursive Making all in . gcc -g -O2 -Wall -I/usr/local/include/libol -D_GNU_SOURCE -o syslog-ng main.o sources.o center.o filters.o destinations.o log.o cfgfile.o cfg-grammar.o cfg-lex.o affile.o afsocket.o afunix.o afinet.o afinter.o afuser.o afstreams.o afprogram.o afremctrl.o nscache.o utils.o syslog-names.o getopt.o getopt1.o -lpthread -ldoor /usr/local/lib/libol.a -lsocket -lnsl -lxnet Undefined first referenced symbol in file res_init main.o yywrap cfg-lex.o ld: fatal: Symbol referencing errors. No output written to syslog-ng collect2: ld returned 1 exit status *** Error code 1 make: Fatal error: Command failed for target `syslog-ng' Current working directory /usr/local/syslog-ng-1.5.23/src *** Error code 1 make: Fatal error: Command failed for target `all-recursive' Current working directory /usr/local/syslog-ng-1.5.23/src *** Error code 1 make: Fatal error: Command failed for target `all-recursive-am' Current working directory /usr/local/syslog-ng-1.5.23/src *** Error code 1 make: Fatal error: Command failed for target `all-recursive' # ------------------------------------------------------------------------------------- The information contained in this message is proprietary of Amdocs, protected from disclosure, and may be privileged. The information is intended to be conveyed only to the designated recipient(s) of the message. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, use, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by replying to the message and deleting it from your computer. Thank you. ------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------- The information contained in this message is proprietary of Amdocs, protected from disclosure, and may be privileged. The information is intended to be conveyed only to the designated recipient(s) of the message. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, use, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by replying to the message and deleting it from your computer. Thank you. -------------------------------------------------------------------------------------

2 1

Option to determine which IP syslog-ng listens on?
by Jason Howell 27 Dec '02

27 Dec '02

Is there an option to force syslog-ng to only listen on a certain IP address on a multihomed machine? Ja

2 1

Can Not Compile on Solaris 2.8
by Stephen Bargon 26 Dec '02

26 Dec '02

I'm recieving the following error(s) when trying run make on syslog-ng-1.5.23. Any suggestions or help would be appreciated. # make Making all in src make all-recursive Making all in . gcc -g -O2 -Wall -I/usr/local/include/libol -D_GNU_SOURCE -o syslog-ng main.o sources.o center.o filters.o destinations.o log.o cfgfile.o cfg-grammar.o cfg-lex.o affile.o afsocket.o afunix.o afinet.o afinter.o afuser.o afstreams.o afprogram.o afremctrl.o nscache.o utils.o syslog-names.o getopt.o getopt1.o -lpthread -ldoor /usr/local/lib/libol.a -lsocket -lnsl -lxnet Undefined first referenced symbol in file res_init main.o yywrap cfg-lex.o ld: fatal: Symbol referencing errors. No output written to syslog-ng collect2: ld returned 1 exit status *** Error code 1 make: Fatal error: Command failed for target `syslog-ng' Current working directory /usr/local/syslog-ng-1.5.23/src *** Error code 1 make: Fatal error: Command failed for target `all-recursive' Current working directory /usr/local/syslog-ng-1.5.23/src *** Error code 1 make: Fatal error: Command failed for target `all-recursive-am' Current working directory /usr/local/syslog-ng-1.5.23/src *** Error code 1 make: Fatal error: Command failed for target `all-recursive' # ------------------------------------------------------------------------------------- The information contained in this message is proprietary of Amdocs, protected from disclosure, and may be privileged. The information is intended to be conveyed only to the designated recipient(s) of the message. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, use, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by replying to the message and deleting it from your computer. Thank you. -------------------------------------------------------------------------------------

2 1