August 2002 - syslog-ng - lists.balabit.hu

PID of programs
by harald schmidt 20 Aug '02

20 Aug '02

hello, I'm using syslog-ng to let my loghosts log to MySQL-databases as described under www.vermeer.org/syslog. Is it possible to filter the PID of a program to write it to an extra column of the MySQL table? This would be very useful to filter for example the sendmail-process of one mail. There are ~7 entries from sendmail for one mail, because of the virus scanner. Would it furthermore be possible to disperse the message in "to=.." and "from=.." to let that fields go to extra columns, too. thanks Harald -- Neu: Fakten testen jetzt noch günstiger 35% billiger + ein Geschenk Ihrer Wahl!! http://bonus.gmx.net/?gid=46842444&prid=687

1 0

output format to remote syslog duplicates info
by Clemson, Chris 20 Aug '02

20 Aug '02

Hello! I've been playing with syslog-ng, which is great, but I have a question. I am logging locally, and forwarding on entries to a remote syslogd (SL4NT). both work fine, and the entries in the /var/log/ syslog files are fine. However, on the SL4NT machine, the message entries include the date and machine name in the message log, even though it is included as fields in SL4NT. This behaviour doesn't happen when normal syslogds forward to my SL4NT server. Is there a way of turning this off by default? I'd prefer not to have to write templates for each rule I want sending to a remote syslogd. does anyone know if there is an option to turn this off? thanks! chris -- The content of this e-mail is confidential, may contain privileged material and is intended solely for the recipient(s) named above. If you receive this in error, please notify Software AG immediately and delete this e-mail. Software AG (UK) Limited Registered in England & Wales 1310740 Registered Office: Hudson House, Hudson Way, Pride Park, Derby DE24 8HS

1 0

[bug] Output templates.
by Sergey Y. Afonin 19 Aug '02

19 Aug '02

Hello. I think what I find one small bug in templates. The messages in log file have "\" before some symbols (",' etc). For example, log file definition: destination d_lan_by_host { file("/var/log/lan/$HOST.log" template("$DATE $MESSAGE\n")); }; The log file line is: Aug 18 12:28:53 At 07:28:50, Facility \"Auth Facility\", Level \"VERBOSE\":: A call, call id = 34996311, has arrived on interface slot:3/mod:23 The normal line is: Aug 18 12:28:53 At 07:28:50, Facility "Auth Facility", Level "VERBOSE":: A call, call id = 34996311, has arrived on interface slot:3/mod:23 Is it for normal file name generation, isn't it ? But it's not very suitable for log file... The version: 1.5.19 -- Regards, Sergey Afonin asy(a)kraft-s.ru

2 2

example config file
by Joerg Sommer 18 Aug '02

18 Aug '02

Hi, 2 changes on the sample config file for syslog-ng, I would prefer: * set the fileoptions (owner(), group(), perm()) in the global section * the file for logs of the facility user should have permission 644, that users can see, what they have done. Joerg.

1 0

logging into database
by hicham rahimi 16 Aug '02

16 Aug '02

hi, i want to setup logging syslog-ng'messages into a mysql database ,in order to make the messages organization more powerful. i have tested an exemple given in www.vermeer.com but it doesn't work. if any one have tested it , tell me please if it contain a bug or not thanks hicham; here the exemple: Centralized syslog-ng to mysql database by Michael Earls To established a centralized location for syslog-ng collection in order to facilitate: Immediate log reporting (both real time and summary reporting) Long term log storage (for archival and for possible later analysis). Tools used: syslog-ng -- template() destinations are only supported by the 1.5.x releases. mysql My Own PhP Files phpmyedit phpmyadmin Syslog-ng/Pipe Scripts MySql Replication Demo https://daffy.chmcc.org/syslog// This site is still in testing phase Demo-files The files I used for the Demo -- NEW DEMO FILE -- To setup syslog-ng to log to a mysql database. This assumes that you have installed and setup syslog-ng and mysql. Edit the syslog-ng.conf file This tells syslog-ng to pipe to a fifo template Add the following lines -- ## Log syslog-ng to mysql database ## destination d_mysql { pipe("/tmp/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; log { source(net); destination(d_mysql); }; Then comment out this line -- # This is the default behavior of sysklogd package # Logs may come from unix stream, but not from another machine. # #source src { unix-dgram("/dev/log"); internal(); }; Then uncomment out this line -- # If you wish to get logs from remote machine you should uncomment # this and comment the above source line. # source src { unix-dgram("/etc/log/log"); internal(); }; source net { udp(); }; Create the database for syslog-ng -- Create a file called syslog.sql and paste this below, this will create the database " syslog" and table " logs" in mysql. # # Table structure for table `logs` # CREATE DATABASE syslog; USE syslog; CREATE TABLE logs ( host varchar(32) default NULL, facility varchar(10) default NULL, priority varchar(10) default NULL, level varchar(10) default NULL, tag varchar(10) default NULL, date date default NULL, time time default NULL, program varchar(15) default NULL, msg text, seq int(10) unsigned NOT NULL auto_increment, PRIMARY KEY (seq), KEY host (host), KEY seq (seq), KEY program (program), KEY time (time), KEY date (date), KEY priority (priority), KEY facility (facility) ) TYPE=MyISAM; Run this command mysql -u root -p < syslog.sql This will install the database into mysql. Create a fifo pipe file -- This is the file that syslog-ng will store records before writing to the database. mkfifo /tmp/mysql.pipe You need to restart syslog-ng -- /etc/init.d/syslog-ng stop # Stop syslog-ng /etc/ini.d/syslog-ng start # Start syslog-ng Run this command to pipe the file mysql.pipe to mysql database You need to create a script that will check to make sure this command is running and restart if stopped. Syslog-ng/Pipe scripts When this file is started it will hang, You need to create a script and have it run on startup. mysql -u root --password=passwd syslog < /tmp/mysql.pipe -------------------------------------------------------------------------------- This is a mini how-to and is in the second draft::Thanks again to everyone.. Michael Any questions just email me --Michael(a)michaelearls.com-- Created Jan 18, 2002 Last Updated May 18, 2002 ___________________________________________________________ Do You Yahoo!? -- Une adresse @yahoo.fr gratuite et en français ! Yahoo! Mail : http://fr.mail.yahoo.com

2 1

Adding log format parameters
by 조은진 16 Aug '02

16 Aug '02

1 0

RE: [syslog-ng]Problems compiling libol / syslog-ng
by Camacho, Christian 14 Aug '02

14 Aug '02

For the record, I deleted the part of the script that gives the error and resolved (workaround) the problem. Maybe there's a bug in the config file... :( Thanks, Christian -----Mensaje original----- De: Camacho, Christian Enviado el: Miércoles, 14 de Agosto de 2002 14:50 Para: syslog-ng(a)lists.balabit.hu Asunto: RE: [syslog-ng]Problems compiling libol / syslog-ng That's a weird error, the time was a couple of minutes ahead, I changed it now, but that's not the problem.... Anyone has compiled it on Redhat 7.3 before?? Christian -----Mensaje original----- De: Dustin Trammell [mailto:DTrammell@PENSON.COM] Enviado el: Miércoles, 14 de Agosto de 2002 13:07 Para: 'syslog-ng(a)lists.balabit.hu' Asunto: RE: [syslog-ng]Problems compiling libol / syslog-ng Check the time on your system, it's probably wrong. Usually configure won't continue if it finds files that are newer than current date/time, or it creates a file that is older than current date/time. --- Dustin D. Trammell Information Security Specialist Penson Financial Services, Inc. -----Original Message----- From: Camacho, Christian [mailto:CCamacho@golder.cl] Sent: Wednesday, August 14, 2002 11:04 To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]Problems compiling libol / syslog-ng I have Redhat 7.3 and can't configure libol (I have the same problem with syslog-ng). I don't have an alias for ls, I don't know Why I'm having such problem, I've compiled other tar balls before without this error... Any idea?? [root@agua libol-0.3.3]# ./configure creating cache ./config.cache checking for a BSD compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking whether make sets ${MAKE}... yes checking for working aclocal... found checking for working autoconf... found checking for working automake... found checking for working autoheader... found checking for working makeinfo... found checking whether build environment is sane... configure: error: ls -t appears to fail. Make sure there is not a broken alias in your environment configure: error: newly created file is older than distributed files! Check your system clock Thanks, Chris DISCLAIMER: This message may contain confidential information. If you are not the intended recipient please advise the sender and erase the message. Thank you. DISCLAIMER: This message may contain confidential information. If you are not the intended recipient please advise the sender and erase the message. Thank you. DISCLAIMER: This message may contain confidential information. If you are not the intended recipient please advise the sender and erase the message. Thank you.

1 0

RE: [syslog-ng] logging pauses and log entry truncation
by Dustin Trammell 14 Aug '02

14 Aug '02

I can't comment on the /var/log/kern or /var/log/bootup stuff, as I simply have ALL logs go to a single file and do post-processing on it nightly which filters, splits, etc., and so far I have not had any problems with it failing to log to the logfile at any point. Because I'm sending all logs to a single destination (actually multiple destinations, but the point here is that ALL logs go to them), I don't set the log-level in my iptables statements, but I do set the log prefix: iptables -A INPUT -j LOG --log-prefix "$pkey input drop cleanup " iptables -A INPUT -j DROP So perhaps the log-prefix option could be the cause of the problem since we're both using them. If I get some time this afternoon I'll remove those from a test firewall and see if it continues to mangle/truncate the iptables log entries. --- Dustin D. Trammell Information Security Specialist Penson Financial Services, Inc. -----Original Message----- From: Caylan Van Larson [mailto:caylan@cs.und.edu] Sent: Wednesday, August 14, 2002 10:57 To: Dustin Trammell Cc: 'syslog-ng(a)lists.balabit.hu' Subject: RE: [syslog-ng] logging pauses and log entry truncation Also, I would like to add that syslog-ng stops appending to /var/log/kern after about 20k of data. It just stops, nothing. I restart syslog-ng and here is what happens: /var/log/bootup Aug 14 10:49:44 smack syslog-ng: klogd shutdown succeeded Aug 14 10:49:48 smack syslog-ng: syslog-ng startup succeeded Aug 14 10:49:48 smack syslog-ng: klogd startup succeeded (never says anything about syslog-ng shutdown ducceeded, is this normal?) /var/log/kern Populated for about 30k of data, truncated. I would also like to know why when I restart iptables it puts any output from the scripts to /var/log/bootup??? For instance, the iptables init script calls a firewall script that runs a bunch of iptable commands. This script also outputs some settings: --SNIP (from /var/log/bootup on iptables restart) Aug 14 10:50:53 smack fw-iptables: Serving port 22 (tcp) only to 134.129.212.0/24 134.129.217.128/26 Aug 14 10:50:53 smack fw-iptables: Serving port 113 (tcp) only to 134.129.212.0/24 Aug 14 10:50:53 smack fw-iptables: Serving port 515 (tcp) to the world. Aug 14 10:50:53 smack fw-iptables: Serving needs client as well: Client may access port 9100 (tcp) anywhere. Aug 14 10:50:53 smack fw-iptables: Enabling DNS Server Communications: Aug 14 10:50:53 smack fw-iptables: Serving port 53 (udp) to the world. Aug 14 10:50:53 smack fw-iptables: Serving needs client as well: Client may access port 53 (udp) anywhere. Aug 14 10:50:53 smack fw-iptables: Serving port 53 (tcp) Zone Transfers to 134.129.217.44 Aug 14 10:50:53 smack fw-iptables: Serving port 53 (tcp) Zone Transfers to 134.129.217.46 Aug 14 10:50:53 smack fw-iptables: Serving port 67 (udp) to the world. Aug 14 10:50:53 smack fw-iptables: Serving port 58884 (tcp) only to 134.129.212.0/24 Aug 14 10:50:53 smack fw-iptables: Client may access port 515 (tcp) only on 134.129.212.0/24 Aug 14 10:50:53 smack fw-iptables: Client may access port 389 (tcp) only on 134.129.212.0/24 Aug 14 10:50:53 smack fw-iptables: Client may access port 636 (tcp) only on 134.129.212.0/24 Aug 14 10:50:53 smack fw-iptables: Client may access port 37 (tcp) only on 134.129.212.0/24 Aug 14 10:50:53 smack fw-iptables: Client may access port 514 (tcp) only on 134.129.212.0/24 Aug 14 10:50:53 smack fw-iptables: Client may access port 514 (udp) only on 134.129.212.0/24 Aug 14 10:50:53 smack fw-iptables: Client may access port 143 (tcp) only on 134.129.212.0/24 Aug 14 10:50:53 smack fw-iptables: Client may access port 993 (tcp) only on 134.129.212.0/24 Aug 14 10:50:53 smack fw-iptables: Client may access port 110 (tcp) only on 134.129.212.0/24 Aug 14 10:50:53 smack fw-iptables: Client may access port 995 (tcp) only on 134.129.212.0/24 Aug 14 10:50:53 smack fw-iptables: Client may access port 58884 (tcp) only on 134.129.212.0/24 Aug 14 10:50:53 smack fw-iptables: Client may access port 23 (tcp) anywhere. Aug 14 10:50:53 smack fw-iptables: Client may access port 113 (tcp) anywhere. --SNAP Also, it may help to know that I am running all of my iptables LOG commands with the --log-level 6 and with custom --log-prefix "PREFIX" like this: --SNIP # Any tcp not already allowed is logged and then dropped. iptables -A INPUT -i $IFACE -p tcp -j LOG --log-level 6 --log-prefix "IPTABLES TCP-IN: " iptables -A INPUT -i $IFACE -p tcp -j DROP iptables -A OUTPUT -o $IFACE -p tcp -j LOG --log-level 6 --log-prefix "IPTABLES TCP-OUT: " iptables -A OUTPUT -o $IFACE -p tcp -j DROP --SNAP Are you adding these loglevel and logprefix switches also? Could this possibly be the problem? If iptables is adding these prefixes to late? I dont know :P Thanks, Caylan

1 0

RE: [syslog-ng]Problems compiling libol / syslog-ng
by Dustin Trammell 14 Aug '02

14 Aug '02

Check the time on your system, it's probably wrong. Usually configure won't continue if it finds files that are newer than current date/time, or it creates a file that is older than current date/time. --- Dustin D. Trammell Information Security Specialist Penson Financial Services, Inc. -----Original Message----- From: Camacho, Christian [mailto:CCamacho@golder.cl] Sent: Wednesday, August 14, 2002 11:04 To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]Problems compiling libol / syslog-ng I have Redhat 7.3 and can't configure libol (I have the same problem with syslog-ng). I don't have an alias for ls, I don't know Why I'm having such problem, I've compiled other tar balls before without this error... Any idea?? [root@agua libol-0.3.3]# ./configure creating cache ./config.cache checking for a BSD compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking whether make sets ${MAKE}... yes checking for working aclocal... found checking for working autoconf... found checking for working automake... found checking for working autoheader... found checking for working makeinfo... found checking whether build environment is sane... configure: error: ls -t appears to fail. Make sure there is not a broken alias in your environment configure: error: newly created file is older than distributed files! Check your system clock Thanks, Chris DISCLAIMER: This message may contain confidential information. If you are not the intended recipient please advise the sender and erase the message. Thank you.

1 0

Problems compiling libol / syslog-ng
by Camacho, Christian 14 Aug '02

14 Aug '02

I have Redhat 7.3 and can't configure libol (I have the same problem with syslog-ng). I don't have an alias for ls, I don't know Why I'm having such problem, I've compiled other tar balls before without this error... Any idea?? [root@agua libol-0.3.3]# ./configure creating cache ./config.cache checking for a BSD compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking whether make sets ${MAKE}... yes checking for working aclocal... found checking for working autoconf... found checking for working automake... found checking for working autoheader... found checking for working makeinfo... found checking whether build environment is sane... configure: error: ls -t appears to fail. Make sure there is not a broken alias in your environment configure: error: newly created file is older than distributed files! Check your system clock Thanks, Chris DISCLAIMER: This message may contain confidential information. If you are not the intended recipient please advise the sender and erase the message. Thank you.

1 0