- syslog-ng - lists.balabit.hu

RE: [syslog-ng]syslog-ng template does not work
by Earls, Michael 01 Nov '04

01 Nov '04

The template() destinations are only supported by the 1.5.x and up. I would upgrade your version of syslog-ng. Michael PGP Info: KeyID 0x0DFD993C Fingerprint F903 0325 5105 2CDB 4BF4 C88B 72F7 BA7A 28CC 598A -----Original Message----- From: Sirinus [mailto:sirinus@excite.com] Sent: Monday, November 01, 2004 2:39 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]syslog-ng template does not work Hello, Everyone, I just installed syslog-ng on my mandrake-9.2 with the rpm package: syslog-ng-1.4.17-3mdk.src.rpm For generly log, it works fine. However, I would like to forward my logs to MySql database. So I tried to follow the tutorials of Michael Earls ( ohttp://vermeer.org/display_doc.php?doc_id=1) However, after I added the following line to my syslog-ng.conf, each time when I tried to restart syslog-ng, it tells sth is wrong. The following is what I have done: 1. Add the following to my syslog-ng.conf so that I can log to mysql database: ## Log syslog-ng to mysql database ## destination d_mysql { pipe("/tmp/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; log { source(net); destination(d_mysql); }; 2. stop syslog-ng [root@localhost tmp]# service syslog-ng stop Shutting down kernel logger: [ OK ] Shutting down system logger: [ OK ] 3. restart syslog-ng [root@localhost tmp]# service syslog-ng start Starting system logger: parse error at 27 Parse error reading configuration file, exiting. [FAILED] starting kernel logger: [ OK ] NOTE: line 27 is where "template ("INSERT INTO logs (host, facility, priority, level, tag, " showing the first step. I wonder to know what's wrong with this configuration? I tried to read the source code. I cannot find any thing related to "template". Does syslog-ng 1.4.17 support "template"? How could I fix the problem? THanks a lot Yours Ch Zhang _______________________________________________ Join Excite! - http://www.excite.com The most personalized portal on the Web! _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html This e-mail transmission contains information that is confidential and may be privileged. It is intended only for the addressee(s) named above. If you receive this e-mail in error, please do not read, copy or disseminate it in any manner. If you are not the intended recipient, any disclosure, copying, distribution or use of the contents of this information is prohibited. Please reply to the message immediately by informing the sender that the message was misdirected. After replying, please erase it from your computer system. Your assistance in correcting this error is appreciated.

1 0

syslog-ng template does not work
by Sirinus 01 Nov '04

01 Nov '04

Hello, Everyone, I just installed syslog-ng on my mandrake-9.2 with the rpm package: syslog-ng-1.4.17-3mdk.src.rpm For generly log, it works fine. However, I would like to forward my logs to MySql database. So I tried to follow the tutorials of Michael Earls ( ohttp://vermeer.org/display_doc.php?doc_id=1) However, after I added the following line to my syslog-ng.conf, each time when I tried to restart syslog-ng, it tells sth is wrong. The following is what I have done: 1. Add the following to my syslog-ng.conf so that I can log to mysql database: ## Log syslog-ng to mysql database ## destination d_mysql { pipe("/tmp/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; log { source(net); destination(d_mysql); }; 2. stop syslog-ng [root@localhost tmp]# service syslog-ng stop Shutting down kernel logger: [ OK ] Shutting down system logger: [ OK ] 3. restart syslog-ng [root@localhost tmp]# service syslog-ng start Starting system logger: parse error at 27 Parse error reading configuration file, exiting. [FAILED] starting kernel logger: [ OK ] NOTE: line 27 is where "template ("INSERT INTO logs (host, facility, priority, level, tag, " showing the first step. I wonder to know what's wrong with this configuration? I tried to read the source code. I cannot find any thing related to "template". Does syslog-ng 1.4.17 support "template"? How could I fix the problem? THanks a lot Yours Ch Zhang _______________________________________________ Join Excite! - http://www.excite.com The most personalized portal on the Web!

1 0

installing syslog-ng
by Majumdar, Anamitra 30 Oct '04

30 Oct '04

Hello, I am trying to install syslog-ng on a Solaris 8 box. I am able to configure Libol and syslog-ng. When I am trying to run make I am receiving the following error for syslog-ng $ make Making all in src make all-recursive Making all in . source='main.c' object='main.o' libtool=no \ depfile='.deps/main.Po' tmpdepfile='.deps/main.TPo' \ depmode=none /bin/bash /export/home/amajumda/syslog-ng-1.6.5/depcomp \ cc -DHAVE_CONFIG_H -I. -I/export/home/amajumda/syslog-ng-1.6.5/src -I. -g -Wall -I/export/home/amajumda/libol-0.3.14/src -D_GNU_SOURCE -c `test -f 'main.c' || echo '/export/home/amajumda/syslog-ng-1.6.5/src/'`main.c cc: illegal option -Wall *** Error code 1 make: Fatal error: Command failed for target `main.o' Current working directory /export/home/amajumda/syslog-ng-1.6.5/src *** Error code 1 make: Fatal error: Command failed for target `all-recursive' Current working directory /export/home/amajumda/syslog-ng-1.6.5/src *** Error code 1 make: Fatal error: Command failed for target `all' Current working directory /export/home/amajumda/syslog-ng-1.6.5/src *** Error code 1 make: Fatal error: Command failed for target `all-recursive' I am receiving a similar error while trying to install libol-0.3.14 $ make Making all in utils Making all in src make all-am source='format.c' object='format.lo' libtool=yes \ depfile='.deps/format.Plo' tmpdepfile='.deps/format.TPlo' \ depmode=none /bin/bash /export/home/amajumda/libol-0.3.14/depcomp \ /bin/bash ../libtool --mode=compile cc -DHAVE_CONFIG_H -I. -I/export/home/amajumda/libol-0.3.14/src -I. -D_GNU_SOURCE -D_XOPEN_SOURCE_EXTENDED -g -Wall -O2 -c -o format.lo `test -f 'format.c' || echo '/export/home/amajumda/libol-0.3.14/src/'`format.c cc -DHAVE_CONFIG_H -I. -I/export/home/amajumda/libol-0.3.14/src -I. -D_GNU_SOURCE -D_XOPEN_SOURCE_EXTENDED -g -Wall -O2 -c format.c -KPIC -DPIC -o .libs/format.o cc: illegal option -Wall *** Error code 1 make: Fatal error: Command failed for target `format.lo' Current working directory /export/home/amajumda/libol-0.3.14/src *** Error code 1 make: Fatal error: Command failed for target `all' Current working directory /export/home/amajumda/libol-0.3.14/src *** Error code 1 make: Fatal error: Command failed for target `all-recursive' Can you help with a solution of overcoming this problem. Thanks Anamitra X4696

2 1

PIX 2 MySQL
by Cary, Kim 29 Oct '04

29 Oct '04

I just got logging going with syslog-ng in the last couple weeks (first client is our PIX 520). We can have up to 20Gb/day from our PIX. When compressed, the logs are up to 2Gb/day. We do want a record of all sessions for forensics & troubleshooting (already saved us hours of time) but the log format is quite verbose. Because of that verbosity, I was thinking of writing just key fields to a MySQL database as you suggest. However, I don't want to get into a situation where the only reporting is whatever report script I have time to write... If Joseph or someone else has a suggestion for fields to insert into the db and a reporting package to use, I'd appreciate it. Kim Cary InfraSec Admin Pepperdine University On Oct 29, 2004, at 5:43 AM, syslog-ng-request(a)lists.balabit.hu wrote: >> server. The central server is currently piping the information to a >> Mysql >> database. Each incoming device writes to its own table in the >> database. A >> modification to this we would like to accomplish is to key various >> pieces >> of information stored in the "message" field. >> >> For example, syslog messages sent from the mail servers will contain >> the >> sender, recipient, delivery status in the "message" field. Our >> thought is >> to key these pieces of information for quick lookup. Some of the >> systems >> (Cisco Pix) are sending up to 5G of information a day. Another reason >> to >> key the information.

3 2

Re:
by DTrammell 29 Oct '04

29 Oct '04

1 0

Re: Syslog-ng reply script for centralized syslog data
by LEROY ISAAC 29 Oct '04

29 Oct '04

Thanks a lot for both of your suggestions. It is an assumption that QOS will be on the network link, but this still doesn't account for a continually backed up host trying to send data. We are planning to utilize TCP as the transport. I figured TCP, QOS, and adjustments in the size of the log_fifo_size (num - The number of lines fitting to the output queue) might solve the issue. It is my understanding that the log_fifo_size, which does not have flow control, can be used to buffer data bound for destinations (either local disk or remote). I might be wrong in this understanding. I need the replay script to feed into the syslog data stream on the central syslog-ng server, since the central syslog-ng server feeds other applications. LeRoy

2 1

parsing the message field
by Joseph Deck 29 Oct '04

29 Oct '04

All, We have installed syslog-ng on our unix systems pointing them, some network devices, and some Windows systems events to a central syslog-ng server. The central server is currently piping the information to a Mysql database. Each incoming device writes to its own table in the database. A modification to this we would like to accomplish is to key various pieces of information stored in the "message" field. For example, syslog messages sent from the mail servers will contain the sender, recipient, delivery status in the "message" field. Our thought is to key these pieces of information for quick lookup. Some of the systems (Cisco Pix) are sending up to 5G of information a day. Another reason to key the information. Our current thought is to send the output of syslog-ng to a Perl script which will parse the message field based upon the source. The Perl script would then write it to the database. Is there a better approach? Any suggestions would be appreciated. Joseph G. Deck Director of Computing Services Wittenberg University Phone: (937) 525-3800 Fax: (937) 327-7372

2 1

Re: Syslog-ng reply script for centralized syslog data
by LEROY ISAAC 28 Oct '04

28 Oct '04

Thanks a lot for both of your suggestions. It is an assumption that QOS will be on the network link, but this still doesn't account for a continually backed up host trying to send data. We are planning to utilize TCP as the transport. I figured TCP, QOS, and adjustments in the size of the log_fifo_size (num - The number of lines fitting to the output queue) might solve the issue. It is my understanding that the log_fifo_size, which does not have flow control, can be used to buffer data bound for destinations (either local disk or remote). I might be wrong in this understanding. I need the replay script to feed into the syslog data stream on the central syslog-ng server, since the central syslog-ng server feeds other applications. LeRoy

1 0

Syslog-ng replay script for centralized syslog data
by LEROY ISAAC 28 Oct '04

28 Oct '04

I have a need to retrieve syslog data from various remote nodes, and the smallest network link to the remote nodes is 19K. The syslog traffic for the link cannot exceed 9K. I plan to setup a configuration which generates new log files every 10 minutes. These files are then compressed, zipped, and transfered to a centralized loghost. The files are then unzipped, uncompressed, and the data is inserted into the syslog-ng data stream on the central syslog-ng host. Is there a script or utility which will accomplish this task? If not, then does any one have any suggestions on products which may accomplish this same task. LeRoy Isaac --- DTrinh71(a)aol.com wrote: > OK. Thanks. > > So, what does Ray want? Suggestions? > > David >

3 6

Perl for log analysis
by beproj beproj 28 Oct '04

28 Oct '04

Subject: 1. Perl for log analysis 2. Specimen logs hello 1. I am pursuing a project in log analysis(mainly attacklogs..security) and i have a doubt regarding general log analyis. I would like to know why generally Perl, Python etc. are used for log analysis. The SYSLOG MODULE support in Perl is one reason. But a similar library in GNU C lib is available. viz:- syslog.h Why then do programmers go for Perl? Is it due to simpler pattern matching techniques available. Same are then available in C viz:- strtok(). Since i am new to perl and have to decide which of C and Perl is better for syslog log analysis, kindly sugegst the advantages of perl over C wrt log analysis. 2. I want specimen syslog logs esp security...for Red Hat Linux 8.0 Server. Any idea where can i get them? Thank you in advance. bye beproj

3 2