October 2004 - syslog-ng - lists.balabit.hu

Perl for log analysis
by beproj beproj 28 Oct '04

28 Oct '04

Subject: 1. Perl for log analysis 2. Specimen logs hello 1. I am pursuing a project in log analysis(mainly attacklogs..security) and i have a doubt regarding general log analyis. I would like to know why generally Perl, Python etc. are used for log analysis. The SYSLOG MODULE support in Perl is one reason. But a similar library in GNU C lib is available. viz:- syslog.h Why then do programmers go for Perl? Is it due to simpler pattern matching techniques available. Same are then available in C viz:- strtok(). Since i am new to perl and have to decide which of C and Perl is better for syslog log analysis, kindly sugegst the advantages of perl over C wrt log analysis. 2. I want specimen syslog logs esp security...for Red Hat Linux 8.0 Server. Any idea where can i get them? Thank you in advance. bye beproj

3 2

Install 1.6 on RedHat 8.0
by Virgil 28 Oct '04

28 Oct '04

Please excuse my ignorance - I am trying to install syslog-ng on RH 8.0 and have read the INSTALL document. I have the necessary dependancies from what I can tell (I'm also pretty new to RH / Linux); I have done the ./confgure, which seemed to go smooth. Following the ./configure, the INSTALL doc says to do a "make" - this is where I get hung up. I don't understand what the errors below are telling me; some of the files that are reported as missing are actually not present. However I have done everything exactly as the documentation specifies. What am I missing? root@RedHat8 syslog-ng-1.6.5]# make Making all in src make[1]: Entering directory `/misc/downloads/syslog-ng-1.6.5/src' make all-recursive make[2]: Entering directory `/misc/downloads/syslog-ng-1.6.5/src' Making all in . make[3]: Entering directory `/misc/downloads/syslog-ng-1.6.5/src' if gcc -DHAVE_CONFIG_H -I. -I/misc/downloads/syslog-ng-1.6.5/src -I. -g -O2 -Wall -D_GNU_SOURCE -MT main.o -MD -MP -MF ".deps/main.Tpo" \ -c -o main.o `test -f 'main.c' || echo '/misc/downloads/syslog-ng-1.6.5/src/'`main.c; \ then mv -f ".deps/main.Tpo" ".deps/main.Po"; \ else rm -f ".deps/main.Tpo"; exit 1; \ fi In file included from main.c:26: syslog-ng.h:31:21: objects.h: No such file or directory In file included from cfgfile.h:29, from main.c:27: sources.h:30:23: read_line.h: No such file or directory In file included from sources.h:32, from cfgfile.h:29, from main.c:27: log.h:30:16: io.h: No such file or directory In file included from sources.h:32, from cfgfile.h:29, from main.c:27: log.h:39: parse error before "UINT32" log.h:39: warning: no semicolon at end of struct or union log.h:43: parse error before "pri" log.h:43: warning: type defaults to `int' in declaration of `pri' log.h:43: warning: data definition has no type or storage class log.h:44: parse error before "flags" log.h:44: warning: type defaults to `int' in declaration of `flags' log.h:44: warning: data definition has no type or storage class log.h:51: parse error before '}' token In file included from log.h:54, from sources.h:32, from cfgfile.h:29, from main.c:27: log.h.x:4: field `super' has incomplete type log.h.x:9: confused by earlier errors, bailing out make[3]: *** [main.o] Error 1 make[3]: Leaving directory `/misc/downloads/syslog-ng-1.6.5/src' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/misc/downloads/syslog-ng-1.6.5/src' make[1]: *** [all] Error 2 make[1]: Leaving directory `/misc/downloads/syslog-ng-1.6.5/src' make: *** [all-recursive] Error 1 [root@RedHat8 syslog-ng-1.6.5]#

3 2

Syslog-ng reply script for centralized syslog data
by LEROY ISAAC 28 Oct '04

28 Oct '04

I have a need to retrieve syslog data from various remote nodes, and the smallest network link to the remote nodes is 19K. The syslog traffic for the link cannot exceed 9K. I plan to setup a configuration which generates new log files every 10 minutes. These files are then compressed, zipped, and transfered to a centralized loghost. The files are then unzipped, uncompressed, and the data is inserted into the syslog-ng data stream on the central syslog-ng host. Is there a script or utility which will accomplish this task? If not, then does any one have any suggestions on products which may accomplish this same task. LeRoy Isaac

1 0

STATS: dropped 74625 and very high Load
by Stefan Schluppeck 28 Oct '04

28 Oct '04

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, I have running a central syslog-ng under SLES9 (SuSE Linux Enterprice Server). ca. 90 Servers and 150 Network copunents are logging to this server I have some times "STATS: dropped 74625" and i have high load round about 11. What can i do, that no loglines are lost and the load is not so high? My global config: options { ~ long_hostnames(off); ~ sync(1024); ~ log_fifo_size(300000); ~ gc_busy_threshold(1000); ~ gc_idle_threshold(30000); ~ }; - -- Mit freundlichen Grüßen / With regards Stefan Schluppeck Bauer Publishinggroup - Brieffach 6200 - Burchard.11 - D-20077 Hamburg Publishing Support Center - Server & Development - Stefan Schluppeck Fon:+49-40-3019-5519 - Fax:+49-40-3019-5668 - eMail:sschluppeck@hbv.de ~ <°)))))>< ><)))))°> <°)))))>< ><)))))°> <°)))))>< ><)))))°> PGP-Fingerabdruck = 7DC1 E4B4 056C 5380 8C68 4B5F 5E87 227D 5632 E45F Key-ID = 81B7CD29 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFBgOsCXocifVYy5F8RAivMAJ9NUFcNBN4pkvxB3WXJuK7eKGEhcQCfdckM Co0jZoRGSfJejuhQsOxx89U= =S+pY -----END PGP SIGNATURE-----

2 2

RE: [syslog-ng]bind_inet_socket() error
by Walt Rich 28 Oct '04

28 Oct '04

I found the answer by filtering by hosts. Thanks! -----Original Message----- From: syslog-ng-admin(a)lists.balabit.hu [mailto:syslog-ng-admin@lists.balabit.hu] On Behalf Of Walt Rich Sent: Wednesday, October 27, 2004 9:22 AM To: syslog-ng(a)lists.balabit.hu Subject: RE: [syslog-ng]bind_inet_socket() error Is there a link that you could forward that would explain how to do this? I'm a newbie to Linux and Syslog-ng. Thanks for everyone's assistance with this. -----Original Message----- From: syslog-ng-admin(a)lists.balabit.hu [mailto:syslog-ng-admin@lists.balabit.hu] On Behalf Of Nate Campi Sent: Tuesday, October 26, 2004 8:00 PM To: syslog-ng(a)lists.balabit.hu Subject: Re: [syslog-ng]bind_inet_socket() error Right, packet filter and/or compile with TCP wrappers for access control, syslog-ng itself won't do that for you. On Tue, Oct 26, 2004 at 02:11:18PM -0500, Dave Johnson wrote: > * Set you max connections to the number of connections that you anticipate. > * Otherwise-- Firewall. > > On Mon, 25 Oct 2004 10:29:08 -0500, Walt Rich <walt.rich(a)parago.com> wrote: > > OK, I see what you mean. > > > > What is the syntax for setting up a source that is an IP address > > that is not local? I'm trying to set up a centralized logging > > server and would like for it to receive logging from other servers, > > but I would like to be able to specify the IP addresses that it will accept input from. > > > > Thanks! > > > > > > > > -----Original Message----- > > From: syslog-ng-admin(a)lists.balabit.hu > > [mailto:syslog-ng-admin@lists.balabit.hu] On Behalf Of Balazs > > Scheidler > > Sent: Sunday, October 24, 2004 3:53 AM > > To: syslog-ng(a)lists.balabit.hu > > Subject: RE: [syslog-ng]bind_inet_socket() error > > > > On Sun, 2004-10-24 at 00:36, Walt Rich wrote: > > > Response to a) shows: > > > # ifconfig -a | grep 172 > > > inet 172.16.100.202 netmask ffffff00 broadcast > > > 172.16.100.255 > > > > but you are trying to bind to 172.16.65.61, which is not local. > > (your local IP is .202) > > > > (for TCP sources you are specifying the bind IP and not the IP > > address you want to receive messages from) > > > > -- > > Bazsi > > > > _______________________________________________ > > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Frequently asked questions at > > http://www.campin.net/syslog-ng/faq.html > > > > _______________________________________________ > > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Frequently asked questions at > > http://www.campin.net/syslog-ng/faq.html > > > > > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html > -- Nate "Usenet is distributed network anarchy at its best--or worst, depending on what is posted on any particular day." - Fiedler, David in 'Byte' _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 1

Source IP address incorrect
by Singh, Mandeep Mr. Adnet 28 Oct '04

28 Oct '04

All, I seem to be having a common problem with forwarding syslog messages. I have read through archives and the solution that is most popular isn't working for me. I have a central server with is properly receiving messages from my PIX firewalls. On that same server I have an application from Cisco with acts like a syslogd daemon and parses PIX messages into separate categories. Syslog-ng is listening on 514 and the Cisco product is listening on 515. My problem is the source IP address that the Cisco application receives is the IP address of the server not the PIX that produced it. Any help would be greatly appreciated, below is my configuration file. Thanks in advance. Mandeep options { keep_hostname(yes); chain_hostnames(no); create_dirs(yes); use_time_recvd(yes); use_dns(no); use_fqdn(no); }; source s_external { udp(); }; destination d_firewall_file { file("/logs/$YEAR/$MONTH/$DAY/firewall.log" perm(0655) dir_perm(0655)); }; destination d_firewall_Port { udp("192.168.0.1" port(515)); }; log { source(s_external); destination(d_firewall_file); }; log { source(s_external); destination(d_firewall_Port); };

2 1

RE: [syslog-ng]bind_inet_socket() error
by Walt Rich 27 Oct '04

27 Oct '04

Is there a link that you could forward that would explain how to do this? I'm a newbie to Linux and Syslog-ng. Thanks for everyone's assistance with this. -----Original Message----- From: syslog-ng-admin(a)lists.balabit.hu [mailto:syslog-ng-admin@lists.balabit.hu] On Behalf Of Nate Campi Sent: Tuesday, October 26, 2004 8:00 PM To: syslog-ng(a)lists.balabit.hu Subject: Re: [syslog-ng]bind_inet_socket() error Right, packet filter and/or compile with TCP wrappers for access control, syslog-ng itself won't do that for you. On Tue, Oct 26, 2004 at 02:11:18PM -0500, Dave Johnson wrote: > * Set you max connections to the number of connections that you anticipate. > * Otherwise-- Firewall. > > On Mon, 25 Oct 2004 10:29:08 -0500, Walt Rich <walt.rich(a)parago.com> wrote: > > OK, I see what you mean. > > > > What is the syntax for setting up a source that is an IP address > > that is not local? I'm trying to set up a centralized logging > > server and would like for it to receive logging from other servers, > > but I would like to be able to specify the IP addresses that it will accept input from. > > > > Thanks! > > > > > > > > -----Original Message----- > > From: syslog-ng-admin(a)lists.balabit.hu > > [mailto:syslog-ng-admin@lists.balabit.hu] On Behalf Of Balazs > > Scheidler > > Sent: Sunday, October 24, 2004 3:53 AM > > To: syslog-ng(a)lists.balabit.hu > > Subject: RE: [syslog-ng]bind_inet_socket() error > > > > On Sun, 2004-10-24 at 00:36, Walt Rich wrote: > > > Response to a) shows: > > > # ifconfig -a | grep 172 > > > inet 172.16.100.202 netmask ffffff00 broadcast > > > 172.16.100.255 > > > > but you are trying to bind to 172.16.65.61, which is not local. > > (your local IP is .202) > > > > (for TCP sources you are specifying the bind IP and not the IP > > address you want to receive messages from) > > > > -- > > Bazsi > > > > _______________________________________________ > > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Frequently asked questions at > > http://www.campin.net/syslog-ng/faq.html > > > > _______________________________________________ > > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Frequently asked questions at > > http://www.campin.net/syslog-ng/faq.html > > > > > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html > -- Nate "Usenet is distributed network anarchy at its best--or worst, depending on what is posted on any particular day." - Fiedler, David in 'Byte' _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

Sending a log line via e-mail
by Csehi Andras 27 Oct '04

27 Oct '04

Hi! I try it (based on the documentation), but it isn't working. The destination line look like this: destination d_ups { program("mail -s UPSlog acsehi(a)qsoft.hu"); }; the syslog-ng try to send the line(s), but the bsd mailer waits something. If I try it in command line (with pipe-line), the command works perfectly. The environment: Debian 3.0r2, Kernel: 2.4.18, syslog-ng: 1.5.15. Sincerelly, András, Csehi Csehi András acsehi(a)freemail.hu, acsehi(a)qsoft.hu (1) 209-6651, (20) 410-5343

4 4

RE: [syslog-ng]bind_inet_socket() error
by Walt Rich 27 Oct '04

27 Oct '04

OK, I see what you mean. What is the syntax for setting up a source that is an IP address that is not local? I'm trying to set up a centralized logging server and would like for it to receive logging from other servers, but I would like to be able to specify the IP addresses that it will accept input from. Thanks! -----Original Message----- From: syslog-ng-admin(a)lists.balabit.hu [mailto:syslog-ng-admin@lists.balabit.hu] On Behalf Of Balazs Scheidler Sent: Sunday, October 24, 2004 3:53 AM To: syslog-ng(a)lists.balabit.hu Subject: RE: [syslog-ng]bind_inet_socket() error On Sun, 2004-10-24 at 00:36, Walt Rich wrote: > Response to a) shows: > # ifconfig -a | grep 172 > inet 172.16.100.202 netmask ffffff00 broadcast 172.16.100.255 but you are trying to bind to 172.16.65.61, which is not local. (your local IP is .202) (for TCP sources you are specifying the bind IP and not the IP address you want to receive messages from) -- Bazsi _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

3 2

forwarding a syslog packet
by Mark McCurdy 26 Oct '04

26 Oct '04

I have three syslog machines. One sits in the DMZ and sends only. One sits on the LAN, receives traffic only and is our centralized syslog host. The 3rd sits between the above two, and it's duty is to resend incoming DMZ syslog traffic to the LAN server. My problem is that all packets being sent from the DMZ are being shown on the LAN server as being from the middle box. When I inspect the packet on the LAN machine, I see the original machine's hostname is mentioned in the packet, but that hostname is ultimately ignored. My old firewall proxy solution just udp tunneled the dmz syslog; therefore, no resending happened. Help and thanks, Mark

1 0