May 2003 - syslog-ng - lists.balabit.hu

RE: [syslog-ng]syslog-ng blanks in Servicename
by Christian Osbahr 07 May '03

07 May '03

Hi, I just installed syslog-ng, and had a similar experience with a certain type of switches (Alteon's). Only in this case the syslog messages were formatted so that syslog-ng saw the level as the hostname (i.e. my syslog files would be named e.g. NOTICE.log when using $HOST.log for the filename). Using the bad_hostname() option, as described below, solved the problem. However, the question this is all leading to is this: When a hostname is matched by the bad_hostname option, what mechanism does syslog-ng use to establish a new name? Best regards, Christian -----Original Message----- From: Balazs Scheidler [mailto:bazsi@balabit.hu] Sent: 7. maj 2003 13:54 To: syslog-ng(a)lists.balabit.hu Subject: Re: [syslog-ng]syslog-ng blanks in Servicename On Wed, May 07, 2003 at 12:25:28PM +0200, Unger, Christian wrote: > >On Wed, Apr 30, 2003 at 12:22:34PM +0200, Unger, Christian wrote: > >> i log many informations via NT-Syslog to syslog-ng. Windows 2000 > >> have some services with a blank in the Servicename. ex: NTDS ISAM > >> > >> the log entry from syslog-ng seams as following: > >> Apr 15 15:42:33 ntds isam[info] 700 NTDS (268) Online > >> defragmentation is beginning a full pass on database > >> 'C:\WINNT\NTDS\ntds.dit'. > >> > >> NTDS ist NOT my hostname, thats the problem! > >> > >> I think thats the problem is in RFC. But i don't read it. ( no time > >> :[ ) > >> > >> is there a easy escape? or will the next version of syslog-ng have > >> a escape? > > > >try check_hostname(yes) > > > ---- > > check_hostnames(yes) doesn't help. > > I checked some information's with tcpdump. An I found a very interest > handling. Syslog-ng is able to handle hostnames in syslog datagram, > the delimiter between hostname and service name seams to be a space. > So if I have spaces in service name syslog-ng thinks that the first > word must be the hostname, but that's wrong. What can I do know? sorry, check_hostname() is meant for cases when the word seeming to be a hostname contains invalid characters (like '%'). The problem is that there is no way to distinguish between a hostname and a programname containing a space. The workaround I implemented was the bad_hostname() option, which contains a regular expression which covers bad hostnames. E.g. if you have a program that sends messages like: Apr 28 15:13:54 snmp ctld[12345]: message And "snmp ctld" is the program name, then you can use bad_hostname() to make syslog-ng think that "snmp" itself is a bad hostname. (of course this will not work if you really have a host named snmp) options { bad_hostname("snmp"); }; bad_hostname() accepts an extended regular expression so you can list several bad hostnames like this: options { bad_hostname("snmp|iis|foo|bar"); }; -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 1

AW: [syslog-ng]syslog-ng blanks in Servicename
by Unger, Christian 07 May '03

07 May '03

>On Wed, Apr 30, 2003 at 12:22:34PM +0200, Unger, Christian wrote: >> Hi, >> >> i log many informations via NT-Syslog to syslog-ng. Windows 2000 have some services with a blank in the Servicename. ex: NTDS ISAM >> >> the log entry from syslog-ng seams as following: >> Apr 15 15:42:33 ntds isam[info] 700 NTDS (268) Online defragmentation is beginning a full pass on database 'C:\WINNT\NTDS\ntds.dit'. >> >> NTDS ist NOT my hostname, thats the problem! >> >> I think thats the problem is in RFC. But i don't read it. ( no time :[ ) >> >> is there a easy escape? or will the next version of syslog-ng have a escape? > >try check_hostname(yes) > ---- check_hostnames(yes) doesn't help. I checked some information's with tcpdump. An I found a very interest handling. Syslog-ng is able to handle hostnames in syslog datagram, the delimiter between hostname and service name seams to be a space. So if I have spaces in service name syslog-ng thinks that the first word must be the hostname, but that's wrong. What can I do know?

2 1

syslog-ng blanks in Servicename
by Unger, Christian 06 May '03

06 May '03

Hi, i log many informations via NT-Syslog to syslog-ng. Windows 2000 have some services with a blank in the Servicename. ex: NTDS ISAM the log entry from syslog-ng seams as following: Apr 15 15:42:33 ntds isam[info] 700 NTDS (268) Online defragmentation is beginning a full pass on database 'C:\WINNT\NTDS\ntds.dit'. NTDS ist NOT my hostname, thats the problem! I think thats the problem is in RFC. But i don't read it. ( no time :[ ) is there a easy escape? or will the next version of syslog-ng have a escape? Thanks for help Christian Unger

2 1

Re: Oops. I broke it. :(
by Brad Stockdale 06 May '03

06 May '03

Oops... I upgraded my syslog-ng from 1.4 train to 1.6 train, and it doesnt like my old configuration file... It seems to be choking on the following line: [root@internal syslog-ng-1.6.0rc3]# /usr/local/sbin/syslog-ng -f /etc/syslog-ng.conf unresolved reference: DEFAULT Error initializing configuration, exiting. Here's my config: options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { pipe("/proc/kmsg"); unix-stream("/dev/log"); internal(); }; source s_net { udp(); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog"); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_mlal { usertty("*"); }; destination d_udph { file("/var/log/$HOST/$YEAR/$MONTH/$DAY/$FACILITY.log" create_dirs(yes)); }; filter f_filter1 { facility(kern); }; filter f_filter2 { level(info) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_filter3 { facility(authpriv); }; filter f_filter4 { facility(mail); }; filter f_filter5 { level(emerg); }; filter f_filter6 { facility(uucp) or (facility(news) and level(crit)); }; filter f_filter7 { facility(local7); }; filter f_filter8 { facility(cron); }; log { source(s_net); filter(DEFAULT); destination(d_udph); }; #log { source(s_sys); filter(f_filter1); destination(d_cons); }; log { source(s_sys); filter(f_filter2); destination(d_mesg); }; log { source(s_sys); filter(f_filter3); destination(d_auth); }; log { source(s_sys); filter(f_filter4); destination(d_mail); }; log { source(s_sys); filter(f_filter5); destination(d_mlal); }; log { source(s_sys); filter(f_filter6); destination(d_spol); }; log { source(s_sys); filter(f_filter7); destination(d_boot); }; log { source(s_sys); filter(f_filter8); destination(d_cron); }; log { source(s_sys); filter(DEFAULT); destination(d_mesg); }; It seems to dislike this line (I guess): log { source(s_net); filter(DEFAULT); destination(d_udph); }; Is there no more a default filter? Or am I just being stupid and cannot see a simple error? Thanks, Brad

2 1

Re: Ooo! I fixed it!
by Brad Stockdale 06 May '03

06 May '03

I guess RTFM is the quote of the day. I found reference to the DEFAULT idea being removed in preference to the flags(catchall)... It seems to be working fine. Thanks, Brad

1 0

Re: Log analysis
by Brad Stockdale 06 May '03

06 May '03

Hello all, I've been using syslog-ng for about a year now on a centralized logging server. It works great! But now I have 9 gigabytes of past logs... haha... In any case, I am trying to find a good way to analyze the daily logs... Sort of condense them like LogSentry/LogWatch does, but via a web interface... Can anyone out there recommend a good log analysis package? I started to try out lire, but part way through the setup I figured out that it might be a pain to try to get lire to understand the log rotations, etc... I rotate logs daily with the following directory structure: /var/log/$IPADDRESS/$4DIGITYEAR/$2DIGITMONTH/$2DIGITDAY/facilitypriority.log lire expects the logs to be in /var/log/ and use the normal rotation of putting a number at the end of the filename... Any thoughts? Thanks, Brad

1 0

Few general questions.
by Dawid Szymański 06 May '03

06 May '03

Hello, I'd like to get answers for few questions.. 1. Which version of syslogd-ng I should call stable? Its really complicated when I can read in docs that 1.4 is stable on web page that 1.6 is etc etc.. 2. Does any of syslogd-ng versions support encrypted connections over udp/tcp? 3. When/Are you going to support encrypted connections? Impatient ;) -- Kind regards, Dawid Szymanski

2 2

RE: [syslog-ng]Few general questions.
by Hamilton, Andrew 06 May '03

06 May '03

1. Well, I use 1.6 and it is very stable, just get the latest snapshot. There was a release that had a macro bug but it has since been remedied. I have been using 1.6 for the last several months without issue. 2. Currently syslog-ng doesn't support native encrypted sessions. You would have to use an add-on to get that functionality. 3. The encryption support is planned in v2.0. That is still in alpha and may not even be implemented in the series yet. Bazsi would have more details on whether that is available in that line or not. You can tap your foot and sigh with impatience all you want but Bazsi has lots on his plate and the improvements come has he has time. You'll just have to learn a little patience. Syslog-ng still beats the heck out of old syslogd. Regards, Drew -----Original Message----- From: Dawid Szymanski [mailto:dawszy@axit.pl] Sent: Tuesday, May 06, 2003 9:44 AM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]Few general questions. Hello, I'd like to get answers for few questions.. 1. Which version of syslogd-ng I should call stable? Its really complicated when I can read in docs that 1.4 is stable on web page that 1.6 is etc etc.. 2. Does any of syslogd-ng versions support encrypted connections over udp/tcp? 3. When/Are you going to support encrypted connections? Impatient ;) -- Kind regards, Dawid Szymanski _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

RE: [syslog-ng]Sorting data from msg field into correct columns?
by Michael.Semling＠swisscom.com 06 May '03

06 May '03

Hi list, I can be more specific. I have in a DMZ proxies, routers and firewalls. Theses report into a syslog server in the DMZ. I want to have the messages in one central place so I send them to my central syslog server with a MYSQL DB. There they arrive but they look like following example. a.a.a.a = Where it comes from originally (at least what I think ;-)) b.b.b.b = NAME of the syslog server A router message | a.a.a.a | syslog5 | info | info | ae | 2003-05-06 | 04:21:38 | b.b.b.b | b.b.b.b : [ID 72119 syslog5.info] 04:21:36 drop rgw2 >eth0 reason: ACL: Ingress filter dropped packet ; src: a.b.c.d; s_port: 933; dst: f.g.h.i; d_port: 514; | 1015510 | A firewall message | c.c.c.c | syslog5 | info | info | ae | 2003-05-06 | 04:21:38 | b.b.b.b | b.b.b.b: [ID 72119 syslog5.info] 04:21:37 drop fwgw >eth1 product: SmartDefense; TCP sequence validator: dropped packet with invalid ACK number; attack: Bad TCP sequence; src: i.j.k.l; s_port: 80; dst: m.n.o.p; service: shell; proto: tcp; | 1015511 | So I would like to have the machine name as rgw2 or fwgw on the hosts or facility's place. Maybe the word 'drop' as level so I can create an index for fast search. Ideal would be, if I could do a search in the $MSG and then filter the entry and putting parts of the $MSG into different variabls e.g. $HOST etc. I guess this could already be done with a regexp search in the $MSG field then assigning the different variables. But I guess I have to hardcode all cases and I'd just like to take something like $MSG.$1, $MSG.$2 etc. This allowed me to do some kind of "message normalisation". Maybe that's impossible for some reasons (speed). Regards, Michael -----Original Message----- From: Balazs Scheidler [mailto:bazsi@balabit.hu] Sent: Montag, 5. Mai 2003 14:39 To: syslog-ng(a)lists.balabit.hu Subject: Re: [syslog-ng]Sorting data from msg field into correct columns? On Mon, May 05, 2003 at 10:55:07AM +0200, Michael.Semling(a)swisscom.com wrote: > Hi all! > > As I have now added hardware to report into a central syslog but I have > to uses a relay. (Another SYSLOG-NG) > Now I get messages containing > > Host: IP/Name of the relay > Facility, Priority, Level, tag, date, time and program is just all info > from the relay. > > In the msg I have all the data as the original host, time, msg. etc. > > Is there any way to "grep/sed/awk" the data out of the msg field and > sort them into the right columns? > > Is there an example for such a filter? Can you be more specific, can you perhaps provide examples (what you would expect and what happens in reality) -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

Sorting data from msg field into correct columns?
by Michael.Semling＠swisscom.com 05 May '03

05 May '03

Hi all! As I have now added hardware to report into a central syslog but I have to uses a relay. (Another SYSLOG-NG) Now I get messages containing Host: IP/Name of the relay Facility, Priority, Level, tag, date, time and program is just all info from the relay. In the msg I have all the data as the original host, time, msg. etc. Is there any way to "grep/sed/awk" the data out of the msg field and sort them into the right columns? Is there an example for such a filter? Any help is appreciated. Regards, Michael -----Original Message----- From: Hamilton, Andrew [mailto:Andrew.Hamilton@afccc.af.mil] Sent: Freitag, 2. Mai 2003 14:00 To: syslog-ng(a)lists.balabit.hu Subject: RE: [syslog-ng]Simple question, MySQL & Log file Just add an additional log line or additional destination. log { source(src); destination(dmysql); destination(maillog); }; or log { source(src); destination(dmysql); }; log { source(src); destination(maillog); }; Regards, Drew -----Original Message----- From: Bas Koot [mailto:bask@tiscali.nl] Sent: Thursday, May 01, 2003 5:43 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]Simple question, MySQL & Log file Hi everybody! First of all, i like Syslog-NG ! But i have 1 (simple i think) question about it. For my postfix i need to use the /var/log/maillog file (for the pop-before-smtp feature) but at this moment everything is putted into the MySQL database. Somebody knows how to get the mail data in the database AND into the log file? My current config: # sources source src { unix-stream("/dev/log"); internal(); }; source kernsrc { file("/proc/kmsg"); }; # destinations destination dmysql { pipe("/tmp/mysql.pipe" template("INSERT INTO logs (host,facility,priority,level,tag, timestamp,program,msg) VALUES ('$HOST','$FACILITY', '$PRIORITY','$LEVEL','$TAG','$UNIXTIME','$PROGRAM', '$MSG');\n") template-escape(yes)); }; destination maillog { file("/var/log/maillog"); }; # logs log { source(src); destination(dmysql); }; log { source(kernsrc); destination(dmysql); }; Thanks! Greetings, Bas Koot.

2 1