May 2003 - syslog-ng - lists.balabit.hu

Re: [syslog-ng]syslog-ng-1.6.0rc3 - problem with incorrect separation of syslog messages from Cisco PIX
by Ted_Rule＠flextech.co.uk 12 May '03

12 May '03

A bitter missive from last year with my previous comments on the utter stupidity of Cisco PIX syslog formatting and the magic 256 byte message truncation. Cisco still haven't fixed it, and apparently couldn't give a monkey's. Ted Ted Rule 05/09/2002 16:28 To: syslog-ng(a)lists.balabit.hu cc: Subject: Cisco PIX TCP Syslogging via syslog-ng More very sad news I'm afraid. I've been able to glue together a temporary copy of syslog-ng on my central syslog server listening to TCP/1468 for PIX syslogging. It is already apparent that things don't work properly - most of this appears to be entirely Cisco's fault. My original problem was that I'd reported UDP packet truncation for syslog messages > 255 characters. Cisco swore blind that my moving to TCP would fix this. tcpdump's of my test rig strongly suggests Cisco simply lied. Even the TCP stream seems to impose a 255 character limit prior to transmission. Worse still - as intimated from previous missives - the TCP stream doesn't contain EOM characters of any flavour between messages such that multiple syslog messages may appear in a single TCP packet. Moreover, sadly, syslog-ng appears to be unable to pick apart multiple messages correctly on some occasions, and hence one sees things like this: Sep 5 15:58:58 fttvgpsvpn1 %PIX-7-702301: lifetime expiring, (sa) sa_dest= 195. 188.171.5, sa_prot= 50, sa_spi= 0x6a3b8b02(1782287106), sa_trans= esp-3des esp-m d5-hmac , sa_conn_id= 8, (identity) local= 195.188.171.5, remote= 217.34.209.200 , local_proxy= 194.34.198.40/255.255.2<190>%PIX-6-602302: deleting SA, (sa) sa_d est= 195.188.171.5, sa_prot= 50, sa_spi= 0x6a3b8b02(1782287106), sa_trans= esp-3 des esp-md5-hmac , sa_conn_id= 8 where the 2 messages concatenated are actually contained in 2 separate TCP packets, as per the tcpdump below. The general logic I can deduce from the debugs so far is that the log daemon on the PIX assembles a message, truncates to 255 characters, stuffs it out the log stream ( be it UDP or TCP ). If the stream is TCP, I suspect Nagle Algorithm or similar determines when sufficient data within one or more messages have been accumulate sufficient to actually send a TCP packet. As a result, whilst any given log message is limited to 255 bytes, but any given TCP packet need not be. All in all, its a mess. I'm better off leaving the syslogging routing via UDP to ensure correct EOM determination on the syslog server. Needless to say, I'll raise a log with our Cisco resellers to try and get Cisco to fix their code. The minimum 2 requirements are: a) Raise log message size limit to 512 bytes for either UDP or TCP streams b) Terminate TCP messages with NUL or NL to allow sane logservers to unsplit the mess. I suspect there is very little Balazs can do to improve the decode of the PIX log stream without Cisco fixing their broken code. Ted 15:58:58.013327 192.168.82.15.1024 > 172.17.12.19.1468: P 878:1133(255) ack 1 win 4096 4500 0127 9d42 0000 fc06 55b2 c0a8 520f E..'.B....U...R. ac11 0c13 0400 05bc 02ff 1fde 3b30 7c08 ............;0|. 5018 1000 4360 0000 3c31 3931 3e25 5049 P...C`..<191>%PI 582d 372d 3730 3233 3031 3a20 6c69 6665 X-7-702301: life 7469 6d65 2065 7870 6972 696e 672c 2028 time expiring, ( 7361 2920 7361 5f64 6573 743d 2031 3935 sa) sa_dest= 195 2e31 3838 2e31 3731 2e35 2c20 7361 5f70 .188.171.5, sa_p 726f 743d 2035 302c 2073 615f 7370 693d rot= 50, sa_spi= 2030 7836 6133 6238 6230 3228 3137 3832 0x6a3b8b02(1782 3238 3731 3036 292c 2073 615f 7472 616e 287106), sa_tran 733d 2065 7370 2d33 6465 7320 6573 702d s= esp-3des esp- 6d64 352d 686d 6163 202c 2073 615f 636f md5-hmac , sa_co 6e6e 5f69 643d 2038 2c20 2869 6465 6e74 nn_id= 8, (ident 6974 7929 206c 6f63 616c 3d20 3139 352e ity) local= 195. 3138 382e 3137 312e 352c 2072 656d 6f74 188.171.5, remot 653d 2032 3137 2e33 342e 3230 392e 3230 e= 217.34.209.20 302c 206c 6f63 616c 5f70 726f 7879 3d20 0, local_proxy= 3139 342e 3334 2e31 3938 2e34 302f 3235 194.34.198.40/25 352e 3235 352e 32 5.255.2. 15:58:58.026581 172.17.12.19.1468 > 192.168.82.15.1024: . ack 1133 win 32120 (DF ) 4500 0028 6190 4000 4006 0e64 ac11 0c13 E..(a.@.@..d.... c0a8 520f 05bc 0400 3b30 7c08 02ff 20dd ..R.....;0|... . 5010 7d78 82af 0000 P.}x.... 15:58:58.027406 192.168.82.15.1024 > 172.17.12.19.1468: P 1133:1445(312) ack 1 w in 4096 4500 0160 9d43 0000 fc06 5578 c0a8 520f E..`.C....Ux..R. ac11 0c13 0400 05bc 02ff 20dd 3b30 7c08 .......... .;0|. 5018 1000 ca1d 0000 3c31 3930 3e25 5049 P.......<190>%PI 582d 362d 3630 3233 3032 3a20 6465 6c65 X-6-602302: dele 7469 6e67 2053 412c 2028 7361 2920 7361 ting SA, (sa) sa 5f64 6573 743d 2031 3935 2e31 3838 2e31 _dest= 195.188.1 3731 2e35 2c20 7361 5f70 726f 743d 2035 71.5, sa_prot= 5 302c 2073 615f 7370 693d 2030 7836 6133 0, sa_spi= 0x6a3 6238 6230 3228 3137 3832 3238 3731 3036 b8b02(1782287106 292c 2073 615f 7472 616e 733d 2065 7370 ), sa_trans= esp 2d33 6465 7320 6573 702d 6d64 352d 686d -3des esp-md5-hm 6163 202c 2073 615f 636f 6e6e 5f69 643d ac , sa_conn_id= 2038 0a0a 3c31 3930 3e25 5049 582d 362d 8..<190>%PIX-6- 3630 3233 3032 3a20 6465 6c65 7469 6e67 602302: deleting 2053 412c 2028 7361 2920 7361 5f64 6573 SA, (sa) sa_des 743d 2032 3137 2e33 342e 3230 392e 3230 t= 217.34.209.20 302c 2073 615f 7072 6f74 3d20 3530 2c20 0, sa_prot= 50, 7361 5f73 7069 3d20 3078 3137 6563 3539 sa_spi= 0x17ec59 3335 2834 3031 3336 3533 3031 292c 2073 35(401365301), s 615f 7472 616e 733d 2065 7370 2d33 6465 a_trans= esp-3de 7320 6573 702d 6d64 352d 686d 6163 202c s esp-md5-hmac , 2073 615f 636f 6e6e 5f69 643d 2037 0a0a sa_conn_id= 7.. ************************************************************************************************ This E-mail message, including any attachments, is intended only for the person or entity to which it is addressed, and may contain confidential information. If you are not the intended recipient, any review, retransmission, disclosure, copying, modification or other use of this E-mail message or attachments is strictly forbidden. If you have received this E-mail message in error, please contact the author and delete the message and any attachments from your computer. You are also advised that the views and opinions expressed in this E-mail message and any attachments are the author's own, and may not reflect the views and opinions of FLEXTECH Television Limited. ************************************************************************************************

1 0

syslog-ng-1.6.0rc3 - problem with incorrect separation of syslog messages from Cisco PIX
by email lists 11 May '03

11 May '03

Hi, We have upgraded to 1.6.0 RC3 for program() template support (ie - pipe directly to stdin of psql). Since the upgrade, we have also had a requirement to enable PIX URL logging and IPSec ISAKMP SA logging. As a consequence, we have found numerous syslog entries where the record following the PIX URL log entry is merged with the URL log entry. Target system - redhat 8.0 / kernel-2.4.18-26.8.0 / glibc-2.2.93-5 / gcc-3.2-7 with libol 0.3.9. Standard syslog to file: May 10 11:51:02 192.168.100.252 May 10 2003 01:51:02: %PIX-7-702301: lifetime expiring, (sa) sa_dest= 10.0.0.1, sa_prot= 50, sa_spi= 0xe859748d(3898176653), sa_trans= esp-3des esp-md5-hmac , sa_conn_id= 1, (identity) local= 10.0.0.1, remote= 10.1.1.1, local_proxy= 192<166>May 10 2003 01:51:02: %PIX-6-602302: deleting SA, (sa) sa_dest= 10.0.0.1, sa_prot= 50, sa_spi= 0xe859748d(3898176653), sa_trans= esp-3des esp-md5-hmac , sa_conn_id= 1 The same syslog message SQL formated to file - same template as program(): BEGIN; INSERT INTO rawlogs (host, datetime, facility, priority, level, tag, program, msg) VALUES ('192.168.100.252', '2003-05-10 11:51:02', 'local4', 'debug', 'debug', 'a7', 'May', 'May 10 2003 01:51:02: %PIX-7-702301: lifetime expiring, (sa) sa_dest= 10.0.0.1, sa_prot= 50, sa_spi= 0xe859748d(3898176653), sa_trans= esp-3des esp-md5-hmac , sa_conn_id= 1, (identity) local= 10.0.0.1, remote= 10.1.1.1, local_proxy= 192<166>May 10 2003 01:51:02: %PIX-6-602302: deleting SA, (sa) sa_dest= 10.0.0.1, sa_prot= 50, sa_spi= 0xe859748d(3898176653), sa_trans= esp-3des esp-md5-hmac , sa_conn_id= 1'); COMMIT; TCP logging is used (too many messages lost when using UDP). There are also other instances where syslog messages have been merged together. More samples of the above can be supplied if necessary if a direct e-mail address of a developer is provided - I am reluctant to send such data to the list. IP addresses containe in the log entries have also been changed. TIA, Darren syslog-ng.conf that generated the above data: options { sync (0); time_reopen (10); log_fifo_size (8192); long_hostnames (off); use_dns (no); use_fqdn (no); use_time_recvd (yes); create_dirs (yes); keep_hostname (yes); perm (0644); owner (root); group (root); dir_perm (0644); dir_owner (root); dir_group (root); }; source s_tcp { tcp(port(10514) max-connections(64)); }; source s_udp { udp(port(514)); }; destination rawsyslog { file("/monitor/syslog-ng/data/rawsyslog-$YEAR-$MONTH-$DAY"); }; destination syslog2pgsql { file("/monitor/syslog-ng/data/syslog2pgsql-$YEAR-$MONTH-$DAY" template("BEGIN; INSERT INTO rawlogs (host, datetime, facility, priority, level, tag, program, msg) VALUES '$HOST', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$PROGRAM', '$MSG'); COMMIT;\n") template-escape(yes)); }; log { source(s_udp); destination(syslog2pgsql); }; log { source(s_tcp); destination(syslog2pgsql); }; log { source(s_udp); destination(rawsyslog); }; log { source(s_tcp); destination(rawsyslog); };

2 1

FW: syslog-ng-1.6.0rc3 - problem with incorrect separation of syslog messages from Cisco PIX
by email lists 11 May '03

11 May '03

Just realised the example tha were provided only contain IPSec SA data - apoologies, but I think you get the idea. Darten -----Original Message----- From: email lists Sent: Sunday, 11 May 2003 6:06 PM To: 'syslog-ng(a)lists.balabit.hu' Subject: syslog-ng-1.6.0rc3 - problem with incorrect separation of syslog messages from Cisco PIX <snip> May 10 11:51:02 192.168.100.252 May 10 2003 01:51:02: %PIX-7-702301: lifetime expiring, (sa) sa_dest= 10.0.0.1, sa_prot= 50, sa_spi= 0xe859748d(3898176653), sa_trans= esp-3des esp-md5-hmac , sa_conn_id= 1, (identity) local= 10.0.0.1, remote= 10.1.1.1, local_proxy= 192<166>May 10 2003 01:51:02: %PIX-6-602302: deleting SA, (sa) sa_dest= 10.0.0.1, sa_prot= 50, sa_spi= 0xe859748d(3898176653), sa_trans= esp-3des esp-md5-hmac , sa_conn_id= 1 The same syslog message SQL formated to file - same template as program(): BEGIN; INSERT INTO rawlogs (host, datetime, facility, priority, level, tag, program, msg) VALUES ('192.168.100.252', '2003-05-10 11:51:02', 'local4', 'debug', 'debug', 'a7', 'May', 'May 10 2003 01:51:02: %PIX-7-702301: lifetime expiring, (sa) sa_dest= 10.0.0.1, sa_prot= 50, sa_spi= 0xe859748d(3898176653), sa_trans= esp-3des esp-md5-hmac , sa_conn_id= 1, (identity) local= 10.0.0.1, remote= 10.1.1.1, local_proxy= 192<166>May 10 2003 01:51:02: %PIX-6-602302: deleting SA, (sa) sa_dest= 10.0.0.1, sa_prot= 50, sa_spi= 0xe859748d(3898176653), sa_trans= esp-3des esp-md5-hmac , sa_conn_id= 1'); COMMIT; <snip>

1 0

RE: [syslog-ng]syslog-ng blanks in Servicename
by Lixin Liu 09 May '03

09 May '03

Is it possible to DNS to verify the valid hostnames? Our syslog produces many bogus hostnames and it is almost impossible to list all of them in bad_hostname(). Thanks, Lixin. -----Original Message----- From: syslog-ng-admin(a)lists.balabit.hu [mailto:syslog-ng-admin@lists.balabit.hu] On Behalf Of Balazs Scheidler Sent: Friday, May 09, 2003 9:42 AM To: syslog-ng(a)lists.balabit.hu Subject: Re: [syslog-ng]syslog-ng blanks in Servicename On Fri, May 09, 2003 at 10:24:22AM -0600, markallen wrote: > I recently read a thread that described an option you can set so your > not logging blanks in servernames. I seen where I can set the option > to ignore bogus hostnames but how do you ignore bogus hostnames that > start with a tab? > > This is what I'm currently using: > > options { > bad_hostname("snmp|AFSR|AFSR.PSYND|Delete|In|UDBH|Use|ready32"); }; > > I would like to exclude everything that starts with a tab as well. > Does anyone have any suggestions? options { bad_hostname("\t"); }; here's the set of special characters you can use (quoting cfg-lex.l): <string>\\a { append_char(7); } <string>\\n { append_char(10); } <string>\\r { append_char(13); } <string>\\t { append_char(9); } <string>\\v { append_char(11); } e.g \a, \n, \r, \t and \v are all supported special characters (similarly to C) -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

syslog-ng blanks in Servicename
by markallen 09 May '03

09 May '03

I recently read a thread that described an option you can set so your not logging blanks in servernames. I seen where I can set the option to ignore bogus hostnames but how do you ignore bogus hostnames that start with a tab? This is what I'm currently using: options { bad_hostname("snmp|AFSR|AFSR.PSYND|Delete|In|UDBH|Use|ready32"); }; I would like to exclude everything that starts with a tab as well. Does anyone have any suggestions? Mark Allen

2 1

Match host with IP address when using use_dns?
by Marco P. Rodrigues 09 May '03

09 May '03

Couldn't seem to find an answer to this in the documentation. Is it possible to the have the use_dns(yes) option enabled but match hosts based on their IP address? I want to be able to write files as their FQDN but fo the REGEX matching based on the IP address. I can get this to work when I disable the use_dns(yes) option but then the files are the IP. I don't want to match based on hostnames. example: destination net_host_all { file("/tmp/$HOST"); }; filter f_hosts_all { host("^10\.201\.140\.*") or host("^10\.201\.150\.*") or host("^192\.168\.0\.[0-3][0-9]"); }; log { source(net); filter(f_hosts_all); destination(net_host_all); flags(final); }; While I'm hear might as well kill two birds with one stone. Can someone give ma a real life example of the practical use of using the fallback option? I must be inept because I can't see a valid reason to have it that the final/catchall statement couldn't do (with proper logical placements of the log statements in the configuration). MPR. -- "I am kind of a paranoiac in reverse. I suspect people of plotting to make me happy." - J.D. Salinger

2 1

Problem with chained hostnames..
by Michael Boman 09 May '03

09 May '03

I have tried a series of keep_hostname() and chain_hostnames() settings, but I still won't get it as logrelay/server Either I get it as "logrelay" or as "server". I want this as I want a dir structure like: /var/log/HOSTS/$relay/$server/....... What combination should I have on the: Relay? Final destination (ie: the log server)? Best regards Michael Boman -- Michael Boman Security Architect, SecureCiRT Pte Ltd http://www.securecirt.com

1 0

(no subject)
by Marco P. Rodrigues 08 May '03

08 May '03

Couldn't seem to find an answer to this in the documentation. Is it possible to the have the use_dns(yes) option enabled but match hosts based on their IP address? I want to be able to write files as their FQDN but fo the REGEX matching based on the IP address. I can get this to work when I disable the use_dns(yes) option but then the files are the IP. I don't want to match based on hostnames. example: destination net_host_all { file("/tmp/$HOST"); }; filter f_hosts_all { host("^10\.201\.140\.*") or host("^10\.201\.150\.*") or host("^192\.168\.0\.[0-3][0-9]"); }; log { source(net); filter(f_hosts_all); destination(net_host_all); flags(final); }; While I'm hear might as well kill two birds with one stone. Can someone give ma a real life example of the practical use of using the fallback option? I must be inept because I can't see a valid reason to have it that the final/catchall statement couldn't do (with proper logical placements of the log statements in the configuration). MPR. -- "I am kind of a paranoiac in reverse. I suspect people of plotting to make me happy." - J.D. Salinger

1 0

xinet.d
by sjm 08 May '03

08 May '03

Could someone please post the xinet.d script they use with syslog-ng? Thanks for you time, -- /*-----------------------------*\ | | | Steve McKinney | | ARDI - Web Programmer | | sjm(a)porter.appstate.edu | | (828) 262-6553 | | | \*-----------------------------*/

1 0

Problem getting init logs
by arthur.chereau 08 May '03

08 May '03

Hi, I've installed syslog-ng and it is working fine. There's just one remaining problem I can't solve: I use monit, which is started from inittab. Logs go to the file I defined. But they also go to the console (tty1). I tried to remove all syslog-ng entries sending logs to usertty (alert and emerg) and restart, but it didn't change anything. So the problem must be that init logs are different from other logs. Is it possible for syslog-ng to intercept logs sent by init to the console ? ------------------------------------------ Faites un voeu et puis Voila ! www.voila.fr

3 2