February 2007 - syslog-ng - lists.balabit.hu

AIX startup
by Brian Loe 27 Mar '08

27 Mar '08

Okay, my syslog-ng instance died at some point last night and because I don't have it setup properly for respawning... Can anyone running syslog-ng on AIX 5.* give me some tips for this? I'm feeding pipes so i have to start instances of SEC as soon as I start syslog-ng - but the SEC scripts appear to "close" so if they're configured to respawn I'll have 2000 instances of them in an hour.

7 21

incorrect firewall logs with syslogng-2.0.0 [security error?]
by Jacek Kalinski 12 Mar '08

12 Mar '08

Hello, After upgradeing from syslogng-1.6.11 to 2.0.0 I've got a strange messages in /var/log/messages file: Nov 29 08:14:05 denise WINDOW=16985 RES=0x00 ACK URGP=0 Nov 29 11:10:19 denise PT=34536 WINDOW=16985 RES=0x00 ACK URGP=0 Nov 29 16:42:16 denise W=0 RES=0x00 RST URGP=0 Nov 29 21:37:15 denise 116 ID=34901 DF PROTO=TCP SPT=3584 DPT=80 WINDOW=65535 RES=0x00 ACK FIN URGP=0 These are truncated iptables logs. Because in firewall.log file are simmilar entries to truncated one, I think full file should be: Nov 29 11:10:19 denise firewallp=INVALID:1 a=DROP IN=eth1 OUT= MAC=00:30:4f:36:2b:dc:00:04:9a:2c:7f:20:08:00 SRC=193.41.230.81 DST=xxx.xxx.xxx.x LEN=576 TOS=0x00 PREC=0x00 TTL=117 ID=3495 PROTO=TCP SPT=443 DPT=34536 WINDOW=16985 RES=0x00 ACK URGP=0 (xxx.xxx.xxx.x is a real server IP address - one 'x' is one digit) Config file with interesting lines: options { dir_perm(0700); perm(0600); chain_hostnames(no); stats(1200); keep_hostname(yes); dir_owner(root); dir_group(root); owner(root); group(root); }; source local { unix-stream("/dev/log" max_connections(300)); file("/proc/kmsg"); # udp(ip(0.0.0.0) port(514)); internal(); }; # ..... # authpriv.none;cron.none;mail.none;news.none -/var/log/messages filter f_2 { not facility(news); }; filter f_3 { not facility(local5); }; filter f_4 { not facility(local6); }; filter f_5 { not facility(authpriv); }; filter f_6 { not facility(cron); }; filter f_7 { not facility(mail); }; filter f_notlocal7 { not facility(local7); }; filter f_notfirewall { not match("firewallp="); }; destination d_2 { file("/var/log/messages" create_dirs(yes)); }; log { source(local); filter(f_2); filter(f_4); filter(f_5); filter(f_6); filter(f_7); filter(f_notimapd); filter(f_notpopa3d); filter(f_notnamed); filter(f_notfirewall); destination(d_2); }; #..... # firewallp.* -/var/log/firewall.log filter f_16 { level(warning) and match("firewallp="); }; destination d_12 { file("/var/log/firewall.log" create_dirs(yes)); }; log { source(local); filter(f_16); destination(d_12); }; #..... Do you know where could be an error? It is a production server so I can't debug syslog-ng on this machine (sorry). NOTE: the only truncated lines I can find are iptables (AFAIR they are logged by /proc/kmesg) entries. All other lines are complete. And it is probably a critical error in security. PS: simple statistics line from syslog-ng: Nov 30 00:00:06 denise syslog-ng[32682]: Log statistics; dropped='pipe(/dev/tty12)=0', processed='center(queued)=758289', processed='center(received)=369020', processed='destination(d_maillogimapd)=4245', processed='destination(d_5)=1', processed='destination(d_4)=0', processed='destination(d_3)=14259', processed='destination(d_2)=14259', processed='destination(d_13)=0', processed='destination(d_1)=6000', processed='destination(d_12)=6000', processed='destination(d_11)=369020', processed='destination(d_10)=36', processed='destination(d_8)=0', processed='destination(d_maillogpopa3d)=103224', processed='destination(d_7)=241245', processed='destination(d_6)=0', processed='source(local)=369020' I think you see that it could be a little hard to debug (with so many messages parsed by syslog-ng) ;) Jacek

3 6

Disconnected access/tailing real files/web interface
by J Bridge 06 Apr '07

06 Apr '07

Hello. I would like to centralise logs from many different hosts into one place. It's possible that the network connectivity between the log server and the hosts is not reliable. Therefore: 1. Can I get syslog-ng to send new log entries once it can see the central log server again? I would also like to centralise logs from non-syslog programs such as some big database apps. 2. Can syslog-ng work similarly to tail -f logging to the central server? And finally (but least importantly).. 3. Does anyone know a good web interface for viewing or searching the files from a web interface? The logs must remain as files, not be put in a mysql database. Thanks, jbj

2 2

Growing log files (i.e. access.log) and regex.
by Robert.Becker＠motoristsgroup.com 08 Mar '07

08 Mar '07

Here's the scenario. We have a JBoss server that outputs to *.log files. After doing all the research I could, I found out the only way to get these logs sent to syslog-ng was through a tail & logger command. The problem with that is logger appends the date and some other information to that logged message. When attempting to put that information in a mysql database I'm getting errors. So, what I would like to do is use some form of regex to alter the statement so it can be inserted into a database. There are basically two things that need to be changed. First, logger appends the date/time it receives the message and it also says it is coming from logger. I would like to remove that date and switch logger to jboss or something similar. Here is an example of what I am doing. This is a line from the jboss log file: 2005-10-11 13:14:15,848 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 Then I send the log through logger to syslog-ng: #!/bin/sh tail -f /opt/jboss/server/default/log/boot.log | logger -p local7.info The log gets turned into this: Oct 11 13:14:15 src@linuxp17 logger: 2005-10-11 13:14:15,848 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 What I need is: Oct 11 13:14:15 src@linuxp17 jboss: DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 Now, I've thought of using awk in the tail/logger script to remove the date, but that does not take care of the logger/jboss substitution. Anyone have any ideas? Can syslog-ng do this with the match() function? -Rob Becker Systems Engineer Motorists Insurance ********************************************************************** The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error or there are any problems please notify the originator immediately. The unauthorized use, disclosure, copying or alteration of this message is strictly forbidden. Motorists Insurance Group will not be liable for direct, special, indirect or consequential damages arising from the alteration of the contents of this message by a third party or as a result of any virus being passed on. **********************************************************************

6 5

compilation errors with --enable-spoof-source
by Ravi Papisetti -X (rpapiset - HCL＠Cisco) 06 Mar '07

06 Mar '07

Hi, I am using syslog-ng 1.6.11 and trying to compile this package with ./configure --enable-spoof-source, it throws errors as below checking whether to enable Sun STREAMS support... yes checking whether to enable Sun door support... yes checking whether to enable TCP wrapper support... no checking whether to enable spoof_source support... ./configure: test: too many arguments configure: error: libnet-config not found It compiles fine without --enable-spoof-source this option. Could you do the needful. Thanks, Ravi Kumar P.

3 11

Incorrect message: SIGTERM received
by bryanh＠giraffe-data.com 05 Mar '07

05 Mar '07

If you send a SIGINT signal to Syslog-ng, it issues the message SIGTERM received, terminating Looking at the code, it uses the same code for both SIGTERM and SIGINT. I suggest the message be reworded as SIGTERM or SIGINT received, terminating -- Bryan Henderson San Jose, California

2 2

afinet.c:: Compilation help needed on HP-UX 11i
by Shamim 02 Mar '07

02 Mar '07

afunix.c: In function 'afunix_sd_init': afunix.c:73: warning: comparison is always true due to limited range of data type if gcc -DHAVE_CONFIG_H -I. -I. -I.. -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/usr/local/include/eventlog -D_GNU_SOURCE -g -O2 -Wall -g -MT afinet.o -MD -MP -MF ".deps/afinet.Tpo" \ -c -o afinet.o `test -f 'afinet.c' || echo './'`afinet.c; \ then mv -f ".deps/afinet.Tpo" ".deps/afinet.Po"; \ else rm -f ".deps/afinet.Tpo"; exit 1; \ fi afinet.c: In function 'afinet_setup_socket': afinet.c:156: error: storage size of 'mreq' isn't known afinet.c:156: warning: unused variable 'mreq' *** Error exit code 1 Stop. *** Error exit code 1 Stop. *** Error exit code 1 Stop. # uname -a HP-UX wlggh010 B.11.23 U 9000/785 2006586620 unlimited-user license # Packages installed on my Machine are # swlist |grep gcc gcc 4.1.1 gcc libgcc 4.1.1 libgcc # swlist |grep glib glib 1.2.10 glib glib2 2.12.8 glib2 # --------------------------------- New Yahoo! Mail is the ultimate force in competitive emailing. Find out more at the Yahoo! Mail Championships. Plus: play games and win prizes.

2 9

MySQL syslog-ng regex
by Fat Wallet 01 Mar '07

01 Mar '07

Hi, I am having problem inserting logs into MySQL using match() that returns more than 10 arguements. destination d_mysql { pipe("/tmp/mysql.pipe" template("INSERT INTO mydatabase (content_name) VALUES ('$10');\n") template-escape(yes)); }; filter f_filter { match ("^.*?: \"(.*)\",\"(.*)\",\"(.*)\",\"(.*)\",\"(.*)\",\"(.*)\",\"(.*)\",\"(.*)\",\"(.*)\",\"(.*)$"); }; log { source(s_stunnel); filter(f_filter); destination(d_mysql); }; incoming log message is as followed: "test1","test2","test3","test4","test5","test6","test7","test8","test9","testN","testN1" catting mysql.pipe: $>cat /tmp/mysql.pipe INSERT INTO mydatabase (content_name) VALUES ('test10'); So instead of 'testN', I am getting the 'test1' appends with 0 which is 'test10' Thanks for the help! ____________________________________________________________________________________ Any questions? Get answers on any topic at www.Answers.yahoo.com. Try it now.

4 4

Observation of syslog-ng on HP-UX
by Shamim 28 Feb '07

28 Feb '07

Hi, Thanks for the help. It works now. My Observation are for HP-UX : 1. If both syslog and syslog-ng daemons are running on the system, then message sent through logger are being captured by syslog daemon only. 2. While in other OS like, AIX, Linux, Solaris message goes to both the file defined in syslog.conf and syslog-ng.conf. 3. So to captured all message on HP-UX to syslog-ng, I need to stop HP-UX's syslog daemon? Please any further comment on my Observation. Am I correct? Thanks and Regards Shamimuddin CSC India olivier rolland <madmax2010fr(a)yahoo.fr> wrote: Hi You can use logger on HP-UX using priority and Tag option For exemple logger -p local4.info -t WHATIHAVETOCATCH "My message " and you create a match filter like filter f_what {match (WHATIHAVETOCATCH);}; filter f_infolocal { level(info) and facility(local4);}; destination d_what { file(/place/fileyouwant.log);}; log { source(s_sys) ; filter(f_what); filter(infolocal) ; destination(d_what);}; Now you catch your regexp and you log messages into the file that you want. Shamim a écrit : > Hi > > Please can anyone help me how to log message to syslog-ng manually. > e.g. in Linux I can log message as # logger "test message" > ; and it goes to file defined in > syslog-ng.conf(/var/adm/messages), > > However in case of HP-UX it is not working as describe above for > Linux. Is there any other method to log message manually in HP-UX? > Or messages are dropping? > > Please suggest. > > Thanks > Shamimuddin > CSC Noida > > */olivier rolland /* wrote: > > You have to put the log device with the pad_size into your system > source. > source s_sys {pipe("/dev/log" pad_size(2048)); > internal(); }; > It's working on my system so you can try. > You can also add the klog device (/dev/klog) with same options than > /dev/log. > > Shamim a écrit : > > > > Hi, > > I've compile syslog-ng-2.0.2 on HP-UX-11.11, however messages > are not > > going to desired destinations as defined in the syslog-ng.conf . > > > > my syslog-ng.conf > > --------------------------------------------- > > # syslog-ng configuration file. > > # > > # This should behave pretty much like the original syslog on > HP-UX. But > > # it could be configured a lot smarter. > > # > > # See syslog-ng(8) and syslog-ng.conf(8) for more information. > > # > > # 20000925 gb(a)sysfive.com > > options { sync (0); > > time_reopen (10); > > log_fifo_size (1000); > > long_hostnames (off); > > use_dns (no); > > use_fqdn (no); > > create_dirs (no); > > keep_hostname (yes); > > }; > > source s_sys {internal();pipe("/dev/log"); }; > > destination d_cons { file("/dev/console1"); }; > > destination d_mesg { file("/var/adm/syslog/syslog-ng.log"); }; > > destination d_mail { file("/var/adm/syslog/mail-ng.log"); }; > > destination d_mlrt { usertty("root"); }; > > destination d_mlal { usertty("*"); }; > > filter f_filter1 { facility(mail) and level(debug); }; > > filter f_filter2 { (facility(mail) and level(debug)) or > > level(info); }; > > filter f_filter3 { level(alert); }; > > filter f_filter4 { level(emerg); }; > > log { source(s_sys); destination(d_mail); }; > > log { source(s_sys); destination(d_mesg); }; > > log { source(s_sys); destination(d_cons);destination(d_mlrt); }; > > log { source(s_sys); destination(d_mlal); }; > > # > > -------------------------------------------- > > Now If I generate message using "logger" utility on system, message > > should go to "/var/adm/syslog/syslog-ng.log", though they are > not going... > > Is there anything wrong with .conf file? > > > > What should be the conf file for central server and client on HP-UX? > > > > > > syslog-ng daemon is running like: > > > ------------------------------------------------------------------------------- > > syslog-ng service starting. > > # ps -eaf |grep syslog-ng > > root 14437 1 0 10:22:30 ? 0:00 > > /opt/soe/local/syslog-ng-2.0.2/sbin/syslog-ng -f /opt/soe/local > > # > > > ------------------------------------------------------------------------------ > > > > Thanks > > Shamim --------------------------------- What kind of emailer are you? Find out today - get a free analysis of your email personality. Take the quiz at the Yahoo! Mail Championship.

2 1

syslog-ng.conf problem on HP-UX-11.11
by Shamim 26 Feb '07

26 Feb '07

Hi, I've compile syslog-ng-2.0.2 on HP-UX-11.11, however messages are not going to desired destinations as defined in the syslog-ng.conf . my syslog-ng.conf --------------------------------------------- # syslog-ng configuration file. # # This should behave pretty much like the original syslog on HP-UX. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(8) for more information. # # 20000925 gb(a)sysfive.com options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys {internal();pipe("/dev/log"); }; destination d_cons { file("/dev/console1"); }; destination d_mesg { file("/var/adm/syslog/syslog-ng.log"); }; destination d_mail { file("/var/adm/syslog/mail-ng.log"); }; destination d_mlrt { usertty("root"); }; destination d_mlal { usertty("*"); }; filter f_filter1 { facility(mail) and level(debug); }; filter f_filter2 { (facility(mail) and level(debug)) or level(info); }; filter f_filter3 { level(alert); }; filter f_filter4 { level(emerg); }; log { source(s_sys); destination(d_mail); }; log { source(s_sys); destination(d_mesg); }; log { source(s_sys); destination(d_cons);destination(d_mlrt); }; log { source(s_sys); destination(d_mlal); }; # -------------------------------------------- Now If I generate message using "logger" utility on system, message should go to "/var/adm/syslog/syslog-ng.log", though they are not going... Is there anything wrong with .conf file? What should be the conf file for central server and client on HP-UX? syslog-ng daemon is running like: ------------------------------------------------------------------------------- syslog-ng service starting. # ps -eaf |grep syslog-ng root 14437 1 0 10:22:30 ? 0:00 /opt/soe/local/syslog-ng-2.0.2/sbin/syslog-ng -f /opt/soe/local # ------------------------------------------------------------------------------ Thanks Shamim syslog-ng-request(a)lists.balabit.hu wrote: Send syslog-ng mailing list submissions to syslog-ng(a)lists.balabit.hu To subscribe or unsubscribe via the World Wide Web, visit https://lists.balabit.hu/mailman/listinfo/syslog-ng or, via email, send a message with subject or body 'help' to syslog-ng-request(a)lists.balabit.hu You can reach the person managing the list at syslog-ng-owner(a)lists.balabit.hu When replying, please edit your Subject line so it is more specific than "Re: Contents of syslog-ng digest..." Today's Topics: 1. Re: Is this possible in syslog-ng.conf . (v2.0.2) (Balazs Scheidler) 2. Re: Setting permissions on log files (Balazs Scheidler) 3. Re: Is this possible in syslog-ng.conf . (v2.0.2) (Evan Rempel) 4. compilation errors with --enable-spoof-source (Ravi Papisetti -X (rpapiset - HCL at Cisco)) 5. Re: Is this possible in syslog-ng.conf . (v2.0.2) (Balazs Scheidler) 6. Re: compilation errors with --enable-spoof-source (Balazs Scheidler) 7. RE: compilation errors with --enable-spoof-source (Ravi Papisetti -X (rpapiset - HCL at Cisco)) 8. RE: compilation errors with --enable-spoof-source (Balazs Scheidler) ---------------------------------------------------------------------- Message: 1 Date: Mon, 19 Feb 2007 17:14:12 +0100 From: Balazs Scheidler Subject: Re: [syslog-ng] Is this possible in syslog-ng.conf . (v2.0.2) To: Syslog-ng users' and developers' mailing list Message-ID: <1171901652.11781.7.camel(a)bzorp.balabit> Content-Type: text/plain On Sun, 2007-02-18 at 16:48 -0800, Mr. James W. Laferriere wrote: > Ok . IMO counter intuitive , Tho reasonable with your explanation . > One is very used to the 'source' in FW/router/...'s as being the source > device(s) IP from where a packet came from . syslog-ng is not a firewall :) this is sometimes strange to me as well, being involved in firewall products as well. But putting the joke aside, syslog-ng is a "syslog message pipe" processor: sources generate messages, destinations serve as message sinks. Some filtering here and there, that's about syslog-ng's internal structure. So, naming source as a source is consistent with syslog-ng itself. > An aside , Can one do the 'Formatting' like my example above , again > no examples show up like that , but I am hopeful . > > > > To do that you need the netmask() filter. > > Next time I'll go looking at the Blog at Gmane first before shooting my > mouth off . netmask was just the hint I needed . > Tho it sure would be nice for netmask() to support the /xx bits netmask > format . It does support this format. -- Bazsi ------------------------------ Message: 2 Date: Mon, 19 Feb 2007 17:15:57 +0100 From: Balazs Scheidler Subject: Re: [syslog-ng] Setting permissions on log files To: Syslog-ng users' and developers' mailing list Message-ID: <1171901757.11781.10.camel(a)bzorp.balabit> Content-Type: text/plain On Mon, 2007-02-19 at 01:45 +0000, Bryan Henderson wrote: > With the 'file' destination, Syslog-ng modifies the ownership and > permissions of the file when it opens it. There are configuration > file options to choose what it sets them to, but AFAICT, no way to > have Syslog-ng just leave the files as it finds them. > > I prefer to maintain permissions and ownership separately; I set them > when I create the file and expect them to stick. Could there be a > configuration file option for that? > IIRC, you can use -1 for various options, which means "do not touch". But I would need to test it. The code in the C part is there, the only question that remains whether the parser accepts "-1" in the place of owner/group/permissions. -- Bazsi ------------------------------ Message: 3 Date: Mon, 19 Feb 2007 09:02:15 -0800 From: Evan Rempel Subject: Re: [syslog-ng] Is this possible in syslog-ng.conf . (v2.0.2) To: "Syslog-ng users' and developers' mailing list" Message-ID: <45D9D817.7050309(a)uvic.ca> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Balazs Scheidler wrote: > On Sun, 2007-02-18 at 16:48 -0800, Mr. James W. Laferriere wrote: > >> Ok . IMO counter intuitive , Tho reasonable with your explanation . >> One is very used to the 'source' in FW/router/...'s as being the source >> device(s) IP from where a packet came from . > > syslog-ng is not a firewall :) this is sometimes strange to me as well, > being involved in firewall products as well. But putting the joke aside, > syslog-ng is a "syslog message pipe" processor: sources generate > messages, destinations serve as message sinks. Some filtering here and > there, that's about syslog-ng's internal structure. > > So, naming source as a source is consistent with syslog-ng itself. I think that the author of the original comment was refereing to the IP address binding in the source definition source network { tcp( ip(xxxx) ); }; where the IP address is NOT the source at all, it is a local IP address to bind the listener to. Perhaps the syntax should be source network { tcp( bind(xxxx) ); }; since the bind address MUST be ip since the definition is already defined to be tcp. I think it is a little counter intuitive even within the scope of syslog-ng. Evan. > >> An aside , Can one do the 'Formatting' like my example above , again >> no examples show up like that , but I am hopeful . >> >> >>> To do that you need the netmask() filter. >> Next time I'll go looking at the Blog at Gmane first before shooting my >> mouth off . netmask was just the hint I needed . >> Tho it sure would be nice for netmask() to support the /xx bits netmask >> format . > > It does support this format. > -- Evan Rempel erempel(a)uvic.ca Senior Programmer Analyst 250.721.7691 Computing Services University of Victoria ------------------------------ Message: 4 Date: Mon, 19 Feb 2007 13:09:41 -0600 From: "Ravi Papisetti -X (rpapiset - HCL at Cisco)" Subject: [syslog-ng] compilation errors with --enable-spoof-source To: "syslog-ng(a)lists.balabit.hu" Message-ID: <5A8F8213-CAC5-4190-A902-FE91C0DC844D@mimectl> Content-Type: text/plain; charset="iso-8859-1" Hi, I am using syslog-ng 1.6.11 and trying to compile this package with ./configure --enable-spoof-source, it throws errors as below checking whether to enable Sun STREAMS support... yes checking whether to enable Sun door support... yes checking whether to enable TCP wrapper support... no checking whether to enable spoof_source support... ./configure: test: too many arguments configure: error: libnet-config not found It compiles fine without --enable-spoof-source this option. Could you do the needful. Thanks, Ravi Kumar P.

4 4