September 2008 - syslog-ng - lists.balabit.hu

Problem with syslog-ng 1.6.8 and HUP
by Tessier.Marc.2＠hydro.qc.ca 10 Sep '08

10 Sep '08

Hi, I have been working on an issue for the last few days with one of our main centralized syslog servers which is a SPARC Solaris 9 box. It is running syslog-ng 1.6.8. Our log rotation script moves, compresses and then touches a new file depending on size. Then script then sends an HUP to syslog-ng in the following way : /usr/bin/pkill -HUP syslog-ng > /dev/null 2>&1 I have added some debugging to the script to make sure it grabs the proper PID, and logs it independantly : Sep 10 07:58:01 machine01 syslogmgr[8304]: info: signaling Syslog/Syslog-ng (HUP) at PID 4362. And it is : root@spdmze21# ps -ef | grep syslog-ng root 4362 1 2 Sep 07 ? 160:38 /usr/sbin/syslog-ng Now in the log files, it seems syslog-ng isn't always reacting to the kill. This is what I get as far as syslog-ng noticing the signal in the last 2 days : root@machine01# grep syslog-ng complete.log | grep restarting Sep 9 09:58:06 machine01 syslog-ng[4362]: SIGHUP received, restarting syslog-ng Sep 9 15:58:03 machine01 syslog-ng[4362]: SIGHUP received, restarting syslog-ng Sep 9 19:58:07 machine01 syslog-ng[4362]: SIGHUP received, restarting syslog-ng Sep 10 01:58:04 machine01 syslog-ng[4362]: SIGHUP received, restarting syslog-ng Sep 10 03:58:05 machine01 syslog-ng[4362]: SIGHUP received, restarting syslog-ng However, it was signaled everyhour : Sep 10 00:58:02 machine01 syslogmgr[20532]: [ID 702911 daemon.info] info: signaling Syslog/Syslog-ng (HUP) at PID 4362. Sep 10 01:58:04 machine01 syslogmgr[26686]: [ID 702911 daemon.info] info: signaling Syslog/Syslog-ng (HUP) at PID 4362. Sep 10 02:58:02 machine01 syslogmgr[3702]: [ID 702911 daemon.info] info: signaling Syslog/Syslog-ng (HUP) at PID 4362. Sep 10 03:58:05 machine01 syslogmgr[9935]: [ID 702911 daemon.info] info: signaling Syslog/Syslog-ng (HUP) at PID 4362. Sep 10 04:58:04 machine01 syslogmgr[17262]: [ID 702911 daemon.info] info: signaling Syslog/Syslog-ng (HUP) at PID 4362. Sep 10 05:58:02 machine01 syslogmgr[24178]: [ID 702911 daemon.info] info: signaling Syslog/Syslog-ng (HUP) at PID 4362. Sep 10 06:58:02 machine01 syslogmgr[1748]: [ID 702911 daemon.info] info: signaling Syslog/Syslog-ng (HUP) at PID 4362. Sep 10 07:58:01 machine01 syslogmgr[8304]: [ID 702911 daemon.info] info: signaling Syslog/Syslog-ng (HUP) at PID 4362. Now the problem this creates for me is that it leaves a bunch of open file descriptors which don't point to a file anymore : root@machine01# lsof -p 4362 -s | grep d80 syslog-ng 4362 root 7w VREG 85,80 9699746 40 /var/adm/log (/dev/md/dsk/d80) syslog-ng 4362 root 17w VREG 85,80 3223056 500412 /var/adm/log (/dev/md/dsk/d80) syslog-ng 4362 root 18w VREG 85,80 11604669 612873 /var/adm/log (/dev/md/dsk/d80) syslog-ng 4362 root 20w VREG 85,80 8617243 90 /var/adm/log (/dev/md/dsk/d80) syslog-ng 4362 root 23w VREG 85,80 11698446 612889 /var/adm/log (/dev/md/dsk/d80) ... This causes our rotation to mess up and thus the filesystem to eventually become filled. If I fire pkill manually, it works : root@machine01# /usr/bin/pkill -HUP syslog-ng > /dev/null 2>&1 root@machine01# lsof -p 4362 -s | grep d80 root@machine01# The script itself is called every hour from root's crontab. At this point, I am completely stumped. Even looking at the changelog from 1.6.8 to 1.6.12 (latest 1.6 tree version) I don't see anything that addresses this issue. Now since I have not been able to reproduce a non-working HUP, I don't have a truss output of when it happens. Thank you for any help! Marc Tessier Analyste Support Technique Support Technique des Infrastructures courriel : tessier.marc.2(a)hydro.qc.ca <mailto:tessier.marc.2@hydro.qc.ca> Tel: (514) 840-3000 Poste #6467

2 2

Re: [syslog-ng] Syslog-ng - UDP templates
by Thibaud Desodt 10 Sep '08

10 Sep '08

Hi, thanks for your replies. Actually, this morning, restarting syslog-ng doesn't throw an error anymore. I guess my configuration file was incorrect and I must have corrected it without noticing... I may have forgotten a semi-colon somewhere, and the error message was not very clear about the details. UDP-forwarding with a template now works as expected. Regards, Thibaud

1 0

Re: [syslog-ng] Syslog-ng - UDP templates
by Paul Robert Marino 10 Sep '08

10 Sep '08

Its avalible in the open source version. There is probably an error in your configuration can you please attach a copy so we can help you. You may replace the ip addresses with xxx.xxx.xxx.xxx if you are concerned with keeping the ip addresses off the list. Also please give us an example of how you would like the message formated. -----Original Message----- From: "Shlayen, Yakov (CAT)" <Yakov.Shlayen(a)AutoTraderPub.com> Subj: Re: [syslog-ng] Syslog-ng - UDP templates Date: Tue Sep 9, 2008 1:24 pm Size: 1K To: "Syslog-ng users' and developers' mailing list" <syslog-ng(a)lists.balabit.hu> Hello, I'm new user for syslog-ng product. I cannot found libdbi8* pkg for Solaris 10. Are you can help me with this? Best regards, Yakov (Jake) Shlayen Unix System Administrator Cox AutoTrader/AutoMart 5909 Peachtree Dunwoody Road Atlanta, GA 30328 Email:yakov.shlayen@autotraderpub.com Phone: 678-645-8227 Mobile: 404-643-3169 -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Thibaud Desodt Sent: Tuesday, September 09, 2008 11:40 AM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] Syslog-ng - UDP templates Hi, I am trying to configure syslog-ng (running on OpenSUSE 11.0) to forward a message to another server through UDP. I would like to modify the message's format, therefore I tried to use "template()" without success. (parsing error while restarting the syslog) The version installed with this distribution is 1.6.12, and from what I understood, the feature is available in later versions (1.6rc3 ?) . Is it available in the OpenSource Edition or only in the Premium Edition ? and where can I find it ? Thanks in advance Thibaud ________________________________________________________________________ ______ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html --- message truncated ---

1 0

Anyone Using Mailgraph (an RRD based Tool) for Postfix Mail Logs???
by wiskbroom＠hotmail.com 08 Sep '08

08 Sep '08

Hello; I am in search for a way to graph my logs for four separate mailservers, since all four are logging onto the same syslog-ng host, I figured I'd use mailgraph, a tool which creates graphs using RRDtools. I got rrdtools installed and ran mailgraph, and after not receiving any graph data at all in my database (mailgraph.rrd file), I ran strace and saw: write(2, "WARNING: line not in syslog form"..., 209) = 209 write(2, "WARNING: line not in syslog form"..., 167) = 167 write(2, "WARNING: line not in syslog form"..., 172) = 172 write(2, "WARNING: line not in syslog form"..., 173) = 173 write(2, "WARNING: line not in syslog form"..., 194) = 194 write(2, "WARNING: line not in syslog form"..., 172) = 172 My syslog-ng.conf has the following relevant data for this particular source: options { chain_hostnames(no); create_dirs (yes); dir_perm(0755); use_dns (yes); dns_cache(yes); dns_cache_size(1000); dns_cache_expire(604800); keep_hostname(yes); log_fifo_size(10000); log_msg_size(8192); long_hostnames(on); perm(0644); stats(3600); sync(0); time_reopen (10); use_dns(yes); use_fqdn(yes); }; log { source(S_udp); filter(F_mailservers); destination(D_mailservers); flags(final); }; destination D_mailservers { file("/var/log/Acme-Hosts/MailServers/$FULLHOST.log" perm(0644)); }; My logs look like: Sep 5 16:31:45 mailserver.example.net/mailserver.example.net postfix/qmgr[27362]: [ID 197553 mail.info] 597816CB1E: from=<203176.549962556(a)abc.exampletest.com>, size=14602, nrcpt=1 (queue active) Sep 5 16:31:45 mailserver.mailserver.net/mailserver.example.net postfix/qmgr[27362]: [ID 197553 mail.info] 3081A6CB1C: removed Sep 5 16:31:45 mailserver.mailserver.net/mailserver.example.net postfix/qmgr[27362]: [ID 197553 mail.info] 1C0426CB1D: removed Is there a way that I can tel syslog-ng to output to two locations, on e in the format above, which I need, and another in the format that mailgraph expexts to see? I assume that mailgraph is not setup to see the HOSTNAME/HOSTNAME entries shown above. I am using syslog-ng, latest, and am purposely stuffing my data into files named after the host, into a directory named after the same. I am also using a FIFO to redirect all logged data into a mysql DB, but the script mailgraph is receiving it's data from the flat files above. Any help appreciated, especially in a way that would not require rewriting the way the data looks now. Thanks all, .vp

2 5

Oracle logging into syslog-ng?
by Surlow, Jim 08 Sep '08

08 Sep '08

Please provide thoughts and comments - I'm not a DBA and I'm having our DBAs do further research here, but ... I infer from the Oracle documentation, that Oracle logs to a single syslog facility+level per instance, if configured. And the information in the Oracle log cannot be configured/altered. The downside that I'm seeing is that there is no information in regards to the Oracle instance. So, if I have a box with 20+ instances and I wish to relay this to a central server, all this is going to end up as noise. My kludge for this, would be to setup the following on each Oracle server: Instance #1 -> local0.debug Instance #2 -> local0.info Instance #3 -> local0.warn Instance #4 -> local1.debug Instance #5 -> local1.info Instance #6 -> local1.warn Etc. Then, setup syslog-ng on that Oracle server to use templates (one per instance), to route the data to a single facility & level on our centralized syslog-ng log server and imbed an instance name into the log. This config sounds really cumbersome. I'm also trying to avoid using Oracle to send the logs to a different instance which would do the consolidation. Does anyone else have a better solution? Am I right in that Oracle will only send to a single facility+level combo regardless of criticality? Thanks, Jim

2 1

Connection failed; error='Connection refused (111)'
by n r 05 Sep '08

05 Sep '08

hi, I intalled syslog-ng on server (debian etch) and on a client (ubuntu) i have this message on the both ??? syslog-ng[2959]: Connection failed; error='Connection refused (111)', time_reopen='10' any help? thanks nr _________________________________________________________________ Découvrez Windows Live Spaces et créez votre site Web perso en quelques clics ! http://spaces.live.com/signup.aspx

1 0

apache logs over syslog-ng sychronized
by Eli Shemer 04 Sep '08

04 Sep '08

Hey there, I would like to know if this configuration sounds viable to you. Right now it's not really operational performance wise. What I'm basically trying to do is to have my syslog server completely synchronized to the apache access logs of my other servers. On a client I have this configured: source s_apachelogs { file("/etc/apache2/logs/test/2008-09-02-test.log"); }; destination df_apachelogs { tcp("192.168.200.4" port(1999)); }; log { source(s_apachelogs); destination(df_apachelogs); }; and on the server: source s_apachelogs { tcp(ip(192.168.200.4) port(1999)); }; destination df_apachelogs { file("/var/log/apachelogs.log"); }; log { source(s_apachelogs); destination(df_apachelogs); }; This is just a sample case. Later on I would like to have my server keep an /var/log/apachelogs/ directory completely identical to the logs of the other servers. For some reason I don't see any traffic passed over the line unless I run a /etc/init.d/syslog-ng reload which is really bizzar. Thanks!

4 9

incorrect port number in stats
by Joe Shaw 04 Sep '08

04 Sep '08

Hi, Another minor thing I've noticed in 2.1.1... the port number in the log statistics for a remote destination is wrong: dropped='tcp(dest-host:514)=0 [...] The syslog-ng daemon is running on port 2000, and the data is correctly being sent there. In 2.0.9, it said instead: dropped='tcp(AF_INET(10.1.73.18:2000))=0' Joe

2 1

syslog-ng 2 nics
by Hubert Kupper 03 Sep '08

03 Sep '08

Hi, we run a SLES10 SP2 server as a syslog-ng server for a couple of month without errors. Now I added a secondary nic in the server an binded a subnet to the nic. The first switch from this subnet logs to syslog-ng fine, but every switch else in this subnet don't create a log-directory on the server. I've run ethereal an see that all switches send their packets to port 514 on the server, but syslog-ng do nothing with this packets. What can be wrong? Regards, Boert -- GMX startet ShortView.de. Hier findest Du Leute mit Deinen Interessen! Jetzt dabei sein: http://www.shortview.de/wasistshortview.php?mc=sv_ext_mf@gmx

2 4

No output to file with 2.1.1
by Joe Shaw 03 Sep '08

03 Sep '08

Hi, I just tried updating to 2.1.1 on my test instance and none of the incoming data is being written out to disk. It worked for me with 2.0.9. The problem is with the log_iw_size() parameter in my tcp source. Without it, the data is being logged to disk. My configuration file is below. Anything obviously amiss? Thanks, Joe -- # General Options options { sync(0); time_reopen(10); log_fifo_size(131072); long_hostnames(off); use_dns(yes); dns_cache(yes); use_fqdn(no); keep_hostname(yes); use_time_recvd(no); #log_msg_size(65536); log_msg_size(16384); stats_freq(180); #time_sleep(20); }; # Remote source source s_tcp { tcp(port(2000) log_iw_size(131072) log_fetch_limit(1024) max_connections(128)); }; # syslog-ng statistics source s_internal { internal(); }; destination d_file { file("/ita/syslog-messages" perm(0644)); }; destination d_stats { file("/ita/syslog-stats" perm(0644)); }; # Save stats separately log { source(s_internal); destination(d_stats); }; # Take all remote data and save it locally log { source(s_tcp); destination(d_file); flags(flow-control); };

2 1