- syslog-ng - lists.balabit.hu

Reliability of Remote logging via TCP/Syslog-NG
by Alf 16 Jun '05

16 Jun '05

Hi Guys, (New to this list, searched the FAQ for this qn, didnt find it. So I apologize if this was repeated before on the list. It might be related to the topic on TCP timingout though as report @ https://lists.balabit.hu/pipermail/syslog-ng/2005-February/006974.html) I'm planning to deploy a syslog-ng setup. (Central logging server in Asia, and Clients will be in US/HK). The setup is actually as a backup/secure repository of log entries of the appliation we are running. Now the question is, (I know TCP is reliable vs UDP), how reliable will syslog-ng hand the log entries ? If it gets disconnected (due to timeout, reset, etc), will it hold the log entry in memory and queue it for resend ? Is there an option to set before it times out (And at least logs it in a local log that there was an error (as an internal msg i guess) I need to be certain that no messages get lost basically :P Thanks in advance. Elvin -

5 10

Hostnames on output!
by mrgenius 16 Jun '05

16 Jun '05

hi All! I am running syslog-ng logging in msyql database with phpsyslog-ng as front end. Now i have 1 problem. The Devices which are sending logs don't have any hostnames associated with them nor i am running any DNS server. As a result the listing of hosts i am getting are in the form of IP addresses of hosts. Is it possible to show hostname against IP adresses?? Like x.x.x.x will be shown as ABC-ROUTER ?? if i define ABC-Router as x.x.x..x in /etc/hosts .. what configuration do i need to make in syslog-ng.conf?? Regards, -Geni

1 0

Some Questions!
by mrgenius 16 Jun '05

16 Jun '05

Hi all! I belongs to a relatively huge orginization. And we are not going to impliment Centralized Logging. But I have Few considerations and Things in mind whihc i needs to be rectified. We are ISP having around 8-9 Cisco routers ranging from 2600 to 7200 series. around 30MaxTNT Access Servers and aroud 20 Cisco Switches. NOw i have Few Questions - What is Facility?? is it log message type ?? "DEBUG INFO NOTICE WARNING ERROR CRIT ALERT EMERG " . Because All Devices supports defining of facility from local0 to local 7. And Some devices like MaxTNT have options of definit facility as well as log level (having options of DEBUG INFO NOTICE WARNING ERROR CRIT ALERT EMERG) - Altough i know it depends on logs/time duration. But i would like to know what kind of machine is needed to runn syslog server (linux based) with stability? Will Dual XEON with 512k ram be enough?? And how about Harddisk size? Please if some one who has implimented syslog in such orgnization structure as i have.. suggest me answers of above said Question. Regards, -Geni Network Engineer

2 1

OT: why does writes to /dev/log block?
by Jason Haar 15 Jun '05

15 Jun '05

Hi there I just reinstalled my Fedora workstation recently and hadn't moved over to syslog-ng like I normally do - so this is a question about syslog really. In the past two days I've come in to find my workstation fairly screwed up by the syslog daemon being hung/broken. It would be running, but not working any more. Anyway, as the socket file /dev/log exists, all apps that write to syslog were also hung! So I had my mail server screwed, cronjobs unable to finish, etc, etc. Simply stopping the syslog daemon caused my load average to jump to 40+ whilst all the pent-up processes started working again :-) I guess once the socket file disappeared under them, they just carried on nicely... So - am I right in saying /dev/log blocks, and is there some way at an application layer to limit how long a process attempts to write to syslog before giving up? I know an alarm around the syslog call should work - but is that the only way? Thanks! -- Cheers Jason Haar Information Security Manager, Trimble Navigation Ltd. Phone: +64 3 9635 377 Fax: +64 3 9635 417 PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1

2 3

sqlsyslogd sometimes does not starts
by Metal Gear 15 Jun '05

15 Jun '05

Hi, i m using a script which logs certain alerts to mysql db. It mostly wotks fine but sometimes i see that i m not getting the alerts into mysql db i have to then manually start the service. I m using the following code for the above mentioned purpose which is being taken from http://www.vermeer.org/display_doc.php?doc_id=1#scripts. I m getting the logs in flat files but none in the mysql db. Thanks #!/bin/bash # # sqlsyslogd This is a daemon that takes syslog-ng input and pipe it into # a MySQL database. # # chkconfig: 2345 92 10 # description: sqlsyslogd bridges syslog-ng and mysql. # author: Josh Kuo Thu 2004/08/12 13:21:56 PDT . /etc/rc.d/init.d/functions case "$1" in start) if [ -x /tmp/mysql.pipe ]; then mkfifo /tmp/mysql.pipe else # if the service is already running, do not start another one PIDS=`pidofproc mysql` if [ "$PIDS" ]; then gprintf "sqlsyslogd is already running.\n" exit 1 fi mysql -u DBUSERNAME --password=PASSWORD DBNAME < /tmp/mysql.pipe & fi ;; stop ) killproc mysql ;; *) gprintf "Usage: sqlsyslogd {start|stop}\n" exit 1; esac exit 0;

2 1

Lines sorted into the wrong log files
by David Miller 14 Jun '05

14 Jun '05

I upgraded a pair of debian (sarge) boxen to syslog-ng 1.6.5. One is to be the central logging server, the other is a sample client. I changed the destinations of all the logs to just include tcp("ip_address"); On the server side I added tcp() to the source s_all {} statement. My problem is that I'm getting more logging done on the remote side than on the client side. For example, the client has: wiabweb2:/etc/syslog-ng# tail /var/log/cron.log 13 Jun 05:45:01 ntpdate[22494]: ntpdate 4.2.0a@1:4.2.0a+stable-2-r Sun Jan 9 16:13:28 CET 2005 (1) 13 Jun 05:45:03 ntpdate[22494]: step time server 192.168.120.13 offset 1.988834 sec but the remote server has: Jun 13 16:51:01 192.168.120.27 CRON[1779]: pam_ldap: could not open secret file /etc/ldap.secret (No such file or directory) Jun 13 16:51:01 192.168.120.27 /USR/SBIN/CRON[12192]: (root) CMD (/bin/date >> /tmp/date) Jun 13 16:52:01 192.168.120.27 CRON[18762]: pam_ldap: could not open secret file /etc/ldap.secret (No such file or directory) Jun 13 16:52:01 192.168.120.27 /USR/SBIN/CRON[172]: (root) CMD (/bin/date >> /tmp/date) These "extra" entries show up in syslog on both systems. Why would the remote system have the entries logged to more files than the client? ***************** Long set of config files follows ********************* Rather than add a thousand lines of almost all config, I'll just add what I think is relevent. On the server side, # sources # all known message sources source s_all { # message generated by Syslog-NG internal(); # standard Linux log source (this is the default place for the syslog() # function to send logs to) unix-stream("/dev/log"); # messages from the kernel file("/proc/kmsg" log_prefix("kernel: ")); # use the above line if you want to receive remote UDP logging messages # (this is equivalent to the "-r" syslogd flag) # udp(); tcp(); }; [snip] ###### # destinations # some standard log files destination df_auth { file("/var/log/$HOST/auth.log"); }; destination df_syslog { file("/var/log/$HOST/syslog"); }; destination df_cron { file("/var/log/$HOST/cron.log"); }; [snip] ###### # filters # all messages from the auth and authpriv facilities filter f_auth { facility(auth, authpriv); }; # all messages except from the auth and authpriv facilities filter f_syslog { not facility(auth, authpriv); }; # respectively: messages from the cron, daemon, kern, lpr, mail, news, user, # and uucp facilities filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_kern { facility(kern); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_news { facility(news); }; filter f_user { facility(user); }; filter f_uucp { facility(uucp); }; [snip] # *.*;auth,authpriv.none -/var/log/syslog log { source(s_all); filter(f_syslog); destination(df_syslog); }; # this is commented out in the default syslog.conf # cron.* /var/log/cron.log log { source(s_all); filter(f_cron); destination(df_cron); }; ========= On the client side ============= ###### # sources # all known message sources source s_all { # message generated by Syslog-NG internal(); # standard Linux log source (this is the default place for the syslog() # function to send logs to) unix-stream("/dev/log"); # messages from the kernel file("/proc/kmsg" log_prefix("kernel: ")); # use the above line if you want to receive remote UDP logging messages # (this is equivalent to the "-r" syslogd flag) # udp(); }; [snip] ###### # destinations # some standard log files destination df_auth { file("/var/log/auth.log"); tcp("192.168.120.49"); }; destination df_syslog { file("/var/log/syslog"); tcp("192.168.120.49"); }; destination df_cron { file("/var/log/cron.log"); tcp("192.168.120.49"); }; destination df_daemon { file("/var/log/daemon.log"); tcp("192.168.120.49"); }; destination df_kern { file("/var/log/kern.log"); tcp("192.168.120.49"); }; # destination df_lpr { file("/var/log/lpr.log"); tcp("192.168.120.49"); }; destination df_mail { file("/var/log/mail.log"); tcp("192.168.120.49"); }; ###### # filters # all messages from the auth and authpriv facilities filter f_auth { facility(auth, authpriv); }; # all messages except from the auth and authpriv facilities filter f_syslog { not facility(auth, authpriv); }; # respectively: messages from the cron, daemon, kern, lpr, mail, news, user, # and uucp facilities filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_kern { facility(kern); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_news { facility(news); }; filter f_user { facility(user); }; filter f_uucp { facility(uucp); }; # some filters to select messages of priority greater or equal to info, warn, # and err # (equivalents of syslogd's *.info, *.warn, and *.err) filter f_at_least_info { level(info..emerg); }; filter f_at_least_notice { level(notice..emerg); }; filter f_at_least_warn { level(warn..emerg); }; filter f_at_least_err { level(err..emerg); }; filter f_at_least_crit { level(crit..emerg); }; # all messages of priority debug not coming from the auth, authpriv, news, and # mail facilities filter f_debug { level(debug) and not facility(auth, authpriv, news, mail); }; # all messages of info, notice, or warn priority not coming form the auth, # authpriv, cron, daemon, mail, and news facilities filter f_messages { level(info,notice,warn) and not facility(auth,authpriv,cron,daemon,mail,news); }; # messages with priority emerg filter f_emerg { level(emerg); }; # complex filter for messages usually sent to the xconsole filter f_xconsole { facility(daemon,mail) or level(debug,info,notice,warn) or (facility(news) and level(crit,err,notice)); }; # *.*;auth,authpriv.none -/var/log/syslog log { source(s_all); filter(f_syslog); destination(df_syslog); }; # this is commented out in the default syslog.conf ***** Why is ntpdate still logging into /var/log/cron ? # cron.* /var/log/cron.log # log { # source(s_all); # filter(f_cron); # destination(df_cron); #}; [snip]

1 0

does remove_if_older still work?
by sawall 14 Jun '05

14 Jun '05

I have one line in my syslog-ng.conf file, as follows: destination netops { file("/var/log/HOSTS/$HOST" remove_if_older(604800) owner(root) group (root) perm(0644) dir_perm(0700)); }; I'm running version 1.6.7 on Fedora Core 2. I would like to have the files that hit this destination be deleted every 7 days. It does not appear to be working as the files keep growing. Maybe I misunderstood, but this should let the file grow for n seconds, then delete the file and start writing to a new file with the same name, right? I tried setting this to 60 and that made no difference. Any help would be appreciated. Thanks, Chris

4 7

Re: [syslog-ng] syslog-ng and windows
by mibry＠iinet.net.au 13 Jun '05

13 Jun '05

I hope this is able to help someone, I have been working on a solution to get eventlogs from a Windows box to a central syslog-ng server using open source products and have come up with a solution similar to using syslog-ng with stunnel. I have the solution setup in test environment at the moment and it seems to be working fine. I haven't quite finished the documentation at the moment but hope to have the documentation and files on my web site by Tuesday the 14th June. I hope that others will find the information useful. Here is the setup: Windows 2000 server running Eventlog to Syslog Utility available from https://engineering.purdue.edu/ECN/Resources/Documents/UNIX/evtsys. Eventlog to Syslog sends Windows events to the loghost server. The loghost server is running debian sarge and syslog-ng 1.6.5. To make sure that the data from the Windows box is not intercepted during transmission I am using a tunneling program that is available for both *nix and Win32 boxs called Zebedee which is available from http://www.winton.org.uk/zebedee. Zebedee creates a secure tunnel for both TCP and UDP from the windows box to the loghost server. The default binaries under Linux do not spoof the correct ip address in the syslog logs so I have compiled a new set binaries which are available from the web site listed below. The only problem with the build that I have done is the program needs to be run as root. Any comments or feed back on the solution is welcome. http://members.iinet.net.au/~mibry/ Kind Regards Michael Bryant >Just curious.... >What would happen if TCP transmission was not terminated with an nl or >nul char? Would TCP receive buffers fill up and kill communication on >te server? >-----Original Message----- >From: syslog-ng-bounces at lists.balabit.hu >[mailto:syslog-ng-bounces at lists.balabit.hu] On Behalf Of Balazs >Scheidler Sent: Monday, May 30, 2005 6:24 AM To: tilaris at wanadoo.fr; Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] syslog-ng and windows On Mon, 2005-05-30 at 11:08 +0200, JF Suret wrote: > Hello, > > I'm using syslog-ng as a central log server, and I have both linux and windows clients. I know that there are some syslog windows clients ( NTsyslog, snare) but I can't find any open source syslog-ng clients. > > What I'm looking for is (at least if it does not exist) information on the TCP data format used by syslog-ng. > So I could write a little udp to tcp syslog translator that could be used on windows clients (and maybe modify NTsyslog if I have enought time...) It is basically the same as UDP, the only exception is that messages has to be translated by an NL or NUL characters as otherwise there's no way to recognize message boundary. -- Bazsi _______________________________________________ syslog-ng maillist - syslog-ng at lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html ----- End forwarded message -----

1 0

Solaris 10 info
by Dave Johnson 11 Jun '05

11 Jun '05

After playing around with syslog-ng on Solaris 10 today, here are some notes: * As mentioned from Kenneth G, libol / syslog-ng should compile fine on Solaris 10, just make sure --enable-full-dynamic" in your configure for syslog-ng. * You may also need to update your LD_LIBRARY_PATH and PATH (/usr/sfw & /opt/sfw ? ) * Service Management Facility ("svcs" "svcadm" "svccfg" commands) You can choose to run syslog-ng via the traditional /etc/init.d method on solaris 10 and then wrap that start and stop script with the solaris 10's smf facility, or the other option is to do everything via the SMF facility. I've attached two files to do the later. The example files assume your syslog-ng.conf is ln /etc/syslog-ng/ -> /usr/local/etc/syslog-ng/ and the binary is /usr/local/sbin/syslog-ng: 1) Compile and install libol, syslog-ng. 2) mkdir /usr/local/etc/syslog-ng; ln -s /usr/local/etc/syslog-ng /etc/syslog-ng 3) copy your syslog-ng.conf to /etc/syslog-ng/ 4) cp svc.method.syslog-ng /lib/svc/method/syslog-ng 5) cp svc.manifest.syslog-ng.xml /var/svc/manifest/system/syslog-ng.xml 6) `svccfg import /var/svc/manifest/system/syslog-ng.xml` Then to switch from standard syslog to syslog-ng: 1) svcadm disable system-log 2) svcadm enable syslog-ng To revert back to solaris 10's default syslog, and have it running on reboot: 1) `svcadm disable syslog-ng` 2) `svccfg delete syslog-ng` 3) `svcadm enable system-log` ---------- Forwarded message ---------- From: kenneth.gullberg(a)foreningssparbanken.se < kenneth.gullberg(a)foreningssparbanken.se> Date: May 26, 2005 5:11 AM Subject: SV: [syslog-ng] syslog-ng 1.6.8 has been released To: syslog-ng(a)lists.balabit.hu Hi, Here is a small suggestion for the INSTALL file in the next release for unexperienced admin's: Solaris 10 ---------- For successful compilation on Solaris 10 you need to use --enable-full-dynamic when you run configure. // Kenneth

4 5

change localhost to actual hostname in log entries
by deet 10 Jun '05

10 Jun '05

Hello. I'm running the Suse supplied (SLES9) version of syslog-ng (version 1.6.5-10). I have a few hosts that are logging everything to a central log server. The syslog-ng process on the client hosts is listening on the loopback interface on port 514 for inbound syslog writes from applications that cannot use /dev/log. The problem is that the log entries get written as being from localhost (example below) Jun 10 11:35:04 localhost tomcat: INFO: 2005-06-10 11:35:04,103 [StoreLoader] - Find registry server-registry.xml at classpath resource Obviously this is not much of a problem when your looking at the logs on the client host that created the log entry. But on the central log server i'm sorting logs by $HOST and end up with all the logs from various clients which appear to be from "localhost" all get mixed together and written to one file which is confusing. So my question is 2 fold. On the client is it possible to change the hostname from "localhost" to the actual hostname via a syslog-ng configuration on the client before the logs get written to disk and sent the remote syslog server? If that is not possible are their any syslog-ng ways to accomplish the goal of changing "localhost" to the correct hostname either on the client or on the syslog server. TIA. Deet.

1 0