August 2008 - syslog-ng - lists.balabit.hu

[Bug 14] New: Maintain symlinks to current and previous log output files...
by bugzilla＠bugzilla.balabit.com 28 Sep '11

28 Sep '11

https://bugzilla.balabit.com/show_bug.cgi?id=14 Summary: Maintain symlinks to current and previous log output files... Product: syslog-ng Version: 2.0.x Platform: PC URL: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=218688 OS/Version: Linux Status: NEW Severity: minor Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: sasa(a)debian.org Type of the Report: enhancement Estimated Hours: 0.0 >From bugs.debian.org: I'm hoping that upstream might consider adding functionality to syslog-ng to allow it to track current and previous log files and maintain symlinks to them, as a user option. This replicates one of the options that I (and others) find useful in cronolog. This has been previously discussed on the syslog-ng mailing list, though apparently nothing fully came of it. The archive of that discussion is at: https://lists.balabit.hu/pipermail/syslog-ng/2002-June/003452.html Obviously, it's probably somewhat of a low priority, but it would be nice. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

syslog-ng client messages delayed
by mark peters 22 Mar '10

22 Mar '10

Hi, We are running into a issue where syslog-ng clients who are configured to forward to a central syslog-ng destination server that are reasonably chatty are getting there messages delayed at the source (ie. 'tcpdump'ing the client shows it is sending messages from N minutes ago, where N is anything from 5 to 60 minutes on average sometimes more). We have also seen a small percentage of loss at the destination. On these 'delayed' clients we also see the resident memory footprint on the growing. Clients are running 1.6.12 (have tried 2.0.8 also - same symptons). The central syslog-ng server is running 2.0.8. relevant config entries are: syslog-ng client: options { sync (1); time_reopen (10); log_fifo_size (2097152); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; destination d_central { tcp("log2.xyz.com" port(5140)); }; filter f_local5 { facility(local5); }; log { source(s_sys); filter(f_local5); destination(d_central); flags(final); }; syslog-ng central server: options { time_reopen (10); log_fifo_size (2097152); long_hostnames (off); use_dns (yes); dns_cache (yes); dns_cache_expire (28800); use_fqdn (no); create_dirs (yes); dir_perm (0755); perm (0644); keep_hostname (no); use_time_recvd (no); stats_freq(600); flush_timeout(16000); flush_lines(256); time_sleep(80); }; destination d_server { file("/logs/server.$LEVEL.log"); }; filter f_server { facility(local5) and program("server"); }; log { source(s_sys); filter(f_server); destination(d_server); flags(final); }; to give a rough idea of message volume the server is receiving approximately 50 million log lines per day on average any help/information would be appreciated thanks -------------------------------------- Easy + Joy + Powerful = Yahoo! Bookmarks x Toolbar http://pr.mail.yahoo.co.jp/toolbar/

3 4

[Bug 21] New: call shutdown() before closing a socket
by bugzilla＠bugzilla.balabit.com 30 Apr '09

30 Apr '09

https://bugzilla.balabit.com/show_bug.cgi?id=21 Summary: call shutdown() before closing a socket Product: syslog-ng Version: 2.1.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bazsi(a)balabit.hu Type of the Report: --- Estimated Hours: 0.0 syslog-ng should call shutdown() before closing a socket to ensure the contents of the socket buffer are emptied. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

[patch] Add follow_freq_ms option
by Joe Shaw 16 Jan '09

16 Jan '09

Hi, As I mentioned in the last thread, I've found myself wanting to have better than one second resolution for the follow_freq() option. Attached is a patch which adds a follow_freq_ms option, which takes its value in milliseconds rather than full seconds. follow_freq() continues to work as it did previously. Thanks, Joe

3 7

[PATCH] anonymizing filter
by Micah Anderson 06 Nov '08

06 Nov '08

Hello, A couple years ago this patch was submitted to the list for consideration for inclusion into syslog-ng. I am writing this email again to request that it be considered again. The patch provides a simple replace which enables you to strip out IP addresses from your logs before they are written to disk. The patch has been included in the Debian stable distribution, and currently is included in both Debian Sid and Lenny (unstable and testing). It has had a very wide testing base and is non-intrusive, it has existed since 2004 and has been adapted to work with the newer syslog-ng. The goal of this patch is to give an organization the means to implement site logging policies, by allowing for easy control over exactly what data is retained in the logfiles. When I first requested consideration for inclusion the reactions were some suggestions for improvement (which were done), some side discussions about the various states of data retention laws, and a general agreement that this patch is non-intrusive and had a valid use case (at least in the U.S., but also likely in other countries as well[0]). The side-discussions about data-retention laws were mostly around specific geographic localities that were considering laws that would make stripping of addresses illegal, or had already mandated such things. Although these were interesting discussions, as EU data retention laws would prohibit many people from making such configuration changes to their syslog-ng.conf, they were tangential to the point because this patch does not cause those to break such laws. On the other side of the pond, in the U.S., the EFF[1] has made it very clear that this mechanism of anonymizing logs is perfectly (a) legal in the U.S., and (b) advisable. There are many instances where it is preferable to keep less information on users than is collected by default on many systems. In the United States it is not currently required to retain data on users of a server, but you may be required to provide all data on a user which you have retained. OSPs can protect themselves from legal hassles and added work by choosing what data they wish to retain. The current climate in the U.S. makes this problem so much more important now than it was many years ago. Having the ability to implement a site-policy that enables an organization to decide if the trade-off between privacy and analysis is worthwhile. This patch allows organizations to have that choice if they feel that it is more important to avoid retaining sensitive data rather than having a full history of everything logged. Please accept this patch[2], Micah [0] EPIC International Data Retention Page http://www.epic.org/privacy/intl/data_retention.html [1] The EFF is the major civil liberties internet watchdog in the US, their "Best Practices for Online Service Providers" can be found here: http://www.eff.org/osp, they explicitly link to our patch as a recommendation [2] The latest patch can be found at https://code.autistici.org/trac/privacy/browser/trunk/syslog-ng

6 11

syslog-ng 2 nics
by Hubert Kupper 03 Sep '08

03 Sep '08

Hi, we run a SLES10 SP2 server as a syslog-ng server for a couple of month without errors. Now I added a secondary nic in the server an binded a subnet to the nic. The first switch from this subnet logs to syslog-ng fine, but every switch else in this subnet don't create a log-directory on the server. I've run ethereal an see that all switches send their packets to port 514 on the server, but syslog-ng do nothing with this packets. What can be wrong? Regards, Boert -- GMX startet ShortView.de. Hier findest Du Leute mit Deinen Interessen! Jetzt dabei sein: http://www.shortview.de/wasistshortview.php?mc=sv_ext_mf@gmx

2 4

Filtre empty program
by G R 29 Aug '08

29 Aug '08

Hi! I'm trying to filter some logs that have no program field. kind of "logger -t '' " logs. I've try to use program(""); or program(NULL); as filter but none of them works. How can I filter this empty program field logs ? Thanks! Gaël.

4 7

Re: [syslog-ng] destination program is killed due to broken pipe
by prmarino1＠gmail.com 28 Aug '08

28 Aug '08

well the script on the surface sounds like a good simple idea, the simple way is not always the best or most relyable way. There are several better options open to you that should work better and shouldn't be too hard to impliment. The first is to write to a log file then use something like a "tail -f" in your script. Another option open to you is if you know how to write unix network program's in any languages (I use perl for this kind of thing my self) it wouldn't be difficult to write a simple application read from a socket, or even udp syslog on an alternitive port on the loopback address. Syslog is a relitivly simple protocal to write a listener for and there are several perl modules that can do most of the work for you. ----------------------- Sent from my Treo(r) smartphone -----Original Message----- From: "midnight codder" <midnight.codder(a)gmail.com> Date: Tuesday, Aug 19, 2008 5:08 pm Subject: [syslog-ng] destination program is killed due to broken pipe To: syslog-ng(a)lists.balabit.huReply-To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> I am trying to redirect all log messages to a destination program (written in java). This did not do much so I redirected all messages to a single catch-all log file AND also to a bash shell script that runs an endless loop, reads a line and spit it out to a text file. The shell script is killed by syslog-ng after the 5th message. The catch-all file, however, keeps getting all of the messages (that's how I know who is killing who). Here is the config file for syslog-ng: options { ts_format(iso); frac_digits(3); }; template bds_template{ template('<pri=$PRI> <date=$ISODATE> <host=$HOST> <msg=$MSG>\n'); template_escape(no); }; source src{ internal(); unix_stream('/dev/log'); unix-dgram('/dev/log'); file('/proc/kmsg' log_prefix('kernel:')); udp(); tcp(); }; destination test_adapter{ program('bash /usr/local/bds/logger/syslogAdapterTest.sh' template(bds_template)); }; log { source(src); destination(test_adapter); }; destination catchAll{ file('/var/log/syslog-ng-catch-all.log', template(bds_template)); }; log { source(src); destination(catchAll); }; Here is the bash script: #!/bin/bash file='/root/syslogAdapterTest' counter=1 limit=3 while true do read line echo '$line' > $file.$counter ((counter++)) done and finally the content of the catch-all file, in blue are the log messages that the shell script still captures and spits to text files, in red are the messages are the messages the shell does not get: <priE> <date 08-08-19T15:13:56.191-05:00> <host=oracle-test> <msg=syslog-ng[27971]: Server license found, starting in server mode; customer='University of Illinois', serial='49544f30-573e-11dd-9892-0007e9195011', limit='25'> <priF> <date 08-08-19T15:13:56.191-05:00> <host=oracle-test> <msg=syslog-ng[27971]: Starting destination program; cmdline='bash /usr/local/bds/logger/syslogAdapterTest.sh'> <priE> <date 08-08-19T15:13:56.193-05:00> <host=oracle-test> <msg=syslog-ng[27971]: syslog-ng starting up; version='2.1.11a'> <priF> <date 08-08-19T15:13:56.193-05:00> <host=oracle-test> <msg=syslog-ng[27971]: Initializing destination file writer; template='/var/log/syslog-ng-catch-all.log', filename='/var/log/syslog-ng-catch-all.log'> <priE> <date 08-08-19T15:13:56.193-05:00> <host=oracle-test> <msg=syslog-ng[27971]: License validity expires in one week, logging service will terminate at that point; error='License outside the validity period (option=syslog-ng, not-before 08/07/21, not-after 08/08/21)'> <priC> <date 08-08-19T15:13:56.198-05:00> <host=oracle-test> <msg=syslog-ng[27971]: I/O error occurred while writing; fd='11', error='Broken pipe (32)'> <priF> <date 08-08-19T15:13:56.198-05:00> <host=oracle-test> <msg=syslog-ng[27971]: Sending child a TERM signal; child_pid='27972'> Any idea?

3 6

Can syslog-ng support TCP and UDP simultaneously?
by Guy Fleegman 28 Aug '08

28 Aug '08

Hello: Currently we have all of our devices sending syslog data to a 2.05 syslog-ng server. However... there is one instance where we need to use TCP as a delivery mechanism Can syslog-ng support having both TCP and UDP sources simultaneously? If so ..where might I find a configuration example? Please advise and thanks, -Guy

3 2

Lots of dropped=tcp(AF_INET(
by Ivan Lezhnjov Jr. 27 Aug '08

27 Aug '08

Hello I have two hosts on my LAN running syslog-ng. Host A acts as central loghost and host B sends its log messages via tcp/ip network connection to host A to store. The issue is that I've got a lot of messages like this one (and almost no normal ones) generated by host B: Aug 19 19:25:23 xerxes syslog-ng[20806]: Log statistics; dropped='tcp(AF_INET(217.117.75.2:1999))=0', processed='center(queued)=112', processed='center(receive d)=23', processed='destination(newsnotice)=0', processed='destination(console)=0', processed='destination(debug)=23', processed='destination(mailinfo)=0', proc essed='destination(mail)=0', processed='destination(user)=0', processed='destination(newserr)=0', processed='destination(newscrit)=0', processed='destination(u ucp)=0', processed='destination(messages)=19', processed='destination(mailwarn)=0', processed='destination(ppp)=1', processed='destination(loghost)=23', proces sed='destination(lpr)=0', processed='destination(cron)=0', processed='destination(syslog)=23', processed='destination(authlog)=0', processed='destination(maile rr)=0', processed='destination(kern)=0', processed='destination(daemon)=0', processed='destination(xconsole)=0', processed='destination(console_all)=23', proce ssed='source(net)=0', processed='s While it is obvious that dropped tcp speaks of network issues it isn't really clear to me (being a novice syslog-ng user) what exactly goes wrong here. So, could you please explain what's wrong with host B and how I can fix that? -- Ivan Lezhnjov Jr. Europe, Ukraine, Simferopol Running Source Mage GNU/Linux, kernel version 2.6.24 build #5 +----------------------------------------------------------------------+ Key ID 0x5811D90C Key Fingerprint 2A52 5C8C 38BE C04F D8DE A169 19E2 E49A 5811 D90C Use GPG Exercise Your Right To Privacy

5 9