- syslog-ng - lists.balabit.hu

1.6.11 stops logging on Garbage Collect (Solaris 9)
by Kim Cary 11 Oct '06

11 Oct '06

Our installation of syslog-ng writes to 3 pipes and a dozen files from 2 IP sources. About 2-3 times a week it will stop logging in 1-12 files. The last record in these files will almost always coincide with a Garbage Collect record in /var/log/syslog (which always continues to collect records). Solaris syslog is NOT running. Solaris logadm sends HUP to syslog-ng on any rotate. We tried TCP logging from some systems for a while, but its all UDP now. Important to note, it may not be all files that stop receiving records, or even all the files from a single source. We need reliable logs. How do we begin to debug this issue?

2 1

logging by hostname
by Istvan Szukacs 11 Oct '06

11 Oct '06

Hi! I would like to log several computer which are behind nat. The problem is i cannot find the right metod to log by the hostname not by the ip reverse name from where the packets come. But i got this: Oct 1 02:34:27 external ip/external ip Is there a possibility to devide the logs by the original hostname? thx

4 3

Re : [syslog-ng] Syslog and syslog-ng on HP-UX
by olivier "madmax"rolland 11 Oct '06

11 Oct '06

I have to run syslog and syslog-ng because syslog-ng have not been probated in my compagny yet. I want to log the same message in syslog and in syslog-ng but i conclude that's not possible for the two daemons to read the same pipe. Syslog-ng extract only messages that syslogd don-t want to. Thanks for your help and if you have an idea for a solution contact me. ----- Message d'origine ---- De : Gellér Sándor <wildy(a)balabit.hu> À : Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Envoyé le : Mercredi, 11 Octobre 2006, 11h29mn 17s Objet : Re: [syslog-ng] Syslog and syslog-ng on HP-UX olivier "madmax"rolland wrote: > Hello! > I have a problem, > I want that syslog and syslog-ng 1.6.7 to coexist on a HP-UX 11.11 and HP-UX 11.23. > I install syslog-ng 1.6.7 and it works but the problem is that syslog-ng don't log when syslogd runs. I'n not familiar with HP-UX, but my first guess is that you are using the same source for both daemons. Remove the standard HP-UX log source from syslog-ng.conf and reload syslog-ng. What is the purpose to use two syslog daemons on the same machine? Do you want to log the same messages with syslog and syslog-ng, or do you want to separate them? -- Gellér Sándor wildy(a)balabit.hu _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html ___________________________________________________________________________ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Demandez à ceux qui savent sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com

2 1

Re: Subject: [syslog-ng] 1.6.11 stops logging on Garbage Collect (Solaris 9)
by Kim Cary 11 Oct '06

11 Oct '06

Whilst working further on this problem, I was tailing the log and this popped up: /var/log/syslog:Oct 10 13:50:01 mylogserver syslog-ng[20167]: do_stream_read: getmsg() trying to return too much ctl data, res=4294967295 Resource temporarily unavailable Is this any clue to the problem we are having? How do I interpret that message? I don't find any info on it in Google or this list... > Date: Mon, 9 Oct 2006 09:19:42 -0700 > From: Kim Cary <Kim.Cary(a)pepperdine.edu> > Subject: [syslog-ng] 1.6.11 stops logging on Garbage Collect (Solaris > 9) > To: syslog-ng(a)lists.balabit.hu > Message-ID: <9B0A23E1-566A-465A-BB00-BA3FF0EA0FBC(a)pepperdine.edu> > Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed > > Our installation of syslog-ng writes to 3 pipes and a dozen files > from 2 IP sources. > > About 2-3 times a week it will stop logging in 1-12 files. The last > record in these files will almost always coincide with a Garbage > Collect record in /var/log/syslog (which always continues to collect > records). > > Solaris syslog is NOT running. > Solaris logadm sends HUP to syslog-ng on any rotate. > We tried TCP logging from some systems for a while, but its all UDP > now. > > Important to note, it may not be all files that stop receiving > records, or even all the files from a single source. > > We need reliable logs. How do we begin to debug this issue?

2 1

Syslog and syslog-ng on HP-UX
by olivier "madmax"rolland 11 Oct '06

11 Oct '06

Hello! I have a problem, I want that syslog and syslog-ng 1.6.7 to coexist on a HP-UX 11.11 and HP-UX 11.23. I install syslog-ng 1.6.7 and it works but the problem is that syslog-ng don't log when syslogd runs. My syslog-ng.conf look like: options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { pipe("/dev/log" pad_size(2048)); internal(); unix-stream ("/dev/log.un"); }; What I have to do to resolve this problem? Thanks for help! ___________________________________________________________________________ Yahoo! Mail réinvente le mail ! Découvrez le nouveau Yahoo! Mail et son interface révolutionnaire. http://fr.mail.yahoo.com

2 1

Syslog-ng and filter question ...
by Sébastien CRAMATTE 10 Oct '06

10 Oct '06

Hello list, I've just setup a centralized syslog-ng server with mysql Remote clients work well but I'v got a little problem with a filter ... On each client I've got snmp daemon running and nagios network manager server polling clients ... My problem is that each time my nagios NMS connect it to a client it appears this line in daemon.info client log "Oct 10 03:15:48 vlanrouter1 snmpd[1868]: Connection from 192.168.0.12 " I would like to filter this line to exclude it from the remote log ... So I 've added this filter filter f_not_daemon_info { (not facility(daemon) and not level(info)); }; but not works and all "daemon" and all "info" levels are removed not just "daemon.info" I've search to remove this log from snmp daemon directly but seems to be a bug on my snmpd release ... I hope that someone have got the solution Regards Sebastien

4 7

bug report
by Matt Zagrabelny 09 Oct '06

09 Oct '06

i sent out an email a couple of days ago asking about this. i will try once more... does anyone on the list know the preferred way of submitting bug reports to the developers? i have considered using reportbug and letting the debian maintainer facilitate my bug. thanks, -- Matt Zagrabelny - mzagrabe(a)d.umn.edu - (218) 726 8844 University of Minnesota Duluth Information Technology Systems & Services PGP key 1024D/84E22DA2 2005-11-07 Fingerprint: 78F9 18B3 EF58 56F5 FC85 C5CA 53E7 887F 84E2 2DA2 He is not a fool who gives up what he cannot keep to gain what he cannot lose. -Jim Elliot

4 8

Re: question regarding program name logs
by G.W. Haywood 09 Oct '06

09 Oct '06

Hi there, On Sun, 8 Oct 2006 Balazs Scheidler wrote: > > I would really like to release 2.0.0. But if your findings indeed > > turn out to be a bug in syslog-ng that'd prevent me from doing so. > > So helping me to track this down in a timely manner would be > > appreciated. Thanks. > > An important question relating this: Does anyone else experience the > same problem? For the avoidance of doubt, I'm still using 2.0.rc1 with your patch of 04 Sep 2006 12:18:46 but I'm only using local logging. Most of my log messages are from sendmail and a number of sendmail milters, and it's quite common for them to produce a few thousand log messages per second although messages at that rate would be written direct to a file - just a small subset of them is piped to a script for storage in our database. I haven't noticed any problem with missing messages. If this were a problem for us I'd have expected someone here to have noticed by now, although we haven't looked for the issue specifically. -- 73, Ged.

2 1

program() doesn't restart if it dies
by Hari Sekhon 09 Oct '06

09 Oct '06

I have an email alert script written in python to send me security alerts depending on criteria I have defined. I was working on something else and wanted to clean up so I did a pkill -9 python without remembering that I'm on the syslog server. Of course this killed the alert script, which I noticed didn't restart and syslog-ng kept running without it. So I tested it by creating a condition to generate a log and therefore an alert and I didn't get the email. Not very good. Syslog-ng didn't restart the script defined in program() in syslog-ng.conf, even when it had more input for it, which has gotten me thinking, what happens if the script really dies or is accidentally killed? Syslog-ng must have just dropped the output on the floor or something. The only way I can see to restart it is to restart syslog-ng itself. What is the best way to make this more resilient? I'm currently thinking a cron job to test for the python script and then restart the server if it is not found. Anybody got anything better? -h -- Hari Sekhon

3 4

Re: bug report
by G.W. Haywood 07 Oct '06

07 Oct '06

Hi there, On Sat, 7 Oct 2006 Balazs Scheidler wrote: > > > using the "flow-control" flag, like this: > > > > > > log { source(src); destination(dst); flags(flow-control); }; > > > > Hmmm. I knew I'd seen that syntax somewhere. But where...? :) > > Probably in the 2.0 reference manual: Blast. I was looking in the man pages. Would you like a patch? :) -- 73, Ged.

3 2

RE: [syslog-ng] Re: question regarding program name logs
by Justin Randall 07 Oct '06

07 Oct '06

Sorry I pasted the wrong "log" statement, here it is: log { source(src_udp); source(src_tcp); filter(fltr_PROGRAM); destination(dst_ PROGRAM); flags(final, flow-control); }; -----Original Message----- From: Justin Randall Sent: Wednesday, September 27, 2006 10:07 AM To: Syslog-ng users' and developers' mailing list Subject: RE: [syslog-ng] Re: question regarding program name logs Hi and thanks for the response. This is actually relating to UDP logging. I have devices sending Syslog messages via UDP to a central Syslog-NG server. Usually the devices log messages to Syslog-NG via UDP in bursts (about 20-25 message within a couple seconds). The source, filter, destination, log statements for the handling are as follows: source src_udp { udp( ip(IPADDRESS) port(PORT) ); }; filter fltr_PROGRAM { program("PROGRAM") or match("PROGRAM"); }; destination dst_PROGRAM { file( "/var/log/PROGRAM/messages" owner(root) group(tomcat) perm(0640) ); }; log { source(src_udp); source(src_tcp); destination(dst_remote_system); flags(final, flow-control); }; What ends up happening when a host falls into this flow, the first log message is missed. The format of the bursts of messages are exactly the same. This did not happen in 2.0rc2, I have downgraded and verified this. Let me know if there's any other info I can provide that would help out. Regards, Justin. -----Original Message----- From: G.W. Haywood [mailto:ged@jubileegroup.co.uk] Sent: Wednesday, September 27, 2006 7:42 AM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] Re: question regarding program name logs Hi there, On Wed, 27 Sep 2006 Justin Randall wrote: > Can anyone confirm if the patch that was to fix this issue was included > in 2.0rc3? Yes, it was. Note that it only affects logging via UDP. -- 73, Ged. _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 2

Re: [syslog-ng] bug report
by G.W. Haywood 07 Oct '06

07 Oct '06

Hi there, On Fri, 6 Oct 2006 Balazs Scheidler wrote: > Syslog-ng 2.0.x ... can stop reading input messages if the output > queues are full. This can be enabled on a per-log-statement basis, > using the "flow-control" flag, like this: > > log { source(src); destination(dst); flags(flow-control); }; Hmmm. I knew I'd seen that syntax somewhere. But where...? :) 73, Ged.

2 1

Re: question regarding program name logs
by G.W. Haywood 07 Oct '06

07 Oct '06

Hi there, On Wed, 27 Sep 2006 Justin Randall wrote: > Can anyone confirm if the patch that was to fix this issue was included > in 2.0rc3? Yes, it was. Note that it only affects logging via UDP. -- 73, Ged.

2 1

RE: [syslog-ng] Re: question regarding program name logs
by Justin Randall 07 Oct '06

07 Oct '06

After upgrading and monitoring, the logs which would previously get logged incorrectly due to the angle bracket prefixing are now not being logged at all. These logs don't come in with any peculiarity about them. The only thing if interest to note is that it is always the first log message in a group of log messages (i.e. host sends 5 Syslog messages, the first one is missed). What kind of logs, information would be helpful to diagnose? I'm currently running on 2.0rc3. Regards, Justin. -----Original Message----- From: G.W. Haywood [mailto:ged@jubileegroup.co.uk] Sent: Tuesday, September 12, 2006 6:27 AM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] Re: question regarding program name logs Hi there, On Tue, 12 Sep 2006 Justin Randall wrote: > I am currently using Syslog-NG version 2.0rc1. > > I've noticed that sometimes when Syslog-NG is logging messages, it > prefixes something onto the program name which disrupts filters > while matching against a program name. For example, in some logs > for a program named "foo" I'll see logs in the form of "<15>foo" > which fail to be matched against a filter of 'program("foo") '. > This seems to only be noticed on logs which originate from remote > hosts. > > Could anyone provide an explanation for what is going on with this? I reported this problem a couple of weeks ago, and Balazs kindly provided a patch which fixed it. I imagine the patch is in .rc2, try upgrading. -- 73, Ged. _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 1

Controlling Logging to Central Syslog-ng Server, DoS etc
by Hari Sekhon 06 Oct '06

06 Oct '06

Hi all, What is the best way to control logging to my central log server? I have read syslog-ng docs quite a lot when I was first setting it up but I don't recall seeing anything on access control. Doesn't this leave me open to DoS attacks etc? After all, you couldn't somebody just write a loop to send garbage to it and fill the whole machine up, not to mention drown out all other valid logs so you miss any important events (oops, I am giving away too much here?). I'm actually tempted to write an attack for this right now... So what do I do to stop some other person from doing this to me? I guess I could use iptables to make sure that only authorized ips can connect to my tcp and udp ports. Is there a better way I haven't thought of? -h -- Hari Sekhon

6 6

connecting to syslog-ng
by Torsten Curdt 06 Oct '06

06 Oct '06

Hi there! Is there a way to connect to a syslog-ng on a machine via a client to receive the events? ...as opposed to just having the syslog-ng pushing the event to another machine all the time. Basically I want only to connect to those machines running syslog-ng in certain situations. Like ssh root@machine tail -f /var/log/messages ...without the need for a ssh account cheers -- Torsten

6 12

FAQ site down - found temporary home
by catenate 06 Oct '06

06 Oct '06

The web site hosting the syslog-ng FAQ has been down for about two days now. This will continue for about another day (or more) until the telco guys come by and figure out where the line is broken. Until it's back up, I recommend using the copy that I put on Splunk Base: http://www.splunk.com/base/syslog-ng-faq Once www.campin.net is back up, I'll evaluate how much sense it makes to keep the FAQ in a wiki. So if you always wanted to update or reformat the FAQ - now is your chance while the FAQ is on one. I'll look closely at any mods made and be sure to keep the good ones. Note that the page isn't formatted very well, and none of the old links actually work. If you want anything from the old real site, google's cache is your friend: http://66.102.7.104/search?q=cache:meyHWPX8PUYJ:www.campin.net/syslog-ng/fa… Input appreciated. Nate Campi

2 1

proper procedure for filing bug reports
by Matt Zagrabelny 06 Oct '06

06 Oct '06

just curious what people do for bug reports with syslog-ng... thanks, -- Matt Zagrabelny - mzagrabe(a)d.umn.edu - (218) 726 8844 University of Minnesota Duluth Information Technology Systems & Services PGP key 1024D/84E22DA2 2005-11-07 Fingerprint: 78F9 18B3 EF58 56F5 FC85 C5CA 53E7 887F 84E2 2DA2 He is not a fool who gives up what he cannot keep to gain what he cannot lose. -Jim Elliot

2 1

problem with syslog-ng hanging on saturday after working all week
by Steven Meacham 06 Oct '06

06 Oct '06

Hi all, Need help. My syslog-ng is hanging on Saturdays. It is like the service just stops responding even though it is running. The system having the problem is just basically a syslog bridging/filtering server. I am using syslog-ng to accept syslog messages and filter for only what I need out of them, and then forwarding them on. It is basiclly setup like below: Webserver>>(udp 514)>syslog-ngfileterserver>>(udp1514)>Syslog Loghost Any help you could give to a newbie about debugging or find the issue would be great..Also any scripts to check the service and restart would be great to. Thanks Steven Meacham

4 4

Gentoo issues with new portage (maybe)
by Rob Munsch 04 Oct '06

04 Oct '06

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 At least i think that's what this is. After a successful install of eventlog, i get this at syslog configure: - ----- checking for pkg-config... /usr/bin/pkg-config checking pkg-config is at least version 0.9.0... yes checking for GLIB... yes checking for EVTLOG... configure: error: Package requirements (eventlog) were not met: No package 'eventlog' found Consider adjusting the PKG_CONFIG_PATH environment variable if you installed software in a non-standard prefix. Alternatively, you may set the environment variables EVTLOG_CFLAGS and EVTLOG_LIBS to avoid the need to call pkg-config. - ----- Why can't pkg-config find eventlog? Should i go bug Gentoo about this? (there's an eventlog ebuild, but it's masked...). And syslog-ng itself is at 1.6.9 (in ebuilds). Yeah, i looked at man pkg-config, I'm not sure how to proceed. Should i be telling p-c where to find it, or bypassing p-c entirely..? I see this: /usr/local/lib/pkgconfig/eventlog.pc which contains: - ----- prefix=/usr/local exec_prefix=${prefix} libdir=${exec_prefix}/lib includedir=${prefix}/include/eventlog Name: EventLog Description: General system logging format library Version: 0.2.5 Cflags: -I${includedir} Libs: -L${libdir} -levtlog - ----- Seems like enough info, yah? What went wrong? Anyone else on Gentoo hit this..? TIA, - -- Rob Munsch Solutions For Progress IT www.solutionsforprogress.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFIT4rBvBcJFK6xYURAsAGAKCPCygDfOkVaIoxgK5rybqRF6bavwCdHoX7 hcXJsQ1UPHt8O+kUIn0vjmE= =mt/P -----END PGP SIGNATURE-----

2 1

Question regarding filters
by Ian Meyer 03 Oct '06

03 Oct '06

I don't want to assume anything about how the program deals with messages, so my question is how do messages that match or do not match a filter cascade down? For instance if I have a filter: filter f_test { program('test'); }; and a destination that specific filter, under what conditions would it NOT make it to said destination? What's going on is I log output from a program, but it's making it's way into /var/log/messages, when it should be going to it's own definition /var/log/services/app/app.log. Does the order of log { } make a difference? filters? destinations? I looked around the faq and the reference manual and didn't find anything explaining the process flow. Thanks in advance! Ian

2 1

syslog drops stats
by Niki Hammler 02 Oct '06

02 Oct '06

Hi syslog-ng pro's, I always get messages like these: Oct 2 09:28:05 nobaq syslog-ng[2207]: STATS: dropped 548 I searched google but I could not find a solution or a good explanation why this happens. If this is true and syslog dropped 548 (!) log messages this would be bad. I think a syslog daemon is an important thing that should NOT just drop messages. I can't imagine that my system is not strong enough. 512MB RAM, 1.7Ghz. I've already set the FIFO size to a high value: $ grep size /etc/syslog-ng/syslog-ng.conf # log_fifo_size(2048); log_fifo_size(32768); log_msg_size(2048); But it does not help. Then I grepped the source to see where the problem comes from. There is a variable pktbuf_dropped_pkts and in the whole source I can not find any reference that increments this var. Is this warning uncritical? Is there any solution on this? Best regards, Niki PS: Debian sarge, v. 1.6.5

2 2

EventViewer to SysLog - looking for opinions
by Tom Valdes 01 Oct '06

01 Oct '06

I recently started evaluating tools to convert Windows Event Viewer messages to SysLog and I'm looking for opinions on the different ones and what to look for. I first started with EvtSys and it worked pretty well but it left out the hostname so it was hard to make server specific swatch statements. I then tried ntsyslog which shared the same problem. I'm now trying Snare (thanks Kevin for the tip) and it looks a lot more flexible as to what type of events get sent and it sends the hostname as well. When I first installed it, it worked fine and send messages.. After fiddling with it, it stopped working correctly. I'm going to start working with it again, but I'd like to here what else people are using. The 2 things the tool should have are: be free and transmit the Hostname. thanks, tom

3 3

RE: [syslog-ng] EventViewer to SysLog - looking for opinions
by Justin Randall 29 Sep '06

29 Sep '06

* I first started with EvtSys and it worked pretty well but it left out the hostname * so it was hard to make server specific swatch statements. I then tried ntsyslog * which shared the same problem. This shouldn't be a problem for Syslog-NG because you can have it do a DNS lookup on the source IP address of a Syslog message to get the hostname, and then use the $FULLHOST_FROM macro in your 'destination' directive to log it to a file including the hostname. Here's a link that explains more regarding the macros you can use to refine log handling: http://www.balabit.com/products/syslog_ng/reference-2.0/syslog-ng.html/i ndex.html#macros If you don't have DNS in your internal network, you can simply make the IP to hostname correlation in your /etc/hosts file. I hope this helps, Justin. _____ From: Tom Valdes [mailto:tom.valdes@gmail.com] Sent: Friday, September 29, 2006 1:06 PM To: Syslog-ng users' and developers' mailing list Subject: [syslog-ng] EventViewer to SysLog - looking for opinions I recently started evaluating tools to convert Windows Event Viewer messages to SysLog and I'm looking for opinions on the different ones and what to look for. I first started with EvtSys and it worked pretty well but it left out the hostname so it was hard to make server specific swatch statements. I then tried ntsyslog which shared the same problem. I'm now trying Snare (thanks Kevin for the tip) and it looks a lot more flexible as to what type of events get sent and it sends the hostname as well. When I first installed it, it worked fine and send messages.. After fiddling with it, it stopped working correctly. I'm going to start working with it again, but I'd like to here what else people are using. The 2 things the tool should have are: be free and transmit the Hostname. thanks, tom

1 0

create_dirs(yes) doesn't actually create directories
by Ian Meyer 29 Sep '06

29 Sep '06

[root@xxxxx ~]# syslog-ng --version syslog-ng 1.6.9 global options from syslog-ng.conf: options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (yes); use_fqdn (no); create_dirs (yes); keep_hostname (yes); }; Here's three lines from the config, the /var/log/services directory exists, but the network dir doesn't get created. From what I understand of the reference manual.. create_dirs(yes) *should* create them, yes? I have about 30 directories, and I'd rather not create them by hand. :) Permissions on all directories above /var/log/services are correct. (rwxr-x-r-x) Any insight would be appreciated! Thanks in advance, Ian

2 1