- syslog-ng - lists.balabit.hu

empty syslog-ng log on Solaris 10
by list＠net-secured.com 25 Sep '09

25 Sep '09

2 1

flags(final)
by Michael J. Bauer 25 Sep '09

25 Sep '09

I think I am misunderstanding what flags(final) is supposed to do. I'm running syslog-ng 2.1.4 on RHEL 5.4 (Tikanga). I have a fairly simple syslog-ng configuration, which I've attached below. I'm trying to pick off individual groups of log entries and put them in their own individual files. I want to ensure that each gets logged exactly once, so I'm using flags(final). I also have a catch-all at the end in case I've missed something, but the ultimate goal is to have that file present, but empty. However, with this configuration, the log entries that appear in d_network_address_translation (/var/log/network-address-translation) also appear in d_default (/var/log/default) despite the presence of flags(final) on an earlier log() line. Should it work this way? If so, what can I do to get the desired behavior? Thanks, MJB options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); udp(ip(0.0.0.0) port(514)); }; destination d_network_address_translation { file("/var/log/network-address-translation"); }; destination d_default { file("/var/log/default"); }; filter f_network_address_translation { host("router-service-interface") and priority(info) and facility(local2) and match("FWNAT"); }; log { source(s_sys); filter(f_network_address_translation); destination(d_network_address_translation); flags(final); }; log { source(s_sys); destination(d_default); };

2 2

Can't get syslog-ng to accept my ldap log-conf
by Wilhelm Linder Rosén 25 Sep '09

25 Sep '09

Hi people! I can't seem to get syslog-ng to start with my conf. Can anyone help me out with my conf? The ldap part looks like this: source s_all { unix-stream("/dev/log"); internal(); }; destination d_ldap { file("/var/log/ldap[?].log"); }; filter f_syslog { not facility(auth, authpriv) and not match(slapd); }; filter f_ldap { match("slapd"); }; log { source(s_all); filter(f_syslog); destination(d_syslog); }; log { source(s_all); filter(f_ldap); destination(d_ldap); }; Thanks in advance, and have a great weekend =) // Wilhelm Linder Rosén

2 1

Syslog-ng is always one message behing with a tcp destination
by Srinivasan Sreenivasan 25 Sep '09

25 Sep '09

> Hi, > > I have a "client" syslog-ng that reads from a unix domain socket and writes to > a file destination as well as a tcp destination. I have a "server" syslog-ng > listening on 0.0.0.0 on the port specified in the "client" tcp destination. > > When I write a sample payload to the domain socket, syslog-ng writes the > message in full to the file destination immediately. However I see no file > created on the host referred to by tcp destination. > > If I write a second payload, the first payload is now written to the remote > location specified in the tcp destination. > > It seems like syslog-ng is always one message behind in writing to the tcp > destination. > > I am able to flush the "client" syslog-ng buffer by sending a kill -1. This > causes the payload to be written to the tcp destination. > > Is this the only way to force syslog-ng to write all its buffer contents to > the tcp destination? Or is there someway to control the flushing using > flow-control? > > > "client" syslog-ng configuration: > > @version: 3.0 > > options { > flush_timeout(1000); > }; > > source s_socks { > unix-stream ("/home/srini/dev/log" > max-connections(5) > log_msg_size(3145728) > flags(no-parse)); > > }; > > destination d_logs { > file("/logs/app.log"); > }; > > destination d_tcp { > tcp("192.168.1.102" port(60999) > frac_digits(6) > keep-alive(yes) > ); > }; > > rewrite r_rewrite_newlines{ > subst("#012", "\n", value("MESSAGE"), flags("global")); > }; > > log { > source(s_socks); > rewrite(r_rewrite_newlines); > destination(d_logs); > }; > > log { > source(s_socks); > destination(d_tcp); > }; > > > "Server" syslog-ng: This is the "centralized" log server: > > @version: 3.0 > > source s_tcp { > tcp(port(60999)) > log_msg_size(3145728) > keep-alive(yes) > }; > > > destination d_logs { > file("/logs/app.llog" > }; > > rewrite r_rewrite_newlines{ > subst("#012", "\n", value("MESSAGE"), flags("global")); > }; > > log { > source(s_tcp); > rewrite(r_rewrite_newlines); > destination(d_logs); > }; > > > Thanks. > Srini >

1 0

config with many rules
by Vasiliy G Tolstov 19 Sep '09

19 Sep '09

Hello. I'm new in this list, and may be my quistion answered already, but i can't founded any solutions: Problem: I have many filters, like this filter ff_auth { program(sshd) or program(dropbear) or facility(auth, authpriv); }; filter ff_cron { facility(cron); }; filter ff_daemon { facility(daemon); }; filter ff_user { facility(user); }; filter ff_debug { not facility(auth, authpriv, news, mail); }; filter ff_scm { program(git-daemon); }; filter ff_db { program(postgres) or program (mysql); }; filter ff_mail { facility(mail); }; filter ff_ftp { program(vsftpd) or program(proftpd); }; filter ff_rsync { program(rsync); }; filter ff_snmp { program(snmpd); }; filter ff_named { program(named); }; filter fl_alert { level(alert); }; filter fl_crit { level(crit); }; filter fl_err { level(err); }; filter fl_warn { level(warn); }; filter fl_notice { level(notice); }; filter fl_info { level(info); }; filter fl_debug { level(debug); }; If i want to add new filter to select kernel messages, that does not contain already defained data filter ff_kernel { facility(kern) and not match(firewall) and not match(mail) and not match(snmp) and not match(vsftpd) and not match(firewall); .....}; I think - not best solution to write all filters in not match and not program. Solution: Can You provide some help, how can i reduce unneded strings, may be not like ff_db and not like ff_debug ? P.S. Sorry if my english is bad and i can't fully discribe problem. -- Vasiliy G Tolstov <v.tolstov(a)selfip.ru> Selfip.Ru

2 2

Keeping hostname when forwarding
by PAUL WILLIAMSON 18 Sep '09

18 Sep '09

I have keep_hostname(yes) set in my global options, and no other mention of it anywhere else in my config. However, when I look at the remote system's logs, they are all tagged with the ip address of my syslog-ng host. I'm just trying to forward Cisco syslogs from my machine to another one, and using a filter for local7 is working great. I just need to figure out why the actual host is not being detected. Thanks, Paul ************************************ This email may contain privileged and/or confidential information that is intended solely for the use of the addressee. If you are not the intended recipient or entity, you are strictly prohibited from disclosing, copying, distributing or using any of the information contained in the transmission. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. This communication may contain nonpublic personal information about consumers subject to the restrictions of the Gramm-Leach-Bliley Act and the Sarbanes-Oxley Act. You may not directly or indirectly reuse or disclose such information for any purpose other than to provide the services for which you are receiving the information. There are risks associated with the use of electronic transmission. The sender of this information does not control the method of transmittal or service providers and assumes no duty or obligation for the security, receipt, or third party interception of this transmission. ************************************

1 0

syslog-ng v3.0.4 stops receiving remote logs
by Mike Duncan 18 Sep '09

18 Sep '09

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 We are running into an issue using syslog-ng v3.0.3 and v3.0.4. After about 1minute or so, all external/remote hosts can no longer log to the logging server (syslog-ng). We run udp only and have something around 300-400 hosts logging to the server. The local logging seems to be okay even after the remote logging ceases. We are not getting any indication from syslog-ng (logs or stdout) that there is an issue unless we run it in debug mode. Firstly, we tried to ensure the rmem and wmem were set to handle everything. We were seeing some 30million packet receive errors, so bumped up the settings from 8MB to 16MB. This did not help resolve the issue. Secondly, after looking through mailing lists and various posts online, we thought the issue to be the 0byte-udp-packet bug as mentioned here: https://bugzilla.balabit.com/show_bug.cgi?id=32. When running v3.0.3 in debug mode with trace enabled, we saw the EOF error messages which lead us to download v3.0.4, apply the patch and compile. Unfortunately, we are still having the same issue as before but no EOF error messages this time. ulimit is set to unlimited and at most, we only get 30-40 fd's opened. So, we are at a loss and hoping to get some input from here. Here are some details...(sorry for wrapping issues)... netstat -su snippet =========================== Udp: 259879531 packets received 709190 packets to unknown port received. 29579106 packet receive errors 328899408 packets sent RcvbufErrors: 29579106 netstat -anp snippet =========================== udp 16776712 0 0.0.0.0:514 0.0.0.0:* - sysctl snippet =========================== net.core.rmem_default = 16777216 net.core.rmem_max = 16777216 net.core.wmem_default = 16777216 net.core.wmem_max = 16777216 syslog-ng.conf snippets =========================== options { chain_hostnames(0); time_reopen(10); time_reap(360); log_fifo_size(2048); create_dirs(yes); group(adm); perm(0640); dir_perm(0755); use_dns(yes); #log_msg_size(2048); stats_freq(0); bad_hostname("^gconfd$"); }; source s_all { internal(); unix-stream("/dev/log"); file("/proc/kmsg" program_override("kernel: ")); udp(ip(0.0.0.0) port(514)); }; #example remote logging destination df_windows { file("/var/log/windows.log"); }; filter f_windows { program("MSWinEventLog"); }; log { source(s_all); filter(f_windows); destination(df_windows); flags(final); }; #another example destination df_networkdevice { file("/var/log/networkdevice.log"); }; filter f_2e { host("2e[0-9]") or host("2eg[0-9]"); }; #2nd floor East log { source(s_all); filter(f_2e); destination(df_networkdevice); flags(final); }; uname -a ============================= Linux logalot 2.6.24-24-server #1 SMP Fri Jul 24 23:27:55 UTC 2009 i686 GNU/Linux ps ax | grep syslog ============================= 2201 ? S 0:00 supervising syslog-ng 2202 ? Ss 0:00 /opt/syslog-ng/sbin/syslog-ng --no-caps Are we running into a limit of some sort or a bug or more likely -- are we missing something? Any help is appreciated. If you need more information, just let me know. Thanks. - -- Mike Duncan ISSO, Application Security Specialist Government Contractor with STG, Inc. NOAA :: National Climatic Data Center -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkqzyA4ACgkQnvIkv6fg9hZdiQCgkO6wW8yo9AC49f7E+sEQxD7J nH4AmwYNG/zsLJugvy+apqrueG+UR4TS =Y6Ol -----END PGP SIGNATURE-----

2 1

the match() filter without the use of the value() option is deprecated
by Jeremy M. Guthrie 18 Sep '09

18 Sep '09

Can anyone shed light on what the the the below error message means? I cannot find anything on what value() is and why my syntax is wrong. I am migrating a older configuration to 3.0.4. WARNING: the match() filter without the use of the value() option is deprecated and hinders performance, please update your configuration; -- -------------------------------------------------- Jeremy M. Guthrie jeremy.guthrie(a)cdw.com Managed Cisco Security Services Hosting and Managed Services Technical Architect Phone: 608-298-1061 CDW Fax: 608-288-3007 5520 Research Park Drive NOC: 608-298-1102 Madison, WI 53711 NOC Email: hmshelp(a)cdw.com

2 1

dropped log?
by fedora fedora 18 Sep '09

18 Sep '09

Hello, i am testing syslog-ng on a server which will be used to collect logs from different devices on our network, at this point the server is not busy at all and no outside device is sending syslog to it, i enabled stats_freq() in the global config last night and just now i notice there are 5k logs got dropped since then, see below Aug 14 08:32:59 test syslog-ng[14059]: Log statistics; dropped='pipe(/dev/xconsole)=5093', processed='center(queued)=2795', processed='center(received)=705', So what exactly does "dropped='pipe(/dev/xconsole)=5093'" mean? The load of this server is not heavy at all, i guess most of these messages are firewall logs, why so many logs got dropped? Thanks!

5 8

Re: [syslog-ng] Python help
by chris packham 16 Sep '09

16 Sep '09

On Wed, 2009-09-16 at 11:03 +0530, Jain, Vaibhav (GE Healthcare) wrote: > Hi, > > I am using a python parser for parsing the incoming message from > syslog-ng. In my log messages some fields are optional and some are > not.I want to know how to handle these optional parameters in the > python regual expression. > Your question is about python not syslog so you might have more luck from a python forum/mailing list. Having said that, the standard way of making a group optional in a regex is to add ? after it e.g. \d(\w+)? means a single digit followed by and optional string of word characters. Here's a small snippet I whipped up based on your example data. --- import re; data = ['Sep 15 22:23:37 ecis007 codeSystemName=5 UserID="Admin"', 'Sep 15 22:23:37 ecis007 UserID="Admin"'] r = re.compile (r'(\w+)\s+(\d+)\s+(\d\d:\d\d:\d\d)\s+([^\s]+)\s+(codeSystemName=(\d+)\s+)?(.*)') for d in data: print "d="+d m = r.match(d); if m: print repr(m.groups()) --- # python test.py d=Sep 15 22:23:37 ecis007 codeSystemName=5 UserID="Admin" ('Sep', '15', '22:23:37', 'ecis007', 'codeSystemName=5 ', '5', 'UserID="Admin"') d=Sep 15 22:23:37 ecis007 UserID="Admin" ('Sep', '15', '22:23:37', 'ecis007', None, None, 'UserID="Admin"')

1 0