- syslog-ng - lists.balabit.hu

Syslog-ng writing to files once per minute
by Eric Cables 01 Jul '10

01 Jul '10

I'm hoping this is a really stupid question, but after implementing what I thought to be the fix to this, I can't seem to get it working. I have a number of hosts logging to facility local5, and the destination log file is only updated once per minute (HH:MM.30s to be precise). I've implemented flush_lines(0); and flush_interval(0);, to the global options, but it doesn't seem to have an effect. syslog-ng 3.0.3 My goal is to have logs written to the destination file with minimal latency after syslog-ng receives the message. -- Eric Cables

2 6

[Bug 88] New: rewrite FACILITY does not work.
by bugzilla＠bugzilla.balabit.com 01 Jul '10

01 Jul '10

https://bugzilla.balabit.com/show_bug.cgi?id=88 Summary: rewrite FACILITY does not work. Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Windows Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: alex-pub.balabit(a)reflexion.net Type of the Report: --- Estimated Hours: 0.0 Here is the fragment of my syslog-ng.conf file that demonstrates the problem source remote_sys { tcp( ip( "0.0.0.0" ) port( 514 ) max-connections( 1152 ) ); udp( ip( "0.0.0.0" ) port( 514 ) ); }; destination d_remote_consolidated_log { file( "/var/log/remote/$FACILITY/$FACILITY.log" create_dirs( yes ) perm( 0644 ) dir_perm( 0755 ) template( "$R_ISODATE $HOST -$LEVEL- $PROGRAM[$PID]: $MSGONLY\n" ) frac_digits( 6 ) template_escape( no ) ); }; filter f_apache { facility( local1 ); }; rewrite r_rewrite_apache_facility { set("apache" value("FACILITY")); }; log { source( remote_sys ); filter( f_apache ); rewrite( r_rewrite_apache_facility ); destination( d_remote_consolidated_log ); flags( final ); }; If I issue the following command from the host connected to this syslog-ng server: $ echo this is apache log test | logger -t apache-test -p local1.info Then the output is sent to "/var/log/remote/local1/local1.log" instead of expected "/var/log/remote/apache/apache.log". Also, if I include $FACILITY into the destination template it prints "local1" If I run syslog-ng with -d flag I see this in output: >>Rewrite expression evaluation result; value='FACILITY', new_value='apache' Below is a full log for this test >>Incoming log entry; line='<142>Jun 22 17:36:11 host-1 apache-test: this is a test 2' >>Filter rule evaluation begins; filter_rule='f_filter1' >>Filter node evaluation result; filter_result='not-match', filter_type='facility' >>Filter rule evaluation result; filter_result='not-match', filter_rule='f_filter1' >>Filter rule evaluation begins; filter_rule='f_filter2' >>Filter node evaluation result; filter_result='not-match', filter_type='facility' >>Filter rule evaluation result; filter_result='not-match', filter_rule='f_filter2' >>Filter rule evaluation begins; filter_rule='f_apache' >>Filter node evaluation result; filter_result='match', filter_type='facility' >>Filter rule evaluation result; filter_result='match', filter_rule='f_apache' >>Rewrite expression evaluation result; value='FACILITY', new_value='apache' So this makes me believe that rewrite does happen, but the value is not picked up later. BTW, the documentation for `set` command says that there should be a comma between "apache" and value, but other samples do not put it there. I've tried it both ways to no avail. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 9

spoof-source performance issues
by Martin Holste 01 Jul '10

01 Jul '10

I'm finding that with a destination of udp("10.x.x.x", port(514) spoof_source(yes)) about half of messages get lost when going from one syslog-ng host to another at a high message rate (> 3k/sec). This is on 3.1 OSE and the hosts are on the same subnet and switch, so there shouldn't be any network devices interfering. Has anyone else had this same issue? My hunch is that it's either a performance issue with the way the libnet (I'm using 1.1.2.1 on SuSE 10.2) API is implemented or it's an issue within the libnet API. Has anyone else noticed performance problems when using spoof_source?

5 12

[Bug 91] New: Cisco ASA log parsing issues
by bugzilla＠bugzilla.balabit.com 01 Jul '10

01 Jul '10

https://bugzilla.balabit.com/show_bug.cgi?id=91 Summary: Cisco ASA log parsing issues Product: syslog-ng Version: 3.1.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: jon(a)jonjames.net Type of the Report: bug Estimated Hours: 0.0 Orignal LOG as seen my TCPDUMP 16:42:08.734684 IP 172.16.32.157.syslog > 172.16.32.172.syslog: SYSLOG local4.notice, length: 95 E..{.......^.. ... ......g<.<165>Jul 01 2010 00:32:59: %ASA-5-111001: Begin configuration: 172.16.16.154 writing to memory After being relayed as seen my TCPDUMP 16:42:59.208826 IP 172.16.32.172.33753 > 172.16.32.225.syslog: SYSLOG local4.notice, length: 120 E...a.@.@.?... ... ........?<165>1 2010-07-01T00:33:50-04:00 172.16.32.157 %ASA-5-111001 - - - Begin configuration: 172.16.16.154 writing to memory As you can see, syslog-ng appears to be messing with the datestamp and therefore the RSA envision device I'm relaying to does not recognize the log as being from a cisco asa. Is there a way to update the log parser in syslog to resolve this issue ? syslog-ng.conf (parts) @version: 3.0 options { time_sleep(30); time_reap(30); mark_freq(30); dns_cache(yes); use_fqdn(no); keep_hostname(yes); chain_hostnames(no); use_dns(no); dns_cache_size(250000); dns_cache_expire(300); dns_cache_expire_failed(300); stats_freq(3600); log_msg_size(10000); }; source s_extranet { udp(ip("172.16.32.172") port(514)); }; log { source(s_extranet); destination(d_syslog_udp); }; destination d_syslog_udp { syslog("172.16.32.225" transport("udp") port(514)); }; -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

3 3

Memory leak can occur with supressed duplicate log messages
by andy kitchingman 30 Jun '10

30 Jun '10

Hi Due to my company firewall, I do not have access to the git public repository, so I will post this here. I believe there is a memory leak when duplicate messages are suppressed and messages cannot be consumed: logwriter.c: log_writer_last_msg_flush(LogWriter *self) ------------------------------------------- log_queue_push_tail(self->queue, m, &path_options) can return FALSE (message cannot be consumed), in which case the local LogMessage m is leaked. The fix should be something like if (!log_queue_push_tail(self->queue, m, &path_options)) { log_msg_unref(m); } Regards Andy Kitchingman

2 1

[Bug 89] New: DOCUMENTATION: incorrect structure of embedded log statement.
by bugzilla＠bugzilla.balabit.com 30 Jun '10

30 Jun '10

https://bugzilla.balabit.com/show_bug.cgi?id=89 Summary: DOCUMENTATION: incorrect structure of embedded log statement. Product: syslog-ng Version: 3.1.x Platform: PC OS/Version: Windows Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: alex-pub.balabit(a)reflexion.net Type of the Report: --- Estimated Hours: 0.0 In syslog-ng-ose-v3.1-guide-admin-en.pdf. Topic "4.5.1. Using embedded log statements" Shows incorrect structure of embedded log statement: log { source(s1); source(s2); ... optional_element(filter1|parser1|rewrite1); optional_element(filter2|parser2|rewrite2);... destination(d1); destination(d2); ... <<<<<<<<<<<<<<<<<<<<<< ! Wrong position ! flags(flag1[, flag2...]); <<<<<<<<<<<<<<<<<<<<<< #embedded log statement log { optional_element(filter1|parser1|rewrite1); optional_element(filter2|parser2|rewrite2);... destination(d1); destination(d2); ... #another embedded log statement log { optional_element(filter1|parser1|rewrite1); optional_element(filter2|parser2|rewrite2);... destination(d1); destination(d2); ... }; }; }; If you follow this structure, the resulting config file will not compile, because `flags` element must appear after the embedded log statement, at least according to cfg-grammar.y: log_stmt : '{' log_items log_forks log_flags '}' { LogPipeItem *pi = log_pipe_item_append_tail($2, $3); $$ = log_connection_new(pi, $4); } ; -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

WARNING: the match() filter without the use of the value() ...
by ml ml 29 Jun '10

29 Jun '10

Hello List, i am getting this error: WARNING: the match() filter without the use of the value() option is deprecated and hinders performance, please update your configuration; Restarting syslog-ng: Stopping syslog-ng: OK. Starting syslog-ng: WARNING: the match() filter without the use of the value() option is deprecated and hinders performance, please update your configuration; with this config: @version: 3.0 #Default configuration file for syslog-ng. # # For a description of syslog-ng configuration file directives, please read # the syslog-ng Administrator's guide at: # # http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/bk01-toc.html # ### 1.) OPTIONS options { long_hostnames(off); }; source src { unix-stream("/dev/log"); internal(); }; source kernsrc { file("/proc/kmsg"); }; ### 2.) DESTINATION destination ldap { file("/var/log/ldap.log"); }; destination authlog { file("/var/log/auth.log"); }; destination syslogmsg { file("/var/log/syslog"); }; destination cron { file("/var/log/cron.log"); }; destination daemon { file("/var/log/daemon.log"); }; destination kern { file("/var/log/kern.log"); }; destination user { file("/var/log/user.log"); }; destination mail { file("/var/log/mail.log"); }; destination lighttpd { file("/var/log/lighttpd.log"); }; destination mailinfo { file("/var/log/mail.info"); }; destination mailwarn { file("/var/log/mail.warn"); }; destination mailerr { file("/var/log/mail.err"); }; destination all { file("/var/log/all.log"); }; destination fcron { file("/var/log/fcron.log"); }; destination sshd { file("/var/log/sshd.log"); }; destination debug { file("/var/log/debug"); }; destination messages { file("/var/log/messages"); }; destination console { usertty("root"); }; ### 3.) FILTERS filter f_ldap { match("slapd"); }; filter f_auth { facility(auth); }; filter f_authpriv { facility(auth, authpriv); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_kern { facility(kern); }; filter f_mail { facility(mail); }; filter f_user { facility(user); }; filter f_debug { not facility(auth, authpriv, news, mail) ; }; filter f_messages { level(info..warn) and not facility(auth, authpriv, mail, news); }; filter f_emergency { level(emerg); }; filter f_info { level(info); }; filter f_notice { level(notice); }; filter f_warn { level(warn); }; filter f_crit { level(crit); }; filter f_err { level(err); }; filter f_lighttpd { match("lighttpd"); }; filter f_fcron { match("fcron"); }; filter f_sshd { match("sshd"); }; filter f_syslog { not facility(authpriv, mail) and not filter(lapd) and not filter(sshd) and not filter(fcron) ; }; log { source(src); filter(f_sshd); destination(sshd); }; log { source(src); filter(f_fcron); destination(fcron); }; log { source(src); filter(f_ldap); destination(ldap); }; log { source(src); filter(f_syslog); destination(syslogmsg); }; log { source(src); filter(f_authpriv); destination(authlog); }; log { source(src); filter(f_cron); destination(cron); }; log { source(src); filter(f_daemon); destination(daemon); }; log { source(kernsrc); filter(f_kern); destination(kern); }; log { source(src); filter(f_mail); destination(mail); }; log { source(src); filter(f_user); destination(user); }; log { source(src); filter(f_mail); filter(f_info); destination(mailinfo); }; log { source(src); filter(f_mail); filter(f_warn); destination(mailwarn); }; log { source(src); filter(f_mail); filter(f_err); destination(mailerr); }; log { source(src); filter(f_debug); destination(debug); }; log { source(src); filter(f_messages); destination(messages); }; log { source(src); filter(f_emergency); destination(console); }; log { source(src); destination(all); }; Any idea what i am doing wrong?

3 5

Two instances of syslog-ng on the same host?
by John R. Dunning 29 Jun '10

29 Jun '10

Hi wizards. Apologies if this is an FAQ or something, but I've dug all around and failed to find the answer. I have a system on which, for reasons I'd rather not go into here, it makes sense to run two instances of syslog-ng, one for standard logging of local events, the other acting as a proxy for a flock of other systems. The proxy starts first, very early in the init sequence, the regular one starts later. This all worked great with syslog-ng 2, but I recently upgraded to version 3.1.1 and I can't get it to work correctly. The proxy instance is supposed to only be listening on a tcp socket, but it seems to also be opening the AF_UNIX socket to /dev/log. This causes the launch of the main instance to fail. I've been through the docs, but it's not obvious to me how to get syslog-ng to start without opening the socket to /dev/log. Hints? Thanks in advance...

4 7

Re: [syslog-ng] Two instances of syslog-ng on the same host?
by Rory Toma 27 Jun '10

27 Jun '10

As a point of reference, I have a setup with over 200 syslog-ng processes and it works fiine. "Patrick H." <syslogng(a)feystorm.net> wrote: >Yes, that should work just fine. I run a box with 4 syslog instances and >they all place nice together. My setup isnt exactly the same, but I have >no problems which you are experiencing. My setup: I have a master >syslog-ng process that does nothing but accept connections from remote >hosts and from the local system. That process then relays the messages >to other local syslog-ng processes which do the actual work (lots and >lots of parsing). >I only included the options and source sections as theyre the only ones >that matter. > >Master syslog-ng process that accepts from local & remote & forwards to >other local processes >@version: 3.0 ># syslog-ng configuration file. ># ># This should behave pretty much like the original syslog on RedHat. But ># it could be configured a lot smarter. ># ># See syslog-ng(8) and syslog-ng.conf(5) for more information. ># > >options { > time_reopen(1); > use_dns(no); > use_fqdn(yes); > keep_hostname(yes); > create_dirs(yes); > perm(0644); > dir_perm(0755); > log_iw_size(50000); > log_fifo_size(100000); > #time_sleep(1); >}; > >source s_sys { > file("/proc/kmsg" program-override("kernel")); > unix-stream ("/dev/log"); > internal(); >}; >source s_net { > tcp(ip(0.0.0.0) port(514) max-connections(1000)); > udp(ip(0.0.0.0) port(514)); >}; > > >syslog-ng that accepts the forwarded messages >@version: 3.0 ># syslog-ng configuration file. ># ># This should behave pretty much like the original syslog on RedHat. But ># it could be configured a lot smarter. ># ># See syslog-ng(8) and syslog-ng.conf(5) for more information. ># > >options { > time_reopen(1); > long_hostnames(off); > use_dns(no); > use_fqdn(no); > keep_hostname(yes); > create_dirs(yes); > perm(0644); > dir_perm(0755); > #log_fetch_limit(100000); > #log_iw_size(200000); > #log_fifo_size(400000); > flush_lines(50); > flush_timeout(5000); > #stats_freq(10); > #stats_level(2); > #time_sleep(1); >}; > > >source s_master { > #syslog(ip(127.0.0.1) port(515) transport('tcp') so_keepalive(yes) >log_iw_size(1000)); > tcp(ip(127.0.0.1) port(515) flags('syslog-protocol')); >}; >source s_syslog { > internal(); >}; > > > > > > > > > > >Sent: Saturday, June 26, 2010 6:08:42 PM >From: John R. Dunning <jrd(a)jrd.org> >To: syslog-ng(a)lists.balabit.hu >Subject: [syslog-ng] Two instances of syslog-ng on the same host? >> Hi wizards. Apologies if this is an FAQ or something, but I've dug >> all around and failed to find the answer. >> >> I have a system on which, for reasons I'd rather not go into here, it >> makes sense to run two instances of syslog-ng, one for standard >> logging of local events, the other acting as a proxy for a flock of >> other systems. >> >> The proxy starts first, very early in the init sequence, the regular >> one starts later. >> >> This all worked great with syslog-ng 2, but I recently upgraded to >> version 3.1.1 and I can't get it to work correctly. The proxy >> instance is supposed to only be listening on a tcp socket, but it >> seems to also be opening the AF_UNIX socket to /dev/log. This causes >> the launch of the main instance to fail. >> >> I've been through the docs, but it's not obvious to me how to get >> syslog-ng to start without opening the socket to /dev/log. Hints? >> Thanks in advance... >> ______________________________________________________________________________ >> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng >> FAQ: http://www.campin.net/syslog-ng/faq.html >> >> > >______________________________________________________________________________ >Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng >FAQ: http://www.campin.net/syslog-ng/faq.html >

1 0

few questions about patterndb
by Hendrik Pahl 25 Jun '10

25 Jun '10

Hi folks, i'm currently testing the patterndb with syslog-ng. After a few test runs,there are some open questions. Will there be some future releases of the patterndb by balabit? Last Snapshot has a timestamp from sep. 2009. Or is there a community somewhere out there, that i haven't seen yet?:-) I tested the classification into different logfiles (system,violation,unknown) with 1.5M lines of logfiles, and there was just one rule matching 'em all: 734adfc8-8a4d-40ad-923d-9d3fbabda34b, which is something about a newly connected usb mass storage device. This behaviour can't be correct? The patternfile "system-apache2.xml" still containts just plain messages, no XML anywhere - i thougt this should have been fixed yet? After some testing with logging messages to file like "messages-system/messages-violation/messages-unknown" i'm just asking myself about the profit i get when using patterndb. With a standard syslog-ng i usually can log things into files divided by the priority of the messages - that's all i want it to do, just have a logfile where all errors und failures are logged, just to get the pure number of lines down. When controlling a logfile i usually don't really care about a cronjob to be successful. What is the profit of using patterndb right there? Thanks in advance, i.A. Hendrik Pahl System Engineering team! datentechnik GmbH & Co.KG Werner von Siemens Straße 12a 49124 Georgsmarienhuette Tel.: +49 (0)5401-8226-50 Fax : +49 (0)5401-8226-55 E-Mail: pahl(a)team-datentechnik.de Internet: www.team-datentechnik.de HRA 110397, Amtsgericht Osnabrück Geschäftsführung: Reemt Lükenga Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese E-Mail. Vielen Dank. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail ist nicht gestattet. This e-mail contains confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender and delete this message. Thank you. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

5 8

proper filtering messages with java Exceptions
by Franciszek Zmarły 24 Jun '10

24 Jun '10

Hello I have a problem with filtering messages having java Exceptions. I send logs from many jvm processes to one syslog-ng server and save them to different files. Example one message from log4j (dump from wireshark) <139>Jun 18 10:53:03 td2/td2 [PROGRAM NAME - MainThread] ERROR - FATAL ERROR <139>Jun 18 10:53:03 td2/td2 java.lang.Exception: Invalid value TryCount: 0 <139>Jun 18 10:53:03 td2/td2 at com.td.Sender.initialize(Sender.java:30) <139>Jun 18 10:53:03 td2/td2 td2 at com.td.Main.main(Main.java:123) I use changed thread name (PROGRAM NAME) as filter in syslog-ng configuration filter f_td_p1 { match("PROGRAM NAME"); }; but in this configuration ony first line of message are saved (from log file) 2010-06-18 10:53:03 td2 [PROGRAM NAME-MainThread] ERROR - FATAL ERROR Is it possible to correct save all logs ? Of course all messages in examle is one log message. Log4j save it like this: 2010-06-18 10:53:03,242 [PROGRAM NAME-MainThread-MainThread] ERROR - FATAL ERROR java.lang.Exception: Invalid value TryCount: 0 at com.td.Sender.initialize(Sender.java:30) at com.td.Main.main(Main.java:123) I want to obtain same effect in syslog-ng. Have you any suggestion ? Please help. Regards Frank

4 4

FreeBSD 8 LOG_SECURITY and LOG_AUTH confused
by Brian A. Seklecki 23 Jun '10

23 Jun '10

Guys something shit the bed with 3.0.3 on 8.0-PL: ----- config ---------------- source src { unix-dgram("/var/run/log"); filter f_security { facility(security); }; filter f_auth { facility(auth); }; destination security { file("/var/log/security" template(cfi_template)); }; destination authlog { file("/var/log/auth.log" template(cfi_template)); }; log { source(src); filter(f_security); destination(security); flags(final); }; log { source(src); filter(f_auth); destination(authlog); flags(final); }; ------------------------- # /usr/local/sbin/syslog-ng -p /var/run/syslog.pid -tevdF Syslog connection established; fd='5', server='AF_INET(192.168.128.XXX:514)', local='AF_INET(0.0.0.0:0)' Running application hooks; hook='1' Running application hooks; hook='3' syslog-ng starting up; version='3.0.3' ^Z [5]+ Stopped /usr/local/sbin/syslog-ng -p /var/run/syslog.pid -tevdF [root@cfi-db-corp ~]# bg [root@cfi-db-corp ~]# echo test | logger -p auth.info Incoming log entry; line='<38>Mar 18 15:57:40 bseklecki: test' Filter rule evaluation begins; filter_rule='f_ams' Filter node evaluation result; filter_result='not-match', filter_type='level' Filter node evaluation result; filter_result='not-match', filter_type='AND' Filter node evaluation result; filter_result='not-match', filter_type='AND' Filter rule evaluation result; filter_result='not-match', filter_rule='f_ams' Filter rule evaluation begins; filter_rule='f_auth' Filter node evaluation result; filter_result='match', filter_type='facility' Filter rule evaluation result; filter_result='match', filter_rule='f_auth' Filter rule evaluation begins; filter_rule='f_ams' Filter node evaluation result; filter_result='not-match', filter_type='level' Filter node evaluation result; filter_result='not-match', filter_type='AND' Filter node evaluation result; filter_result='not-match', filter_type='AND' Filter rule evaluation result; filter_result='not-match', filter_rule='f_ams' Filter rule evaluation begins; filter_rule='f_authpriv' Filter node evaluation result; filter_result='not-match', filter_type='facility' Filter rule evaluation result; filter_result='not-match', filter_rule='f_authpriv' Filter rule evaluation begins; filter_rule='f_kern' Filter node evaluation result; filter_result='not-match', filter_type='facility' Filter rule evaluation result; filter_result='not-match', filter_rule='f_kern' Filter rule evaluation begins; filter_rule='f_user' Filter node evaluation result; filter_result='not-match', filter_type='facility' Filter rule evaluation result; filter_result='not-match', filter_rule='f_user' Filter rule evaluation begins; filter_rule='f_mail' Filter node evaluation result; filter_result='not-match', filter_type='facility' Filter rule evaluation result; filter_result='not-match', filter_rule='f_mail' Filter rule evaluation begins; filter_rule='f_security' Filter node evaluation result; filter_result='match', filter_type='facility' Filter rule evaluation result; filter_result='match', filter_rule='f_security' Initializing destination file writer; template='/var/log/security', filename='/var/log/security' The two fixes I can find are: - Move: "log { source(src); filter(f_security);.." below "log { source(src); filter(f_auth);..." ...wait, what?! - Remove reference to LOG_SECURITY /usr/include/syslog.h defines for Facility->index mappings haven't changed since RELENG_6, so I'm not sure what to make of this. Very strange, ~BAS

2 2

lists.balabit.com / lists.balabit.hu SSL Cert
by Brian A. Seklecki 23 Jun '10

23 Jun '10

All: The list headers have balabit.hu, but the SSL cert is for *.com. Sounds like a nightmare. I suspect Mailman domain name change is a nightmare. Maybe a good chance to experiment with RFC3546 / SNI (SSL Vhosting) :) ~BAS

1 0

Syslog-ng 3.0.5 TLS support
by Prashant Vaidya 16 Jun '10

16 Jun '10

Hi Guys, I think I'm at the right forum and asking right question:- I have installed Syslog-ng 3.0.5 on my linux box. Can some body please guide me how I can use RFC 5426 (UDP support) and RFC5425 (TLS encryption support) between two servers. My current setup is having 4 linux mahcines, I want to configure and use the TLS feature. Can some body please answer to my quaries:- 1:- What could be the corrective steps to test whether TLS is working or not. 2. So I have to configure Syslog-ng with TCP wrapper, SSL , IPV6 and spoofing support. I would be very grateful If someone can answer to above question. Regards, Prashva

3 3

assigning structured data from IETF syslog messages to macros
by Risto Vaarandi 14 Jun '10

14 Jun '10

hi all, I have run into an issue when trying to process structured data from IETF syslog messages. According to syslog-ng administrators guide (I am using syslog-ng 3.0.7), the SDATA.SDID.SDNAME macro can be used for referring structured data. However, when I tried to test the example from the administrator guide, I found that only the $SDATA macro works fine. The example is somewhat confusing, since the SDATA.EXAMPLE.EVENTSOURCE macro allegedly points to the SD ID "exampleSDID@0" (which contains the name eventSource). In order to test syslog-ng with a clearer case, I created the following messages with nc: echo -n '<165>1 2010-06-09T18:00:00.000000+02:00 192.168.1.1 myproc - - [timeQuality tzKnown="0" isSynced="0"] This is a test' | nc -w 1 -u localhost 8514 The syslog-ng configuration looks like this: template format { template("$DATE $HOST [$FACILITY.$LEVEL] $MSG StructuredData = ${SDATA.TIMEQUALITY.TZKNOWN}\n"); }; #template format { template("$DATE $HOST [$FACILITY.$LEVEL] $MSG StructuredData = ${SDATA}\n"); }; source ietfmsg { syslog(ip(0.0.0.0) port(8514) transport(udp)); }; destination netlog { file("/var/log/syslog-ng.log" template(format)); }; log { source(ietfmsg); destination(netlog); }; If the second template definition is active (i.e., the first definition is commented out), everything works fine and I have the following messages in the log: Jun 9 18:00:00 localhost [local4.notice] This is a test StructuredData = [timeQuality tzKnown="0" isSynced="0"] However, when the first template definition is activated, I will get the following output from syslog-ng: Incoming log entry; line='<165>1 2010-06-09T18:00:00.000000+02:00 192.168.1.1 myproc - - [timeQuality tzKnown="0" isSynced="0"] This is a test' No such value known; value='SDATA.TIMEQUALITY.TZKNOWN' Also, the log messages look like this: Jun 9 18:00:00 localhost [local4.notice] This is a test StructuredData = What am I doing wrong? The example from the manual does not seem to work, neither does my simple test case. kind regards, risto

2 2

syslog-ng and network problem
by Giovanni Mancuso 11 Jun '10

11 Jun '10

Hello guys I've installed two syslog-ng server instances behind a load balancer (LB) and configured N syslog-ng clients pointing to my virtual ip address. The clients doesn't send too much messages to the servers at this time. For some reason the LB closes the connections and when my client have to send a message to the server it realize that the connection was closed and it reopen a new one. The problem is that the messages even if the connection is stablished once again doesn't arrive to the server. I tryed also to enable the mark_freq(10) option but nothing changed. It starts to work if I restart the client or if I generate a lot of messages ( echo "MESSAGE" >> FILE.log ) I realize that the problem is in the network, but is someone have another idea to workaround this issue i'll be thankfull! Here is the config of one of my clients: @version: 3.0 #Default configuration file for syslog-ng. # # For a description of syslog-ng configuration file directives, please read # the syslog-ng Administrator's guide at: # # http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/bk01-toc.html ######### # options options { time_reopen(10); keep_hostname(yes); log_msg_size(65535); flush_lines(0); }; ######### # sources source s_balabit { internal(); }; source s_source { file("FILE.log"); }; filter f_filter { match ("OUTGOING" value("MESSAGE")); }; ######### # destinations destination d_internal { file("/var/log/balabit.log"); }; destination d_network { tcp("vip" port(10001) tls( ca_dir("/etc/ssl/syslog-ng/ca.d")) log_fifo_size(4194304) ); }; ######### # logs log { source(s_source); filter(f_filter); destination(d_network);}; log { source(s_balabit); destination(d_internal); };

2 2

syslog-ng 3.0.7 has been released
by devel＠balabit.hu 11 Jun '10

11 Jun '10

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.0.7 SUMMARY : new stable release DATE : Jun 7, 2010 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.0.7) has been released. For latest fixes in the 3.0.x branch you are recommended to upgrade to this version. CHANGES: 3.0.7 Sun, 06 Jun 2010 13:27:38 +0200 Bugfixes: * When syslog-ng is reloaded, the local hostname value was not refreshed, causing syslog-ng to remember the hostname until the next restart. This may not play nice with DHCP configured hostnames, which may change dynamically. * When several SIGHUP signals are received in quick succession, the last one may have been dropped. This problem is fixed. * Make sure that numbers never get resolved using getpwnam/getgrnam (e.g. libnss functions), because this may cause deadlocks if the NSS provider is LDAP and the LDAP server is trying to log a message to syslog about invalid usernames. * Fixed flush_timeout() handling in for non-file destinations. Enabling flush_lines()/flush_timeout() for a non-file destination, either using a global option, or a per-destination option could cause excess CPU usage as long as the destination wasn't writable. Other changes: * Introduced a new, cleaner syntax to specify that the permissions of a file are not to be changed. Use owner/group/perm options of a file without any parameters instead of "-1". The old mechanism had the problem that both per-file and global settings had to use "-1" to effectively stop syslog-ng from changing the uid/gid/perm values. The new syntax will make it possible to specify defaults in the global options section and customize those on a per-file basis. * syslog-ng will not go into the background if either the --syntax-only or --debug options are specified. DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system…

1 0

Message uniqueness
by Dragan Zubac 09 Jun '10

09 Jun '10

Hello I'm using syslog-ng to log messages into PostgreSQL via file logging. I have a problem,because syslog-ng actually sends two messages from log file,instead one at a time. In attachment,You may see two messages that has the same ID number and were sent together,as 'one' message into database procedure. Is it possible to configure syslog-ng to send one message at a time only ? Sincerely Dragan Zubac

3 4

[Bug 85] New: wrong $_YEAR after logrotate
by bugzilla＠bugzilla.balabit.com 09 Jun '10

09 Jun '10

https://bugzilla.balabit.com/show_bug.cgi?id=85 Summary: wrong $_YEAR after logrotate Product: syslog-ng Version: 2.0.x Platform: PC OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: cedric.jeanneret(a)camptocamp.com Type of the Report: --- Estimated Hours: 0.0 Hello, We're using syslog-ng on debian lenny. We use macro like this : destination df_haproxy { file("/srv/syslog/archive/haproxy/$S_YEAR-$S_MONTH-$S_DAY-haproxy.log" create_dirs(yes) group(toto) ); }; We see there's an error when date changes, syslog recreate a new file with the wrong year (i.e. one year too old...) This happens each 1st of month: -rw-r----- 1 root toto 15782746 2010-05-31 23:59 2009-06-01-haproxy.log -rw-r----- 1 root toto 1756029229 2010-06-01 23:39 2010-06-01-haproxy.log More over, syslog-ng seems to use the wrong file for about 20 minutes, then "see" it's the wrong one, and create the good one just later... As said, this behavior is seen only on the 1st of each month... It's really strange. I have to confess, I really don't know if it's a problem for debian, or upstream. Here are informations about version and package : apt-cache show syslog-ng Package: syslog-ng Priority: extra Section: admin Installed-Size: 448 Maintainer: SZALAY Attila <sasa(a)debian.org> Architecture: amd64 Version: 2.0.9-4.1 If you think it's a debian problem, let me know so that I'll open a bug on debian's tracker. Thank you. Best regards, C. Jeanneret -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

Which of these config.php files should I use?
by Wagner Pereira 07 Jun '10

07 Jun '10

Hi, folks. I posted this issue in the cacti mailing list, but maybe the Syslog mailing list should be the better place to post this: ------------------ Hi, friends. After I've installed the syslog-ng plugin for cacti, there are much more files config.php than I'd like! See below: ------------------ /etc/syslog-ng/syslog/config.php /usr/share/cacti/site/include/config.php /var/www/cacti-0.8.1/include/config.php /var/www/cacti-0.8.1/plugins/syslog/config.php ------------------ In the /var/www/cacti-0.8.1/include/config.php one, I've the lines as follows (which is responsible to connect to the cacti database, right?): ------------------ $database_type = "mysql"; $database_default = "cacti"; $database_hostname = "localhost"; $database_username = "admin"; $database_password = "xxxxxxxxx"; $database_port = ""; ------------------ Even after I've made this below, I am still not able to see the Syslog tab in the Cacti web page: /mysqladmin --user=root -p create syslog / /mysql syslog///--user=root -p/// < syslog.sql/ Thanks in advance. Att, -- Wagner Pereira PoP-SP/RNP - Ponto de Presença da RNP em São Paulo CCE/USP - Centro de Computação Eletrônica da Universidade de São Paulo http://www.pop-sp.rnp.br Tel. (11) 3091-8901

1 1

Quick Questions
by Dyne Lai 04 Jun '10

04 Jun '10

Hi Syslog-ng Developers, I have a few general questions regarding syslog-ng as below. 1) What is the type of transmission (Asynchronous/Synchronous) used for the transfer of log messages between server and remote client? 2) I understand that Syslog-ng is able to store unsent messages from remote client to server in the local hardisk during network/system outrage. Will syslog-ng be able to resume with the unsent messages upon the revive of the network/system? If possible, how can we configure/setup to achieve that? 3) In view of security, how can we configure remote client and server to perform authentication (using X.509 certificates) during log message forwarding? 4) Is the forwarding of log message between remote client and server consider real-time? (assume syn=0) Do pardon if the questions appears ignorant.Thanks for the time! Best Regards, Dyne Lai

5 7

mis-claimed features: network link compression w/ TLS
by Robin H. Johnson 04 Jun '10

04 Jun '10

The comparison table: http://www.balabit.com/network-security/syslog-ng/comparing/detailed/ Lists: "Support for network link compression when using TLS" But I can't find it in the OSE at all. There are NO calls to the OpenSSL COMP* functions, nor any usage of compression functions that I can find. The Flex and Yacc grammar DO include a 'compress' keyword, but data from it is not actually used. I'm looking for some reasonable compression to cut down on network traffic one some Gentoo boxes where there is a very limited upstream network connection. Traffic is about 2.4GiB/day, over a 1Mbit link. Plain gzip after the fact gets >90% compression, but I'd like the logs in near-realtime to the remote logging destinations (multiple). I do also realize that we're going to be limited by packet compression, unless we keep some form of running dictionary, but I think even with that, we should get a good size reduction. -- Robin Hugh Johnson Gentoo Linux: Developer, Trustee & Infrastructure Lead E-Mail : robbat2(a)gentoo.org GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85

2 1

[Bug 83] New: sql (oracle) queries dont escape single quotes
by bugzilla＠bugzilla.balabit.com 03 Jun '10

03 Jun '10

https://bugzilla.balabit.com/show_bug.cgi?id=83 Summary: sql (oracle) queries dont escape single quotes Product: syslog-ng Version: 3.1.x Platform: Other OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bugzilla.syslogng(a)feystorm.net Type of the Report: --- Estimated Hours: 0.0 SQL inserts with values containing single quotes dont escape the quotes causing the inserts to fail. Jun 1 22:08:24 slider syslog-ng.515[20417]: Error running SQL query; type='oracle', host='', port='', user='report_agent', database='oru23', error='1400: ORA-01400: cannot insert NULL into ("CMS_DBO"."MESSAGE_TRACKING"."MSGID")\x0a', query='INSERT INTO cms_dbo.message_tracking (host, msg_dt, pname, message, msgid, xid, from_name, from_domain, to_name, to_domain, addr) VALUES (\'iceman.dev.usa.net\', \'2010-06-01 22:08:22\', \'smtad\', \'[4155186080] 000000007 ok XID: XID007oFawiw8111Xem MsgID: <4C0584D4.50007(a)devo0402.dev.usa.net> Dest: cms:gwsin IP: 165.212.101.13 From: <phemmer(a)devo0402.dev.usa.net> Rcpt: <phemmer(a)devo0402.dev.usa.net> Subject: \'\'Re: asdf\'\' Size: 22400 Response: 250 Mail accepted (292oFawiw3072Mem)\', NULL, \'XID007oFawiw8111Xem\', \'phemmer\', \'devo0402.dev.usa.net\', \'phemmer\', \'devo0402.dev.usa.net\', \'165.212.101.13\')' The part that is causing the problem is the 'message' field with value: [4155186080] 000000007 ok XID: XID007oFawiw8111Xem MsgID: <4C0584D4.50007(a)devo0402.dev.usa.net> Dest: cms:gwsin IP: 165.212.101.13 From: <phemmer(a)devo0402.dev.usa.net> Rcpt: <phemmer(a)devo0402.dev.usa.net> Subject: ''Re: asdf'' Size: 22400 Response: 250 Mail accepted (292oFawiw3072Mem) ^^that is what is passed in the insert statement. As you can see, there are 2 single-quotes around the subject. The string as it gets passed to syslog-ng is: ...Subject: 'Re: asdf' Size... So instead of escaping the single quote, it adds another quote. The afsql.c code calls dbi_conn_quote_string_copy to escape it, but this is obviously failing to do so (and not only fails, but for some reason adds a second quote. I'm attempting to find a simple fix for this, but there doesnt appear to be a clean & proper way of doing this. But I'll keep digging as this is a fairly significant flaw which I have to get fixed before using this in production. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

syslog-ng
by Kaushal Shriyan 03 Jun '10

03 Jun '10

Hi, Whats the message size a syslog-ng application can accommodate ? Thanks, Kaushal

2 1

log smtp mail messages to a specific file
by Thanasis 01 Jun '10

01 Jun '10

Postfix messages were logged in /var/log/messages. Here is how syslog-ng.conf was (before my changes): ---------------------------------------------------- # cat /etc/syslog-ng/syslog-ng.conf.orig @version: 3.0 # $Header: /var/cvsroot/gentoo-x86/app-admin/syslog-ng/files/syslog-ng.conf.gentoo.3,v 1.1 2010/04/06 02:11:35 mr_bones_ Exp $ # # Syslog-ng default configuration file for Gentoo Linux options { chain_hostnames(no); # The default action of syslog-ng is to log a STATS line # to the file every 10 minutes. That's pretty ugly after a while. # Change it to every 12 hours so you get a nice daily update of # how many messages syslog-ng missed (0). stats_freq(43200); }; source src { unix-stream("/dev/log" max-connections(256)); internal(); file("/proc/kmsg"); }; destination messages { file("/var/log/messages"); }; # By default messages are logged to tty12... destination console_all { file("/dev/tty12"); }; # ...if you intend to use /dev/console for programs like xconsole # you can comment out the destination line above that references /dev/tty12 # and uncomment the line below. #destination console_all { file("/dev/console"); }; log { source(src); destination(messages); }; log { source(src); destination(console_all); }; ------------------------------------------------- I wanted to log postfix messages to a new file: /var/log/mail.log I changed /etc/syslog-ng/syslog-ng.conf like so: # diff syslog-ng.conf.orig syslog-ng.conf 15a16,23 > filter mail { > facility(mail); > }; > > filter notmail { > not facility(mail); > }; > 22a31 > destination mail { file("/var/log/mail.log"); }; 31,32c40,42 < log { source(src); destination(messages); }; < log { source(src); destination(console_all); }; --- > log { source(src); filter(mail); destination(mail); }; > log { source(src); filter(notmail); destination(messages); }; > log { source(src); filter(notmail); destination(console_all); }; ---------------------------------------------------------------------- It worked, or at least I think so, by looking at both /var/log/messages and /var/log/mail.log. So ,my questions are: 1) Are my edits OK? Did I do anything wrong? 2) Should I have configured it otherwise, perhaps more efficiently? Thanks. Thanasis

3 4