March 2013 - syslog-ng - lists.balabit.hu

Upcoming changes regarding my Debian/Ubuntu packages
by Gergely Nagy 04 Jul '13

04 Jul '13

Hi! For a good while now, I've been providing Debian and Ubuntu packages[1] for syslog-ng, and rest assured, I will continue to do so. However, the way they are prepared and how the repository is structured, will change. At the moment, there are two different components one can choose from: syslog-ng and syslog-ng-devel, which have packages for 3.3 and 3.4, respectively. However, -devel is not appropriate for 3.4 anymore, but I don't want to move it to syslog-ng either, partly because up to this point, there were no Ubuntu Lucid packages of 3.4. So, I will be making the following changes: * I'll create syslog-ng-3.3, syslog-ng-3.4 and syslog-ng-3.5 components * syslog-ng will point to the latest stable branch (3.4) * syslog-ng will point to the current development branch (3.5) * I'll be preparing Ubuntu Lucid packages for 3.4 too, so all branches will be available for every debian or ubuntu system that are still supported by their respective vendors. To be able to provide Ubuntu Lucid (10.04) packages sanely, I have to do the packaging a little bit differently: the source package will not be shared between debian and ubuntu anymore, and systemd support will be explicitly disabled on Ubuntu. But since Ubuntu uses upstart by default, ubuntu systems with systemd are fairly unlikely anyway. I plan to make these changes once 3.4.2 comes out. Until then, I'll introduce the syslog-ng-3.3 and -3.4 components, but syslog-ng will point to -3.3 still, and -devel to -3.4. Once 3.4.2 is out, though, I'll change the symlinks. [1]: http://asylum.madhouse-project.org/projects/debian/ -- |8]

1 2

[Bug 223] New: syslog-ng doesn' t start on windows running in a cygwin environment
by bugzilla＠bugzilla.balabit.com 24 May '13

24 May '13

https://bugzilla.balabit.com/show_bug.cgi?id=223 Summary: syslog-ng doesn't start on windows running in a cygwin environment Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Windows Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: tps(a)vr-web.de Type of the Report: bug Estimated Hours: 0.0 After having installed syslog-ng and given dependencies, I've configured with /usr/bin/syslog-ng-config No errors that far. Then started it with "net start syslog-ng": $ net start syslog-ng CYGWIN syslog-ng wird gestartet. CYGWIN syslog-ng konnte nicht gestartet werden. Der Dienst hat keinen Fehler gemeldet. Sie erhalten weitere Hilfe, wenn Sie NET HELPMSG 3534 eingeben. syslog-ng did not start. Trying to start with "cygrunsvc -S syslog-ng": $ cygrunsrv.exe -S syslog-ng cygrunsrv: Error starting a service: QueryServiceStatus: Win32 error 1062: Der Dienst wurde nicht gestartet. Same as above. Again no usefull error message. Next I tried to start syslog-ng directly with the paramaters passed by cygrunsvc: $ /usr/sbin/syslog-ng.exe -F Error opening plugin module; module='afsocket', error='No such process' Error opening plugin module; module='afsocket', error='No such process' Error parsing source, source plugin unix-dgram not found in source confgen system at line 1, column 1: included from /etc/syslog-ng/syslog-ng.conf line 10, column 9 unix-dgram ^^^^^^^^^^ syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng The service does not start because of errors while loading modules. Given "-V" syslog-ng reports: $ /usr/sbin/syslog-ng.exe -V syslog-ng 3.2.5 Installer-Version: 3.2.5 Revision: Compile-Date: Apr 3 2012 14:52:24 Enable-Threads: off Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: on Enable-SSL: on Enable-SQL: off Enable-Linux-Caps: off Enable-Pcre: on Enable-Pacct: off Files installed by cygwin are: $ cygcheck.exe -l syslog-ng /etc/postinstall/syslog-ng.sh /etc/defaults/etc/syslog-ng/syslog-ng.conf /etc/defaults/etc/syslog-ng/scl.conf /etc/defaults/etc/syslog-ng/modules.conf /usr/bin/update-patterndb /usr/bin/loggen.exe /usr/bin/syslog-ng-config /usr/bin/cygsyslog-ng-0.dll /usr/bin/pdbtool.exe /usr/lib/syslog-ng/cygdbparser.dll /usr/lib/syslog-ng/cygcsvparser.dll /usr/lib/syslog-ng/cygafsocket-tls.dll /usr/lib/syslog-ng/cygbasicfuncs.dll /usr/lib/syslog-ng/cygafprog.dll /usr/lib/syslog-ng/cygconvertfuncs.dll /usr/lib/syslog-ng/cygafuser.dll /usr/lib/syslog-ng/cygconfgen.dll /usr/lib/syslog-ng/cygsyslogformat.dll /usr/lib/syslog-ng/cygafsocket-notls.dll /usr/lib/syslog-ng/cygdummy.dll /usr/lib/syslog-ng/cygafsocket.dll /usr/lib/syslog-ng/cygaffile.dll /usr/sbin/syslog-ng.exe /usr/sbin/syslog-ng-ctl.exe /usr/share/syslog-ng/include/scl/syslogconf/plugin.conf /usr/share/syslog-ng/include/scl/syslogconf/convert-syslogconf.awk /usr/share/syslog-ng/include/scl/syslogconf/README /usr/share/syslog-ng/include/scl/system/plugin.conf /usr/share/syslog-ng/include/scl/system/generate-system-source.sh /usr/share/syslog-ng/include/scl/pacct/plugin.conf /usr/share/syslog-ng/xsd/patterndb-1.xsd /usr/share/syslog-ng/xsd/patterndb-2.xsd /usr/share/syslog-ng/xsd/patterndb-3.xsd /usr/share/man/man5/syslog-ng.conf.5.gz /usr/share/man/man8/syslog-ng.8.gz /usr/share/man/man1/pdbtool.1.gz /usr/share/man/man1/syslog-ng-ctl.1.gz /usr/share/man/man1/loggen.1.gz /usr/share/doc/syslog-ng/AUTHORS /usr/share/doc/syslog-ng/README /usr/share/doc/syslog-ng/COPYING /usr/share/doc/syslog-ng/ChangeLog /usr/share/doc/syslog-ng/NEWS /usr/share/doc/Cygwin/syslog-ng.README The generated config-file "/etc/syslog-ng/syslog-ng.conf" holds: $ cat /etc/syslog-ng/syslog-ng.conf ############################################################################# # Default syslog-ng.conf file which collects all local logs into a # single file called /var/log/messages. # @version: 3.2 @include "scl.conf" source s_local { system(); internal(); }; source s_network { udp(); }; destination d_local { file("/var/log/messages"); }; log { source(s_local); # uncomment this line to open port 514 to receive messages #source(s_network); destination(d_local); }; Looks like a bug analyzing the installed files correctly and creating a not working setup. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

syslog-ng 3.3.6 Hangs on Solaris 8 sparc
by Rakesh Rajasekharan 02 May '13

02 May '13

Hi All, I have compiled the syslog-ng version 3.3.6 on a Solaris Sparc machine . The compilation went through fine but when I try to run syslog-ng .. though the daemon gets running somehow the unix prompt is not returned.. and it hangs... For ex:- # ps -ef|grep -i syslog root 14073 11758 0 07:11:37 pts/4 0:00 vi syslog-ng.d root 14097 13526 0 07:15:48 pts/1 0:00 grep -i syslog Now if I start the daemon using the below options... # /opt/soe/local/syslog-ng-3.3.6/sbin/syslog-ng -f /opt/soe/local/syslog_ng/etc/syslog-ng/syslog-ng.conf -p /var/run/syslog-ng.pid I never get back the prompt.. but if on another window I check.. the daemon has started.. ps -ef|grep -i syslog root 14099 11758 0 07:16:17 pts/4 0:00 grep -i syslog root 14098 13526 23 07:16:04 pts/1 0:13 /opt/soe/local/syslog-ng-3.3.6/sbin/syslog-ng -f /opt/soe/local/syslog_ng/etc/s though I guess... i should expect one more child to this... can someone please help me address this issue please... Thanks, Rkaesh.

3 10

[Bug 229] New: json parser does not maintain all json object data
by bugzilla＠bugzilla.balabit.com 30 Apr '13

30 Apr '13

https://bugzilla.balabit.com/show_bug.cgi?id=229 Summary: json parser does not maintain all json object data Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 Then the json parser is used parser p_json { json-parser(); }; log { source(mysource); parser(p_json); ... } tag-value pairs are not maintained if they are ones that are normally populated by syslog-ng. Even with a source that has flags(no-parse) man macros are populated by syslog-ng C_* S_* R_* YEAR_DAY YEAR WEEKDAY_NAME WEEK_DAY_ABBREV WEEK_DAY WEEKDAY WEEK USEC UNIXTIME TZOFFSET TZ TAGS SYSUPTIME STAMP SEC MSEC MONTH_WEEK MONTH_NAME MONTH_ABBREV MONTH MIN LOGHOST ISODATE HOUR12 HOUR FULLDATE DAY DATE AMPM the json parser should populate the macros after the initial population allowing the json data to overwrite/control all of the macros. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

[Bug 227] New: Feature request – Linux /dev/kmsg support
by bugzilla＠bugzilla.balabit.com 12 Apr '13

12 Apr '13

https://bugzilla.balabit.com/show_bug.cgi?id=227 Summary: Feature request – Linux /dev/kmsg support Product: syslog-ng Version: unspecified Platform: PC OS/Version: Linux Status: NEW Severity: minor Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: grawity(a)gmail.com Type of the Report: enhancement Estimated Hours: 0.0 Starting with Linux 3.5 (commits [1], [2]), there is a new kmsg interface "/dev/kmsg", which allows multiple readers, is able to export some structured data (such as device name and subsystem), avoids some other problems /proc/kmsg had, such as the ones described in syslog-ng's FAQ [3]. Are there any plans to support this in syslog-ng? The format is explained in Documentation/ABI/testing/dev-kmsg.txt [4]. [1]: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=… [2]: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=… [3]: http://www.balabit.com/wiki/syslog-ng-faq-kmsg-klogd [4]: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/plain/Docum… -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

how to archive logs efficiently
by Daniel Neubacher 05 Apr '13

05 Apr '13

Hey there, my syslog-ng has gotten quite big with 50k logs per second and the server seems to hit the io limit at night. While a few month ago I could run a gzip with ionice over all old logs the server doesn't like it anymore and quite a lot of logs are storing while the compression lasts. I'm using the ose so I've got no logstore. And for a second I've thought about writing the logs a compressed fuse fs but... fuse :P So how are you guys doing it? -- Daniel Neubacher, Network Administrator daniel.neubacher(a)xing.com<mailto:daniel.neubacher@xing.com> XING AG Gaensemarkt 43, 20354 Hamburg, Germany Tel. +49 40 419131-28, Fax +49 40 419131-11 Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 98807 Exec. Board (Vorstand): Dr. Stefan Groß-Selbeck (Vorsitzender), Dr. Thomas Vollmoeller, Ingo Chu, Dr. Helmut Becker, Jens Pape Chairman of the Supervisory Board (Aufsichtsratsvorsitzender): Dr. Neil Sunderland This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden and may be unlawful.

5 7

OSE 3.4(.1) block() argument parse issue
by Eric Berggren 30 Mar '13

30 Mar '13

syslog-ng OSE 3.4.1 is unable to parse the "Example 5.8: Passing arguments to blocks" excerpt in the v3.4 admin guide, which exemplifies a common construct used in our current 3.2 config : @version: 3.4 block source s_logfile (filename("messages")) { file("/var/log/`filename`" ); }; source s_example { s_logfile(filename("logfile.log")); }; host# ./syslog-ng -s -f ./s2.conf : Error parsing affile, syntax error, unexpected LL_IDENTIFIER in source block s_logfile at line 2, column 19: included from /home/eberggren/proj/syslog-ng/config/s2.conf line 7, column 36 file("/var/log/"logfile.log ^^^^^^^^^^^ 3.3.8 appears to be okay with it, however (and our existing 3.2 config). thanks, -eric

2 2

[Bug 225] New: patterndb can not use HOST as variable in pattern
by bugzilla＠bugzilla.balabit.com 30 Mar '13

30 Mar '13

https://bugzilla.balabit.com/show_bug.cgi?id=225 Summary: patterndb can not use HOST as variable in pattern Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 This fails in the patterndb test --validate with the error ERROR:logmsg.c:540:log_msg_set_value_indirect: assertion failed: (handle >= LM_V_MAX) <rule id="50b9250d-2191-43af-8008-089c50224dc2" class="erempel" provider="UVic"> <patterns> <pattern>replace HOST with @ANYSTRING:HOST@</pattern> </patterns> <examples> <example> <test_message program="something">replace HOST with some.host.here</test_message> <test_values> <test_value name="HOST">some.host.here</test_value> </test_values> </example> </examples> </rule> but this succeeds and replaces the HOST macro <rule id="50b9250d-2191-43af-8008-089c50224dc2" class="erempel" provider="UVic"> <patterns> <pattern>replace HOST with @ANYSTRING:tempHost@</pattern> </patterns> <values> <value name="HOST">$tempHost</value> </values> <examples> <example> <test_message program="something">replace HOST with some.host.here</test_message> <test_values> <test_value name="tempHost">some.host.here</test_value> </test_values> </example> </examples> </rule> I would expect the both to work. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

3 7

How to build a filter where one message from one host not logged
by Niels Graupe 29 Mar '13

29 Mar '13

Hi ! I want to filter with syslog-ng that a message from a special host with a special message was not filtered and everything else goes to the logfile filter f_cisco_switches_knownerrors1 { host ("hostxy") and match("ENVIRONMENT-3-RPS_FAILED"); }; The Log-Message : Mar 29 12:29:19 hostxy err 20979: Mar 29 12:29:18: %ENVIRONMENT-3-RPS_FAILED: Faulty internal power supply detected Thanks, Niels

2 1

Closing destination file when date changes.
by andrea.meconi＠libero.it 23 Mar '13

23 Mar '13

Hello. I'm using a very simple configuration on Ubuntu 12.04.2 LTS @version: 3.3 @include "scl.conf" source from-network { udp(ip("0.0.0.0") port(514)); }; destination local-file { file("/var/log/temporary-encrypted/$YEAR-$MONTH-$DAY.log"); }; log { source(from-network); destination(local-file);}; Log on a destination file using date as filename. I need to close this file when date changes at 00:00. How? Thanks. Andrea

4 3