- syslog-ng - lists.balabit.hu

Using tail option
by James B Horwath 15 Feb '09

15 Feb '09

All: This question refers to syslog-ng 2.0.10 running on AIX. If I use a global syslog-ng client file and define 2 text files that do not exist on every system. What happens when syslog-ng is run on a system that does not contain the files? Does it report errors, have performance problems? Example: source s_tail1 { file("/var/apache/log/access_log" follow_freq(1) flags(no-parse)); }; source s_tail2 { file("/var/apache/log/error_log" follow_freq(1) flags(no-parse)); }; Please advise, Jim ----------------------------------------- This message, and any attachments to it, may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are notified that any use, dissemination, distribution, copying, or communication of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately by return e-mail and delete the message and any attachments. Thank you.

2 1

Ip filtering
by Chris Bland 15 Feb '09

15 Feb '09

Hi guys, I am having a problem filtering based on ip address. My predecessor had a config file with over 240 hosts. I just upgraded to 2.0.10-1 and starting getting YACC overflow stack errors. I condensed the config file by using regex. If I stop using DNS everything works fine. What I am trying to do is filter based on ip address and have each log stored in a directory with the servers dns name. My config looks like this options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); keep_hostname (yes); use_dns (yes); use_fqdn (no); create_dirs (no); }; destination servers { file("/var/log/syslog-ng/hosts/$HOST_FROM/$YEAR/$MONTH/$YEAR-$MONTH-$DAY" owner(root) group(staff) perm(0650) dir_perm(0750) create_dirs(yes)); }; destination teaneck { file("/var/log/syslog-ng/network/TEANECK/$HOST_FROM/$YEAR/$MONTH/$YEAR-$MONTH-$DAY" owner(root) group(staff) perm(0650) dir_perm(0750) create_dirs(yes)); }; destination dslswitches { file("/var/log/syslog-ng/network/TEANECK/Dslswitches/$HOST_FROM/$YEAR/$MONTH/$YEAR-$MONTH-$DAY" owner(root) group(staff) perm(0650) dir_perm(0750) create_dirs(yes)); }; filter f_teaneck { netmask("132.238.11.0/24"); }; filter f_usas { host("132.238.(7|8).*$"); }; filter f_dslswitches { host("132.238.21.40"); }; log { source(s_network); filter(f_usas); destination(servers); }; log { source(s_network); filter(f_teaneck); destination(teaneck); }; log { source(s_network); filter(f_dslswitches); destination(dslswitches); }; The netmask filters work by my host filters don't -Chris

2 1

typo in v.3.0 admin guide
by Philipp 14 Feb '09

14 Feb '09

Hi please note that in the online available syslog-ng-v3.0-guide-admin-en.pdf on page 54 this line seems to have typos in the template() part: destination d_file { file("/var/log/messages" template("$ISODATE $HOST $MSG\n"; template_escape(no))); }; Best regards, Philipp

2 2

question on logging templates: strange log lines
by Philipp 14 Feb '09

14 Feb '09

Dear list users In the near future I would like to migrate from syslog-ng 2 to 3.x. For this reason I've set up a linux testbox. Now, I have a problem I lack to see a solution for. The syslog-ng testing config: @version:3.0 ## GLOBAL options { chain_hostnames(no); keep_hostname(yes); flush_lines(0); stats_freq(3600); use_dns(persist_only); dns_cache_hosts(/etc/hosts); }; ## SOURCES source src { unix-stream("/dev/log"); }; source src-kernel { file("/proc/kmsg" program_override("kernel: ")); }; source src-syslog-ng { internal(); }; ## LOGGING TEMPLATES template t_filetmpl { template("$DATE $FACILITY.$PRIORITY $FULLHOST $MSGHDR$MSG\n"); template_escape(no); }; ## DESTINATIONS destination local4 { file("/var/log/local4.log" template(t_filetmpl)); }; ## FILTERS filter f_local4 { facility(local4); }; ## LOGS log { source(src); filter(f_local4); destination(local4); }; For testing I use the logger command: ` logger -p local4.info -t test testnachricht2`. This configuration causes, and I do not see why, lines like these in the /var/log/local4.log file. [empty line] testnachricht2 test: donald.info local4Feb 11 14:26:01Feb 11 14:26:01 local4.info donald test: testnachricht2 However, if I replace the line below "## DESTINATIONS" with destination local4 { file("/var/log/local4.log" template("$DATE $FACILITY.$PRIORITY $FULLHOST $MSGHDR$MSG\n")); }; all is perfect. The logged line looks like this: Feb 11 14:30:22 local4.info donald test: testnachricht2 Please could you tell me what I've missed? Thanks in advance, Philipp

2 3

Strange template definition problem
by Peter Eckel 14 Feb '09

14 Feb '09

Hi, I am currently using the Open Source edition of syslog-ng version 3.0.1 on RedHat Linux 5.3 and 3.9. So far, there are not too many problems, but one strangeness occurs when defining templates. When I define the template inside the 'destination' configuration, everything works as expected: # + # Definition 1: Works fine. # - source s_local { unix-stream( "/dev/log" ); file( "/proc/kmsg" ); udp( ip( "127.0.0.1" ) port(514) ); tcp( ip( "127.0.0.1" ) port(514) keep-alive(yes) ); }; destination d_test { file( "/opt/sec/var/log/test" template( "$ISODATE -- $FACILITY -- $PRIORITY -- $HOST -- $MESSAGE\n----------\n" ) ); }; log { source(s_local); destination(d_test); }; The log entries look like this: >2009-01-22T19:33:35+01:00 -- authpriv -- notice -- linux4 -- LOG5[17309:3086130064]: syslog_in connected from 192.168.42.129:41105 >---------- >2009-01-22T19:33:35+01:00 -- authpriv -- notice -- linux4 -- LOG5[17309:3086060432]: syslog_in connected from 192.168.42.133:33950 >---------- >2009-01-22T19:33:35+01:00 -- authpriv -- notice -- linux4 -- LOG5[17309:3085990800]: syslog_in connected from 192.168.42.128:43163 >---------- So far, so good. But when I try global templates, I get a rather strange result: # + # Definition 2: Fails miserably. # - source s_local { unix-stream( "/dev/log" ); file( "/proc/kmsg" ); udp( ip( "127.0.0.1" ) port(514) ); tcp( ip( "127.0.0.1" ) port(514) keep-alive(yes) ); }; template t_test { template( "$ISODATE -- $FACILITY -- $PRIORITY -- $HOST -- $MESSAGE\n----------\n" ); }; destination d_test { file( "/opt/sec/var/log/test" template( t_test ) ); }; log { source(s_local); destination(d_test); }; What I'm getting is this (including marked blank lines): > >---------- >-- LOG5[17309:3086130064]: syslog_in connected from 192.168.42.129:41105 -- linux4 -- notice -- authpriv2009-01-22T19:33:35+01:0020 >09-01-22T19:33:35+01:00 -- authpriv -- notice -- linux4 -- LOG5[17309:3086130064]: syslog_in connected from 192.168.42.129:41105 >---------- > >---------- >-- LOG5[17309:3086060432]: syslog_in connected from 192.168.42.133:33950 -- linux4 -- notice -- authpriv2009-01-22T19:33:35+01:0020 >09-01-22T19:33:35+01:00 -- authpriv -- notice -- linux4 -- LOG5[17309:3086060432]: syslog_in connected from 192.168.42.133:33950 >---------- > >---------- >-- LOG5[17309:3085990800]: syslog_in connected from 192.168.42.128:43163 -- linux4 -- notice -- authpriv2009-01-22T19:33:35+01:0020 >09-01-22T19:33:35+01:00 -- authpriv -- notice -- linux4 -- LOG5[17309:3085990800]: syslog_in connected from 192.168.42.128:43163 It looks a bit as if all elements of the filled template are output twice: First in reverse order, then again from start in correct order. This is reproducable on several machines. The only difference is the use of the global template in the second case instead of the inline template in the first. Am I making some stupid mistake, or is this a bug in syslog-ng? Best regards and thanks in advance for any suggestions, Peter. -- ------------------------------------------------------------------------- Peter Eckel System- und Netzwerkmanagement Postfach 1463 D-61284 Bad Homburg v.d.H. Tel.: +49 6172 969495 Mobil: +49 172 6812107 Fax: +49 6172 969845 EMail: Peter.Eckel(a)Eckel-EDV.de -------------------------------------------------------------------------

2 4

patches ported from syslog-ng PE 3.0.2
by Balazs Scheidler 14 Feb '09

14 Feb '09

Hi, Just as I outlined in my previous mail, here goes the list of patches that were ported to syslog-ng OSE from the PE 3.0.2 release. You can find more information in the git tree at http://git.balabit.hu/, but feel free to ask specific questions if you need further information. Balazs Scheidler (30): don't use long double in the parser [tgzbuild] fixed --with-pidfile-dir configure option Fixed: slng_start checks whether syslog-ng already running before starting a new session. (fixes: #16011) Fixed: logic error when checking whether an instance already running. (fixes: #16011) Fixed: On SuSE -n no longer echoed (fixes: #16025) Fixed: obscure missing file error on HP-UX (fixes: #16710) Fixed: Init scripts link /var/run/syslog.pid to syslog-ng's pid. (fixes: #16710) Fixed: Solaris 10 method script links /var/run/syslog.pid to syslog-ng's pid. (fixes: #16710) [config parser] fixed a failed assert in case an include directory does not exist [security] user names were not resolved properly, do it right this time fixed compilation warnings fixed LogMessageMatch structure layout on big-endian 32 bit machines (like PA-RISC) Fixed: init.d script uses different syslogd pid on redhat (fixes: #16710) Fixed: init script don't start syslog-ng when it doesn't run (fixes: #16011) really only move to the newly rotated logfile if the new one already has data [logproto] remember "encoding" value, so that log messages can contain it [logproto] properly handle incomplete encodings (fixes: #15680) [logproto] don't let the conversion buffer increase to unlimited sizes Fixed: init script on solaris 8 links /etc/syslog.pid to pidfile. (fixes: #16710) [security] enable SSL X.509 policy check [tlscontext] reorganize cert validation somewhat, fixed "required-untrusted" (fixes: #16671) [tlscontext] added basicConstraints validation even for trusted CA certificates [afsocket] fixed a possible reference of invalid file descriptors [main] never initialize the configuration in the supervisor process kill the syslog-ng process if reverting to the old config fails fixed compilation error on FreeBSD [templates] use the correct timezone offset when time_zone_info_get_offset() returns -1 [config parser] accept newly introduced keywords as identifiers (fixes: #17653) [config parser] removed unused KW_FRAMED keyword [configure] change default linking mode to "dynamic" Tamas Pal (1): Fixed: on linux, dash no longer prinst -e (fixes: #17559) -- Bazsi

1 0

syslog-ng OSE patch flow
by Balazs Scheidler 14 Feb '09

14 Feb '09

Hi, In order to improve transparency in the syslog-ng development process, I'm going to try to improve the communication about the changes that go into syslog-ng OSE. Currently I see three major ways to get patches into syslog-ng: * Patches that get implemented because one of the list members request it are already posted on this list. * Patches that are submitted to bugzilla.balabit.com are also posted here, because of the bugzilla -> email gateway that I set up. * A third portion of patches that are the results of BalaBit's internal development projects were not posted here so far, simply because I took the internal git tree and ported patches to the open source branch that were relevant. I hope to improve the latter point. At first I thought about "submitting" the patches openly to this mailing list, but then I thought it'd be a mail storm every time I started doing the patch porting. (for instance just today I ported over 30 patches) Also it is not the way I do the work: I use git directly, doing this based on emails would be more difficult for me. Therefore I decided to post a single mail, every time I port patches back-and-forth. This mail would contain a summary for the patches I ported. The summary would be generated using "git-shortlog", more details are always available in the public git repositories at "git.balabit.hu". Currently the author information of my git trees are not perfect, but some changes were already made to improve the accuracy of these fields. Hopefully by the end of the next syslog-ng PE cycle, these would contains the names for the guys who actually worked on this stuff. That's the basic idea. I'm going to submit the current list of patches in a separate mail. -- Bazsi

1 0

3.01 facility(syslog) returns an error
by Tim Boyer 13 Feb '09

13 Feb '09

Continuing with my 3.01 install. My .conf files contains on line 61: filter fprinceton_syslog { facility(syslog); }; [root@princeton src]# ./syslog-ng -s syntax error in /usr/local/etc/syslog-ng.conf at line 61. However, if I change that to the numeric equivalent: filter fprinceton_syslog { facility(5); }; [root@princeton src]# ./syslog-ng -s [root@princeton src]# the error goes away. -- Tim Boyer Denman Tire Corporation

2 2

syslog-ng logging with wrong year
by Mayashankar Mishra 10 Feb '09

10 Feb '09

I have syslog-ng 2.1.1 running, still i face the problem when the date changed from 31 Dec 2008 to 1 Jan 2009, some of the log files still log as 1 Jan 2008 Is it resolved in higher version, and if yes which ? Mayashankar Mishra Consultant E-mail : mmishra(a)isabel.eu Tel : +32 (0)2 403.18.84 Fax : +32 Isabel NV/S.A. Keizerinlaan 13-15 Boulevard de l'Imp?ratrice 1000 Brussels - Belgium RPR Bruxelles / RPM Brussel: BE 0455 530 509 www.isabel.eu<http://www.isabel.eu/> www.zoomit.eu<http://www.zoomit.eu/> Zoomit is a Registered Trademark of Isabel NV/S.A. Disclaimer : www.isabel.eu/gps/en/disclaimer/mailing.php<http://www.isabel.eu/gps/en/disclaimer/mailing.php>

2 2

Still not working at the most basic level
by lance raymond 10 Feb '09

10 Feb '09

ok, so can anyone say how to troubleshoot the most basic scenario to start. 1 server, 1 client, have client write logs to server? I will show both config files, please note tI have tried both UDP and TCP and neither work. I did notice doing an lsof |grep syslog that the port does change from udp to tcp so I know it's reading the config file, I just don't know how to turn on some debugging or another way to trace the log as it's still being written to the client. ok, so *server config is;* #source s_remote { udp(); }; source s_remote { tcp(); }; ## This will create seprate file for each client on central log server and log http messages destination d_clients { file("/var/log/web.$HOST.log"); }; log { source(s_remote); destination(d_clients); }; ################################################################# options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); #udp(ip(0.0.0.0) port(514)); }; ## This will log local http messages to defined file destination send_http_logs { file("/var/log/web.log"); }; filter send_http_logs { program("httpd.*"); }; log { source(s_sys); filter(send_http_logs); destination(send_http_logs); }; and the server shows the following running process; root 30945 0.0 0.0 7240 636 ? Ss 11:24 0:00 /sbin/syslog-ng -p /var/run/syslogd.pid *Client side:* options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames(on); use_dns(yes); dns_cache(yes); use_fqdn(no); create_dirs (yes); keep_hostname (yes); perm(0640); dir_perm(0750); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); }; #destination send_http_logs { udp("192.168.2.54" port(514)); }; destination send_http_logs { tcp("192.168.2.54" port(514)); }; filter send_http_logs { program("httpd.*"); }; log { source(s_sys); filter(send_http_logs); destination(send_http_logs); }; and the running process are; root 1607 0.0 0.0 6216 904 ? Ss 2008 0:48 syslogd -m 0 root 28374 0.0 0.0 7368 564 ? Ss 11:24 0:00 /sbin/syslog-ng -p /var/run/syslogd.pid Not sure if the client is supposed to have 2 processes or not, but any help is really appreciated.

2 3