source file wildcard problems
Hi, I am having trouble getting the new file wildcard option to work. I am testing it with the following very simple file-to-file configuration: @version: 3.0 options { time_sleep(500); # polling interval, in ms (make this once per second) use_fqdn(yes); # use fully qualified domain names ts_format(iso); # use ISO8601 timestamps # for normal load flush_lines (10); # number of lines to buffer before writing to disk flush_timeout (1000); # 1 second timeout log_fifo_size(100); stats_freq(3600); # number of seconds between syslog-ng internal stats events; these are useful # for ensuring syslog-ng is not getting overloaded }; # Debugging source syslog_ng { internal(); }; destination debug_dest { file("/u/dang/local/var/log/syslog-ng- internal.log" perm(0644) ); }; log { source(syslog_ng); destination(debug_dest); }; # Other source simple_src { file("/u/dang/local/var/log/myfile.log" follow_freq(1) flags(no-parse) program_override("my_log ") ); }; source test_src { file ("/u/dang/local/var/log/test*.log" follow_freq(1) flags(no-parse) program_override("test_log ") ); }; destination test_dest { file ("/u/dang/local/var/log/collected.log" perm(0644) ); }; log { source(simple_src); destination(test_dest); }; log { source(test_src); destination(test_dest); }; The behavior I am seeing is that anything appended to the static file "myfile.log" shows up in "collected.log" a second later. But new files of the pattern "test-1.log", etc. are never forwarded, and if there are existing files of this name at startup they are ignored. The internal log says: 2009-03-25T10:02:20-07:00 host.org syslog-ng[29624]: Follow-mode file source not found, deferring open; filename='/u/dang/local/var/log/ test*.log' 2009-03-25T10:02:20-07:00 host.org syslog-ng[29626]: syslog-ng starting up; version='3.0.1' My syslog-ng version is: -bash-3.1$ syslog-ng -V syslog-ng 3.0.1 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng- ose--mainline--3.0#master#93a342dae3a2b0cb15811d0c34ea7f58b3fba14e Compile-Date: Mar 25 2009 09:08:54 Enable-Threads: off Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-Sun-Door: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: on Enable-SSL: on Enable-SQL: off Enable-Linux-Caps: off Enable-Pcre: off Any help or tips would be appreciated. Thanks, -Dan
I've tried all of the following: 1a) destination d_loghost { syslog(ip"169.237.32.44" keep-alive(yes) localport(1999)); }; 1b) destination d_loghost { syslog(ip(169.237.32.44) keep-alive(yes) localport(1999)); }; 2a) destination d_loghost { syslog(ip"169.237.32.44" ; keep-alive(yes) localport(1999)); }; 2b) destination d_loghost { syslog(ip(169.237.32.44) ; keep-alive(yes) localport(1999)); }; and other combinations by removing the options keep-alive(yes) and/or localport(1999)); However, each time I get a syntax error report. I've tried following the manual, but I'm afraid it seems not too carefully written. The reference at points says something while the given examples are following some other syntax. my syslog-ng is built from the head version of the OSE, downloaded today. Thanks for your help... Ritesh On Mar 25, 2009, at 10:04 AM, Dan Gunter wrote:
Hi,
I am having trouble getting the new file wildcard option to work. I am testing it with the following very simple file-to-file configuration:
@version: 3.0 options { time_sleep(500); # polling interval, in ms (make this once per second) use_fqdn(yes); # use fully qualified domain names ts_format(iso); # use ISO8601 timestamps # for normal load flush_lines (10); # number of lines to buffer before writing to disk flush_timeout (1000); # 1 second timeout log_fifo_size(100); stats_freq(3600); # number of seconds between syslog-ng internal stats events; these are useful # for ensuring syslog-ng is not getting overloaded }; # Debugging source syslog_ng { internal(); }; destination debug_dest { file("/u/dang/local/var/log/syslog-ng- internal.log" perm(0644) ); }; log { source(syslog_ng); destination(debug_dest); }; # Other source simple_src { file("/u/dang/local/var/log/myfile.log" follow_freq(1) flags(no-parse) program_override("my_log ") ); }; source test_src { file ("/u/dang/local/var/log/test*.log" follow_freq(1) flags(no-parse) program_override("test_log ") ); }; destination test_dest { file ("/u/dang/local/var/log/collected.log" perm(0644) ); }; log { source(simple_src); destination(test_dest); }; log { source(test_src); destination(test_dest); };
The behavior I am seeing is that anything appended to the static file "myfile.log" shows up in "collected.log" a second later. But new files of the pattern "test-1.log", etc. are never forwarded, and if there are existing files of this name at startup they are ignored. The internal log says:
2009-03-25T10:02:20-07:00 host.org syslog-ng[29624]: Follow-mode file source not found, deferring open; filename='/u/dang/local/var/log/ test*.log' 2009-03-25T10:02:20-07:00 host.org syslog-ng[29626]: syslog-ng starting up; version='3.0.1'
My syslog-ng version is:
-bash-3.1$ syslog-ng -V syslog-ng 3.0.1 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog- ng- ose--mainline--3.0#master#93a342dae3a2b0cb15811d0c34ea7f58b3fba14e Compile-Date: Mar 25 2009 09:08:54 Enable-Threads: off Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-Sun-Door: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: on Enable-SSL: on Enable-SQL: off Enable-Linux-Caps: off Enable-Pcre: off
Any help or tips would be appreciated.
Thanks,
-Dan
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
On Wed, 2009-03-25 at 19:59 -0700, Ritesh Sood wrote:
I've tried all of the following: 1a) destination d_loghost { syslog(ip"169.237.32.44" keep-alive(yes) localport(1999)); }; 1b) destination d_loghost { syslog(ip(169.237.32.44) keep-alive(yes) localport(1999)); };
2a) destination d_loghost { syslog(ip"169.237.32.44" ; keep-alive(yes) localport(1999)); }; 2b) destination d_loghost { syslog(ip(169.237.32.44) ; keep-alive(yes) localport(1999)); };
and other combinations by removing the options keep-alive(yes) and/or localport(1999));
However, each time I get a syntax error report.
Try: destination d_loghost { syslog("169.237.32.44" keep-alive(yes) localport(1999) ); }; Note: indent is not important, just for better readability. M -- Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D
On Mar 26, 2009, at 3:59 AM, ILLES, Marton wrote:
On Wed, 2009-03-25 at 19:59 -0700, Ritesh Sood wrote:
I've tried all of the following: 1a) destination d_loghost { syslog(ip"169.237.32.44" keep-alive(yes) localport(1999)); }; 1b) destination d_loghost { syslog(ip(169.237.32.44) keep-alive(yes) localport(1999)); };
2a) destination d_loghost { syslog(ip"169.237.32.44" ; keep- alive(yes) localport(1999)); }; 2b) destination d_loghost { syslog(ip(169.237.32.44) ; keep- alive(yes) localport(1999)); };
and other combinations by removing the options keep-alive(yes) and/or localport(1999));
However, each time I get a syntax error report.
Try:
destination d_loghost { syslog("169.237.32.44" keep-alive(yes) localport(1999) ); };
Note: indent is not important, just for better readability.
Tell you what, it worked ! Thanks a lot. What also works is source s_remote { syslog(ip(169.237.32.44) max-connections(50)); }; some consistency, please. Ritesh
M -- Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
On Thu, 2009-03-26 at 07:44 -0700, Ritesh Sood wrote:
On Mar 26, 2009, at 3:59 AM, ILLES, Marton wrote:
On Wed, 2009-03-25 at 19:59 -0700, Ritesh Sood wrote:
I've tried all of the following: 1a) destination d_loghost { syslog(ip"169.237.32.44" keep-alive(yes) localport(1999)); }; 1b) destination d_loghost { syslog(ip(169.237.32.44) keep-alive(yes) localport(1999)); };
2a) destination d_loghost { syslog(ip"169.237.32.44" ; keep- alive(yes) localport(1999)); }; 2b) destination d_loghost { syslog(ip(169.237.32.44) ; keep- alive(yes) localport(1999)); };
and other combinations by removing the options keep-alive(yes) and/or localport(1999));
However, each time I get a syntax error report.
Try:
destination d_loghost { syslog("169.237.32.44" keep-alive(yes) localport(1999) ); };
Note: indent is not important, just for better readability.
Tell you what, it worked ! Thanks a lot. What also works is source s_remote { syslog(ip(169.237.32.44) max-connections(50)); }; some consistency, please.
it is consistent, but possibly with a logic different than yours. sources and destinations are not completely symmetric. for sources the IP address denotes the bind address, and as such it is optional (and defaults to 0.0.0.0). For destinations it is the target IP address and as such it is required. -- Bazsi
On Wed, 2009-03-25 at 10:04 -0700, Dan Gunter wrote:
Hi,
I am having trouble getting the new file wildcard option to work. I am testing it with the following very simple file-to-file configuration:
@version: 3.0 options { time_sleep(500); # polling interval, in ms (make this once per second) use_fqdn(yes); # use fully qualified domain names ts_format(iso); # use ISO8601 timestamps # for normal load flush_lines (10); # number of lines to buffer before writing to disk flush_timeout (1000); # 1 second timeout log_fifo_size(100); stats_freq(3600); # number of seconds between syslog-ng internal stats events; these are useful # for ensuring syslog-ng is not getting overloaded }; # Debugging source syslog_ng { internal(); }; destination debug_dest { file("/u/dang/local/var/log/syslog-ng- internal.log" perm(0644) ); }; log { source(syslog_ng); destination(debug_dest); }; # Other source simple_src { file("/u/dang/local/var/log/myfile.log" follow_freq(1) flags(no-parse) program_override("my_log ") ); }; source test_src { file ("/u/dang/local/var/log/test*.log" follow_freq(1) flags(no-parse) program_override("test_log ") ); }; destination test_dest { file ("/u/dang/local/var/log/collected.log" perm(0644) ); }; log { source(simple_src); destination(test_dest); }; log { source(test_src); destination(test_dest); };
The behavior I am seeing is that anything appended to the static file "myfile.log" shows up in "collected.log" a second later. But new files of the pattern "test-1.log", etc. are never forwarded, and if there are existing files of this name at startup they are ignored. The internal log says:
2009-03-25T10:02:20-07:00 host.org syslog-ng[29624]: Follow-mode file source not found, deferring open; filename='/u/dang/local/var/log/ test*.log' 2009-03-25T10:02:20-07:00 host.org syslog-ng[29626]: syslog-ng starting up; version='3.0.1'
My syslog-ng version is:
-bash-3.1$ syslog-ng -V syslog-ng 3.0.1 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng- ose--mainline--3.0#master#93a342dae3a2b0cb15811d0c34ea7f58b3fba14e Compile-Date: Mar 25 2009 09:08:54 Enable-Threads: off Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-Sun-Door: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: on Enable-SSL: on Enable-SQL: off Enable-Linux-Caps: off Enable-Pcre: off
Currently the wildcard based file monitoring is only part of the Premium Edition, sorry. The Open Source edition can only read individual files, and you need to specify the full pathname, you cannot use globbing. -- Bazsi
participants (4)
-
Balazs Scheidler
-
Dan Gunter
-
ILLES, Marton
-
Ritesh Sood