April 2012 - syslog-ng - lists.balabit.hu

afsql error
by Jason Kojro-Badziak 23 Apr '12

23 Apr '12

Hello all: I am running syslog-ng on a Centos 6 server. I installed syslog-ng via yum, and while it is running successfully, I am getting this error every time I start or restart syslog-ng. # /etc/init.d/syslog-ng start Plugin module not found in 'module-path'; module-path='/lib64/syslog-ng', module='afsql' Starting syslog-ng: Plugin module not found in 'module-path'; module-path='/lib64/syslog-ng', module='afsql' [ OK ] Is there something I can do to fix this so I stop getting the error? Thank you! Jason Kojro-Badziak Monolith Software Staff Engineer 311 North 2nd Street, Suite #302 St. Charles, IL 60174 Office: 312-957-6470 x3010 Email: jbadziak(a)monolith-software.com<mailto:jbadziak@monolith-software.com/jb.022411>

3 5

[Bug 156] New: rewrite rule not working on Linux
by bugzilla＠bugzilla.balabit.com 20 Apr '12

20 Apr '12

https://bugzilla.balabit.com/show_bug.cgi?id=156 Summary: rewrite rule not working on Linux Product: syslog-ng Version: 3.3.x Platform: Other OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: ne24(a)georgetown.edu Type of the Report: bug Estimated Hours: 0.0 Hi, I have syslog-ng 3.3.3 running Red Hat Enterprise Linux Server release 6.2 (Santiago). I compiled with the following flags. ./configure --prefix=/usr/local --enable-spoof-source --enable-pcre --enable-ssl \ --enable-debug I have the following rewrite rule: rewrite n_dst_router { subst("^(%(?:ASA|PIX|FWSM)\-\d\-\d{6}):", "", value("MESSAGE"), type("pcre"), flags("global")); }; But it is not working. The line below minus the IP addresses shows that it is not replacing the %FWSM-6-305011 with a blank. Jan 7 15:59:06 HOSTNAME-REMOVED %FWSM-6-305011: Built dynamic tcp translation from GRT:IP-ADDRESS-REMOVED/53328 to BACKBONE-VRF:IP-ADDRESS-REMOVED/29361 I will also attach my whole syslog-ng.conf file so that you would see the whole setup. Here is the output of the debug output and as you can see, it looks like it is doing what it is supposed to, but it does not write it to the file with the change. Incoming log entry; line='<163>%FWSM-3-710003: udp access denied by ACL from IP-ADDRESS-REMOVED/35390 to BACKBONE-VRF:IP-ADDRESS-REMOVED/46897\x0a' Rewrite expression evaluation result; value='MESSAGE', new_value='udp access denied by ACL from IP-ADDRESS-REMOVED/35390 to BACKBONE-VRF:IP-ADDRESS-REMOVED/46897' I searched the bug reports but could not find anything related to this issue. I would appreciate it your help on this. Thank you Nadim El-Khoury -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 163] New: afmongo does not send log
by bugzilla＠bugzilla.balabit.com 20 Apr '12

20 Apr '12

https://bugzilla.balabit.com/show_bug.cgi?id=163 Summary: afmongo does not send log Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: critical Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: whille(a)163.com Type of the Report: bug Estimated Hours: 0.0 i'm using syslog-ng to store mongo, about 2k logs/sec. About half an hour, no message sent out. trying to locate it, I set log_fifo_size(16), it's easy to reproduce the bug. i found codes related: afmongodb_dd_queue(LogPipe *s, LogMessage *msg, const LogPathOptions *path_options, gpointer user_data) { ... g_mutex_lock(self->queue_mutex); self->last_msg_stamp = cached_g_current_time_sec (); queue_was_empty = log_queue_get_length(self->queue) == 0; g_mutex_unlock(self->queue_mutex); log_queue_push_tail(self->queue, msg, path_options); g_mutex_lock(self->suspend_mutex); if (queue_was_empty && !self->writer_thread_suspended) { g_mutex_lock(self->queue_mutex); log_queue_set_parallel_push(self->queue, 1, afmongodb_dd_queue_notify, self, NULL); g_mutex_unlock(self->queue_mutex); } g_mutex_unlock(self->suspend_mutex); } since in afmongodb_worker_thread(){ ... g_mutex_unlock(self->suspend_mutex); g_mutex_lock(self->queue_mutex); if (log_queue_get_length(self->queue) == 0) { g_cond_wait(self->writer_thread_wakeup_cond, self->queue_mutex); } g_mutex_unlock(self->queue_mutex); ... } the whole rough process i think is, worker wait fo afmongodb_dd_queue_notify(), if 0 queue. But in multi-thread race condition, queue_was_empty offend simplicity. Here's a situation: afmongodb_dd_queue() afmongodb_dd_queue() afmongodb_dd_queue() { // worker thread afmongodb_worker_thread{ whille(){... // run in a while circle afmongodb_worker_thread{ whille(){... afmongodb_worker_thread{ whille(){... all queue are empty now, g_cond_wait(self->writer_thread_wakeup_cond, self->queue_mutex); log_queue_push_tail queue_was_empty is FALSE, so afmongodb_dd_queue_notify is not called. afmongodb_dd_queue() // a queue has been put to tail last time, so queue_was_empty will be still FALSE. i'm trying to get queue length later and judge it as <=1, instead of ==0, for a queue just put to tail. wonder if it work, though a bit ugly. afmongodb_dd_queue(){ ... if(!self->writer_thread_suspended){ g_mutex_lock(self->queue_mutex); if (log_queue_get_length(self->queue)<=1){ log_queue_set_parallel_push(self->queue, 1, afmongodb_dd_queue_notify, self, NULL); } g_mutex_unlock(self->queue_mutex); } -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 10

[Bug 174] New: It's unable to analyze entire string by match filter
by bugzilla＠bugzilla.balabit.com 19 Apr '12

19 Apr '12

https://bugzilla.balabit.com/show_bug.cgi?id=174 Summary: It's unable to analyze entire string by match filter Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: olegon(a)olegon.ru Type of the Report: --- Estimated Hours: 0.0 In my Gentoo Linux (compiled syslog-ng by source), I can't filter entire string, coming from external source. My Cisco sends string "Session terminated .....", so I can't match it by match("Session terminated","MESSAGE") nor match("Session terminated","MSGHDR"), because string splitted to PROGRAM and MESSAGE. Old behaviour allowed me to analyze entire string by "MESSAGE", and "MSGONLY" allowed to analyze only message part of string, but nowadays all of this macro points to only message part of string. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

main syslog-ng server needs to relay some logs
by Jerome Yanga 18 Apr '12

18 Apr '12

Hi All! Please be nice as I am still a newbee. I was able to create a syslog-ng server that recieves syslogs from a number of servers. However, i would need to relay some logs from specific hosts to another syslog-ng server to make it available to others. Can someone help me with a sample config? Thank you in advance. Regards, j

2 2

How to rotate log file in syslog-ng file based on file size
by Balbir Singh 17 Apr '12

17 Apr '12

Dear Team, I am receiving a log of around 80GB per day. I want to make a log file of fixed size (suppose 10Gb) and after that file size crossed, new file should be created. How to do that? My present configuration is : ------------------------------------ @version:3.2 options { # sync(0); # stats (0); flush_lines(0); stats_freq(0); ts_format(iso); keep_hostname(yes); log_fifo_size(1024); }; source s_sys { file ("/proc/kmsg" program_override("kernel: ")); unix-stream ("/dev/log"); internal(); udp(ip(0.0.0.0) port(514)); }; destination d_cons { file("/dev/console"); }; #destination d_mesg { file("/home1/log/syslog-ng.log"); }; destination d_mesg { file("/home1/asa-log/syslog-ng/$HOST/$YEAR/$MONTH/$DAY/$FACILITY-$HOST-$YEAR-$MONTH-$DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes) ); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; -- ---- With Regards Balbir Singh Hora Scientist - 'C', Operation and Routing Group M/O of Communication and IT, NIC A-BLOCK, CGO Complex, New Delhi.

2 1

Ubuntu upgrade - syslog-ng fails with undefined symbol: evt_tag_long'
by Clayton Dukes 13 Apr '12

13 Apr '12

Hi Folks, Just upgraded one of our older Ubuntu boxes tonight to Ubuntu 11.10 and now I get the following error when trying to start syslog-ng: Error opening plugin module; module='dbparser', error='/usr/lib/syslog-ng/libdbparser.so: Anyone know what that means and how I can fix it? ______________________________________________________________ Clayton Dukes ______________________________________________________________

3 4

what's the default template of file destination?
by Fu Ecy 13 Apr '12

13 Apr '12

Dear guys: I use syslog-ng to gather the error log of my system, but now I have a problem. The original log message is like this. *[2012-04-10 **18:51:30**] [BlenderProcessor.cpp:452:allocPhase1Request()] [WARN] :: BLENDER: queryparser doRewrite function error* I set the template like this. *file( "/home/admin/logs/blender.log.$S_YEAR-$S_MONTH-$S_DAY" create_dirs(yes) owner(admin) group(admin) perm(0755) template("$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC\t$FULLHOST\t$MESSAGE\n") );* Then I get the log like this. *2012-04-13 14:41:36 s061090.cm5 18:51:30] [BlenderProcessor.cpp:452:allocPhase1Request()] [WARN] :: BLENDER: queryparser doRewrite function error* The date of original log message has been truncated. If I don't use template, it will be ok. So I wonder what is the default template. Thank you very much :)

2 2

Syslog-ng apache filters
by Hugo Deprez 13 Apr '12

13 Apr '12

Dear community, I am currently trying to setup a central syslog-ng server. Almost everything is working, I just have some issue with filtering apache2 logs. On the remote host I use the following to send apache2 : source vhost_access { file("/var/log/apache2/vhost-access.log"); }; This is working the central syslog server get the messages. I setup a filter on the server to filter apache2 log such as : filter f_apache { match("hostname.mydomain.fr"); }; this works. The issue is that my apache2 logs match other filter, so I am logging messages 3 times : filter f_messages { level(info,notice,warn) and not facility(auth,authpriv,cron,daemon,mail,news); }; and : filter f_user { facility(user) and not filter(f_debug); }; It seems that I have to play with default-priority() and default-facility() parameters. I would like to know how do you deal with this kind of situation ? Regards, Hugo

2 8

PROBLEM: Stats stop getting generating after nightly reload in logrotation script
by Paul Braxton 12 Apr '12

12 Apr '12

Version: syslog-ng-premium-edition-4.1.3-1.rhel5 Operating system: CentOS release 6.2 (Final) We are experiencing an issue where the statistics messages stop getting generated after the nightly reload done by logrotation. This happens pretty consistently after the logrotation runs, although sometimes upward of an hour later. This is the command in the log rotation script: /etc/init.d/syslog-ng reload 2>/dev/null We use the stats to track metrics and additional health monitoring. We are concerned that logs also may be dropped when these stats stop getting reported. A full restart of syslog-ng fixes the issue. Any info would be helpful, thanks!!! -Paul

2 1