- syslog-ng - lists.balabit.hu

[Bug 163] New: afmongo does not send log
by bugzilla＠bugzilla.balabit.com 20 Apr '12

20 Apr '12

https://bugzilla.balabit.com/show_bug.cgi?id=163 Summary: afmongo does not send log Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: critical Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: whille(a)163.com Type of the Report: bug Estimated Hours: 0.0 i'm using syslog-ng to store mongo, about 2k logs/sec. About half an hour, no message sent out. trying to locate it, I set log_fifo_size(16), it's easy to reproduce the bug. i found codes related: afmongodb_dd_queue(LogPipe *s, LogMessage *msg, const LogPathOptions *path_options, gpointer user_data) { ... g_mutex_lock(self->queue_mutex); self->last_msg_stamp = cached_g_current_time_sec (); queue_was_empty = log_queue_get_length(self->queue) == 0; g_mutex_unlock(self->queue_mutex); log_queue_push_tail(self->queue, msg, path_options); g_mutex_lock(self->suspend_mutex); if (queue_was_empty && !self->writer_thread_suspended) { g_mutex_lock(self->queue_mutex); log_queue_set_parallel_push(self->queue, 1, afmongodb_dd_queue_notify, self, NULL); g_mutex_unlock(self->queue_mutex); } g_mutex_unlock(self->suspend_mutex); } since in afmongodb_worker_thread(){ ... g_mutex_unlock(self->suspend_mutex); g_mutex_lock(self->queue_mutex); if (log_queue_get_length(self->queue) == 0) { g_cond_wait(self->writer_thread_wakeup_cond, self->queue_mutex); } g_mutex_unlock(self->queue_mutex); ... } the whole rough process i think is, worker wait fo afmongodb_dd_queue_notify(), if 0 queue. But in multi-thread race condition, queue_was_empty offend simplicity. Here's a situation: afmongodb_dd_queue() afmongodb_dd_queue() afmongodb_dd_queue() { // worker thread afmongodb_worker_thread{ whille(){... // run in a while circle afmongodb_worker_thread{ whille(){... afmongodb_worker_thread{ whille(){... all queue are empty now, g_cond_wait(self->writer_thread_wakeup_cond, self->queue_mutex); log_queue_push_tail queue_was_empty is FALSE, so afmongodb_dd_queue_notify is not called. afmongodb_dd_queue() // a queue has been put to tail last time, so queue_was_empty will be still FALSE. i'm trying to get queue length later and judge it as <=1, instead of ==0, for a queue just put to tail. wonder if it work, though a bit ugly. afmongodb_dd_queue(){ ... if(!self->writer_thread_suspended){ g_mutex_lock(self->queue_mutex); if (log_queue_get_length(self->queue)<=1){ log_queue_set_parallel_push(self->queue, 1, afmongodb_dd_queue_notify, self, NULL); } g_mutex_unlock(self->queue_mutex); } -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 10

[Bug 174] New: It's unable to analyze entire string by match filter
by bugzilla＠bugzilla.balabit.com 19 Apr '12

19 Apr '12

https://bugzilla.balabit.com/show_bug.cgi?id=174 Summary: It's unable to analyze entire string by match filter Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: olegon(a)olegon.ru Type of the Report: --- Estimated Hours: 0.0 In my Gentoo Linux (compiled syslog-ng by source), I can't filter entire string, coming from external source. My Cisco sends string "Session terminated .....", so I can't match it by match("Session terminated","MESSAGE") nor match("Session terminated","MSGHDR"), because string splitted to PROGRAM and MESSAGE. Old behaviour allowed me to analyze entire string by "MESSAGE", and "MSGONLY" allowed to analyze only message part of string, but nowadays all of this macro points to only message part of string. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

main syslog-ng server needs to relay some logs
by Jerome Yanga 18 Apr '12

18 Apr '12

Hi All! Please be nice as I am still a newbee. I was able to create a syslog-ng server that recieves syslogs from a number of servers. However, i would need to relay some logs from specific hosts to another syslog-ng server to make it available to others. Can someone help me with a sample config? Thank you in advance. Regards, j

2 2

How to rotate log file in syslog-ng file based on file size
by Balbir Singh 17 Apr '12

17 Apr '12

Dear Team, I am receiving a log of around 80GB per day. I want to make a log file of fixed size (suppose 10Gb) and after that file size crossed, new file should be created. How to do that? My present configuration is : ------------------------------------ @version:3.2 options { # sync(0); # stats (0); flush_lines(0); stats_freq(0); ts_format(iso); keep_hostname(yes); log_fifo_size(1024); }; source s_sys { file ("/proc/kmsg" program_override("kernel: ")); unix-stream ("/dev/log"); internal(); udp(ip(0.0.0.0) port(514)); }; destination d_cons { file("/dev/console"); }; #destination d_mesg { file("/home1/log/syslog-ng.log"); }; destination d_mesg { file("/home1/asa-log/syslog-ng/$HOST/$YEAR/$MONTH/$DAY/$FACILITY-$HOST-$YEAR-$MONTH-$DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes) ); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; -- ---- With Regards Balbir Singh Hora Scientist - 'C', Operation and Routing Group M/O of Communication and IT, NIC A-BLOCK, CGO Complex, New Delhi.

2 1

Ubuntu upgrade - syslog-ng fails with undefined symbol: evt_tag_long'
by Clayton Dukes 13 Apr '12

13 Apr '12

Hi Folks, Just upgraded one of our older Ubuntu boxes tonight to Ubuntu 11.10 and now I get the following error when trying to start syslog-ng: Error opening plugin module; module='dbparser', error='/usr/lib/syslog-ng/libdbparser.so: Anyone know what that means and how I can fix it? ______________________________________________________________ Clayton Dukes ______________________________________________________________

3 4

what's the default template of file destination?
by Fu Ecy 13 Apr '12

13 Apr '12

Dear guys: I use syslog-ng to gather the error log of my system, but now I have a problem. The original log message is like this. *[2012-04-10 **18:51:30**] [BlenderProcessor.cpp:452:allocPhase1Request()] [WARN] :: BLENDER: queryparser doRewrite function error* I set the template like this. *file( "/home/admin/logs/blender.log.$S_YEAR-$S_MONTH-$S_DAY" create_dirs(yes) owner(admin) group(admin) perm(0755) template("$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC\t$FULLHOST\t$MESSAGE\n") );* Then I get the log like this. *2012-04-13 14:41:36 s061090.cm5 18:51:30] [BlenderProcessor.cpp:452:allocPhase1Request()] [WARN] :: BLENDER: queryparser doRewrite function error* The date of original log message has been truncated. If I don't use template, it will be ok. So I wonder what is the default template. Thank you very much :)

2 2

Syslog-ng apache filters
by Hugo Deprez 13 Apr '12

13 Apr '12

Dear community, I am currently trying to setup a central syslog-ng server. Almost everything is working, I just have some issue with filtering apache2 logs. On the remote host I use the following to send apache2 : source vhost_access { file("/var/log/apache2/vhost-access.log"); }; This is working the central syslog server get the messages. I setup a filter on the server to filter apache2 log such as : filter f_apache { match("hostname.mydomain.fr"); }; this works. The issue is that my apache2 logs match other filter, so I am logging messages 3 times : filter f_messages { level(info,notice,warn) and not facility(auth,authpriv,cron,daemon,mail,news); }; and : filter f_user { facility(user) and not filter(f_debug); }; It seems that I have to play with default-priority() and default-facility() parameters. I would like to know how do you deal with this kind of situation ? Regards, Hugo

2 8

PROBLEM: Stats stop getting generating after nightly reload in logrotation script
by Paul Braxton 12 Apr '12

12 Apr '12

Version: syslog-ng-premium-edition-4.1.3-1.rhel5 Operating system: CentOS release 6.2 (Final) We are experiencing an issue where the statistics messages stop getting generated after the nightly reload done by logrotation. This happens pretty consistently after the logrotation runs, although sometimes upward of an hour later. This is the command in the log rotation script: /etc/init.d/syslog-ng reload 2>/dev/null We use the stats to track metrics and additional health monitoring. We are concerned that logs also may be dropped when these stats stop getting reported. A full restart of syslog-ng fixes the issue. Any info would be helpful, thanks!!! -Paul

2 1

paulbraxton3@hotmail.com
by Paul Braxton 12 Apr '12

12 Apr '12

paulbraxton3(a)hotmail.com

1 0

How to remove hosts from Syslog-ng plugin
by Wagner Pereira 07 Apr '12

07 Apr '12

Hello, all. I'm not sure if that's the right mailing list. If it is not, please let me know. I use the CactiEZ solution running Syslog 0.5.2 by Jimmy Conner The thing is: every host configured to log against my server (I asked a friend of mine to test it out), my server accept and include the new host in the list. I need to do two things: 1. To block new unauthorized hosts to log on it and 2. Remove hosts from the list I appreciate any help. Best wishes... Wagner Pereira PoP-SP/RNP - Ponto de Presença da RNP em São Paulo http://www.pop-sp.rnp.br Fixo. +55 11 3091-9799 Móvel. +55 11 9923-9554 fone@RNP. 011-1015-8902 ### At PoP-SP/RNP, every day is an IPv6 day ###

2 1