[syslog-ng] Logging to db fails for multiple tables/destinations
Balazs Scheidler
bazsi at balabit.hu
Sat Feb 28 09:41:38 CET 2009
You are right. Only one of the destinations got initialized during
startup. This patch fixes it for me:
diff --git a/src/apphook.c b/src/apphook.c
index ab9cb02..6115b27 100644
--- a/src/apphook.c
+++ b/src/apphook.c
@@ -54,10 +54,10 @@ run_application_hook(gint type)
if (e->type == type)
{
+ l_next = l->next;
application_hooks = g_list_remove_link(application_hooks, l);
e->func(type, e->user_data);
g_free(e);
- l_next = l->next;
g_list_free_1(l);
}
else
On Mon, 2009-02-23 at 19:31 -0800, Liam Kirsher wrote:
> Balazs,
>
> I'm afraid this message may have gotten overlooked, and I'm hoping to
> get this issue resolved soon so I can deal with my logging issues; so
> I'm sending it again. The debugging suggesting you made turned up
> what I believe is a bug.
>
> I have done as you suggested and run syslog-ng in debugging mode, same
> syslog-ng.conf as before. It appears that the first entry line
> (root.geocode_access) matches the filter but does not trigger the SQL
> insert. However, if I reverse the order of the log{} definitions,
> then it does work and the other one doesn't! I get different results
> depending on the order of the two statements below. It looks like the
> SQL insert only happens for the log definition that is last.
>
> log {
> source(s_sys);
> filter(f_geocode);
> parser(p_geocode);
> destination(d_geocode);
> };
>
> log {
> source(s_sys);
> filter(f_ut_access);
> parser(p_ut_access);
> destination(d_ut_access);
> };
>
> Would you please take a look?
>
> Thanks,
> Liam
>
> /usr/local/sbin/syslog-ng --foreground --verbose --debug --stderr
> -p /var/run/syslogd.pid
> Running application hooks; hook='1'
> Running application hooks; hook='3'
> syslog-ng starting up; version='3.0.1'
> Database thread started;
> Incoming log entry; line='<14>obsidian: 2009-02-17
> 10:47:55,75.101.83.163,/hCi/KM35kk,root.geocode_access,INFO,san
> francisco,"San Francisco, CA, US",37.77916,-122.420049\x0a'
> Filter rule evaluation begins; filter_rule='f_filter2'
> Filter node evaluation result; filter_result='match',
> filter_type='level'
> Filter node evaluation result; filter_result='match',
> filter_type='facility'
> Filter node evaluation result; filter_result='match',
> filter_type='AND'
> Filter rule evaluation result; filter_result='match',
> filter_rule='f_filter2'
> Initializing destination file writer; template='/var/log/messages',
> filename='/var/log/messages'
> Filter rule evaluation begins; filter_rule='f_filter3'
> Filter node evaluation result; filter_result='not-match',
> filter_type='facility'
> Filter rule evaluation result; filter_result='not-match',
> filter_rule='f_filter3'
> Filter rule evaluation begins; filter_rule='f_filter4'
> Filter node evaluation result; filter_result='not-match',
> filter_type='facility'
> Filter rule evaluation result; filter_result='not-match',
> filter_rule='f_filter4'
> Filter rule evaluation begins; filter_rule='f_filter5'
> Filter node evaluation result; filter_result='not-match',
> filter_type='level'
> Filter rule evaluation result; filter_result='not-match',
> filter_rule='f_filter5'
> Filter rule evaluation begins; filter_rule='f_filter6'
> Filter node evaluation result; filter_result='not-match',
> filter_type='facility'
> Filter node evaluation result; filter_result='not-match',
> filter_type='facility'
> Filter node evaluation result; filter_result='not-match',
> filter_type='AND'
> Filter node evaluation result; filter_result='not-match',
> filter_type='OR'
> Filter rule evaluation result; filter_result='not-match',
> filter_rule='f_filter6'
> Filter rule evaluation begins; filter_rule='f_filter7'
> Filter node evaluation result; filter_result='not-match',
> filter_type='facility'
> Filter rule evaluation result; filter_result='not-match',
> filter_rule='f_filter7'
> Filter rule evaluation begins; filter_rule='f_filter8'
> Filter node evaluation result; filter_result='not-match',
> filter_type='facility'
> Filter rule evaluation result; filter_result='not-match',
> filter_rule='f_filter8'
> Filter rule evaluation begins; filter_rule='f_geocode'
> Filter node evaluation result; filter_result='match'
> Filter node evaluation result; filter_result='match',
> filter_type='level'
> Filter node evaluation result; filter_result='match',
> filter_type='AND'
> Filter node evaluation result; filter_result='match',
> filter_type='filter(f_obsidian)'
> Filter node evaluation result; filter_result='match'
> Filter node evaluation result; filter_result='match',
> filter_type='AND'
> Filter rule evaluation result; filter_result='match',
> filter_rule='f_geocode' ### Looks like a match, so SQL Insert should
> go here, right?
> Filter rule evaluation begins; filter_rule='f_ut_access'
> Filter node evaluation result; filter_result='match'
> Filter node evaluation result; filter_result='match',
> filter_type='level'
> Filter node evaluation result; filter_result='match',
> filter_type='AND'
> Filter node evaluation result; filter_result='match',
> filter_type='filter(f_obsidian)'
> Filter node evaluation result; filter_result='not-match'
> Filter node evaluation result; filter_result='not-match',
> filter_type='AND'
> Filter rule evaluation result; filter_result='not-match',
> filter_rule='f_ut_access'
> Incoming log entry; line='<14>obsidian: 2009-02-17
> 10:47:55,75.101.83.163,/hCi/KM35kk,root.ut_access,INFO,,,,,/v1/?loc=san+francisco&start=0&rows=10&f=html,,,37.77916,-122.420049\x0a'
> Filter rule evaluation begins; filter_rule='f_filter2'
> Filter node evaluation result; filter_result='match',
> filter_type='level'
> Filter node evaluation result; filter_result='match',
> filter_type='facility'
> Filter node evaluation result; filter_result='match',
> filter_type='AND'
> Filter rule evaluation result; filter_result='match',
> filter_rule='f_filter2'
> Filter rule evaluation begins; filter_rule='f_filter3'
> Filter node evaluation result; filter_result='not-match',
> filter_type='facility'
> Filter rule evaluation result; filter_result='not-match',
> filter_rule='f_filter3'
> Filter rule evaluation begins; filter_rule='f_filter4'
> Filter node evaluation result; filter_result='not-match',
> filter_type='facility'
> Filter rule evaluation result; filter_result='not-match',
> filter_rule='f_filter4'
> Filter rule evaluation begins; filter_rule='f_filter5'
> Filter node evaluation result; filter_result='not-match',
> filter_type='level'
> Filter rule evaluation result; filter_result='not-match',
> filter_rule='f_filter5'
> Filter rule evaluation begins; filter_rule='f_filter6'
> Filter node evaluation result; filter_result='not-match',
> filter_type='facility'
> Filter node evaluation result; filter_result='not-match',
> filter_type='facility'
> Filter node evaluation result; filter_result='not-match',
> filter_type='AND'
> Filter node evaluation result; filter_result='not-match',
> filter_type='OR'
> Filter rule evaluation result; filter_result='not-match',
> filter_rule='f_filter6'
> Filter rule evaluation begins; filter_rule='f_filter7'
> Filter node evaluation result; filter_result='not-match',
> filter_type='facility'
> Filter rule evaluation result; filter_result='not-match',
> filter_rule='f_filter7'
> Filter rule evaluation begins; filter_rule='f_filter8'
> Filter node evaluation result; filter_result='not-match',
> filter_type='facility'
> Filter rule evaluation result; filter_result='not-match',
> filter_rule='f_filter8'
> Filter rule evaluation begins; filter_rule='f_geocode'
> Filter node evaluation result; filter_result='match'
> Filter node evaluation result; filter_result='match',
> filter_type='level'
> Filter node evaluation result; filter_result='match',
> filter_type='AND'
> Filter node evaluation result; filter_result='match',
> filter_type='filter(f_obsidian)'
> Filter node evaluation result; filter_result='not-match'
> Filter node evaluation result; filter_result='not-match',
> filter_type='AND'
> Filter rule evaluation result; filter_result='not-match',
> filter_rule='f_geocode'
> Filter rule evaluation begins; filter_rule='f_ut_access'
> Filter node evaluation result; filter_result='match'
> Filter node evaluation result; filter_result='match',
> filter_type='level'
> Filter node evaluation result; filter_result='match',
> filter_type='AND'
> Filter node evaluation result; filter_result='match',
> filter_type='filter(f_obsidian)'
> Filter node evaluation result; filter_result='match'
> Filter node evaluation result; filter_result='match',
> filter_type='AND'
> Filter rule evaluation result; filter_result='match',
> filter_rule='f_ut_access'
> Running SQL query; query='SELECT * FROM ut_access_log WHERE 0=1'
> Running SQL query; query='INSERT INTO ut_access_log (datetime,
> query_time, host, program, pid, request_id, level, ip, phone_id,
> phone_type, software_version, client_version, query_string, art_id,
> session_id, lat, lng) VALUES (\'2009-02-17T13:47:55-05:00\',
> \'2009-02-17 10:47:55\', \'127.0.0.1\', \'obsidian\', \'\',
> \'/hCi/KM35kk\', \'info\', \'75.101.83.163\', \'\', \'\', \'\', \'\',
> \'/v1/?loc=san+francisco&start=0&rows=10&f=html\', \'\', \'\',
> \'37.77916\', \'-122.420049\')'
>
>
>
>
>
>
>
> Balazs Scheidler wrote:
> > On Fri, 2009-02-13 at 12:25 -0800, Liam Kirsher wrote:
> >
> > > Hi --
> > >
> > > I am /almost/ there, logging to Postgres database. However, I've
> > > discovered a puzzling and problematic behavior.This is probably just
> > > some simple misunderstanding on my part, since this is my first foray
> > > into syslog-ng.
> > > I am logging to two different db tables. Which table I log to is
> > > determined by a regexp filter. The value is either root.ut_access or
> > > root.geocode.
> > > I can get either one to work, but not both at the same time.
> > > If I comment out the log entry for the geocode, then ut_access works.
> > > However, if both log entries exist, only the gecocode_access_log table
> > > gets a new row. Nothing is logged to the ut_access_log table! (Both
> > > messages are logged to d_obsidian destination file, however.)
> > > I've attached my config file.
> > >
> >
> > Hmm.. could you post two example messages that should go to one or the
> > other destination?
> >
> > Since you didn't specify flags(final) to either log statements, both
> > should be doing their job, independently from the other. The only thing
> > that should control whether one or the other destination is used is the
> > attached filter. You can get filter debugging by enabling the --debug /
> > --verbose options.
> >
> > Be sure that you run syslog-ng in the foreground if you specify these as
> > these easily generate loops in the configuration unless the internal()
> > source is not present. (use --foreground for that, intenral() messages
> > will be printed on the standard error).
> >
> > Judging the config I can't see an obvious problem, that's why I wanted
> > to test it, but I'd need a sample log message for that.
> >
> >
>
> --
> Liam Kirsher
> PGP: http://liam.numenet.com/pgp/
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
--
Bazsi
More information about the syslog-ng
mailing list