Hi, I'm trying to use patterndb correlation to find matching firewall connection startup and ending log messages and emit a consolidated message. I found that with high-volume load, the memory usage of syslog-ng climbs rapidly and it would be beneficial to be able to kick already ended events from the memory. Please find a patch below (against 3.4) that does this, I thought the easy place for the user would be the <action> part of the ending rule, this way it can even be specified when the context should be ended. Balint From: Balint Kovacs <blint@balabit.hu> Date: Wed, 21 Sep 2011 11:46:30 +0200 Subject: [PATCH] correlation: add action to explicitly end context If there is a message that can be associated with the end of a context, the below action can explicitly end it, reducing the memory footprint. Example: <action> <end-context /> </action> Signed-off-by: Balint Kovacs <blint@balabit.hu> --- modules/dbparser/patterndb-int.h | 3 ++- modules/dbparser/patterndb.c | 22 ++++++++++++++++++++++ 2 files changed, 24 insertions(+), 1 deletions(-) diff --git a/modules/dbparser/patterndb-int.h b/modules/dbparser/patterndb-int.h index 0434847..eff77e6 100644 --- a/modules/dbparser/patterndb-int.h +++ b/modules/dbparser/patterndb-int.h @@ -115,7 +115,8 @@ enum enum { RAC_NONE, - RAC_MESSAGE + RAC_MESSAGE, + RAC_EXPIRE }; /* a rule may contain one or more actions to be performed */ diff --git a/modules/dbparser/patterndb.c b/modules/dbparser/patterndb.c index 343ef6d..72e97e0 100644 --- a/modules/dbparser/patterndb.c +++ b/modules/dbparser/patterndb.c @@ -577,6 +577,18 @@ pdb_rule_run_actions(PDBRule *self, gint trigger, PatternDB *db, PDBContext *con emit(genmsg, TRUE, emit_data); log_msg_unref(genmsg); break; + case RAC_EXPIRE: + if (context) + { + msg_debug("Expiring patterndb correllation context as directed by action", + evt_tag_str("last_rule", context->rule->rule_id), + evt_tag_long("remaining_context_count", g_hash_table_size(context->db->state)), + NULL); + g_hash_table_remove(context->db->state, &context->key); + timer_wheel_del_timer(db->timer_wheel, context->timer); + pdb_context_unref(context); + } + break; default: g_assert_not_reached(); break; @@ -921,6 +933,16 @@ pdb_loader_start_element(GMarkupParseContext *context, const gchar *element_name state->current_action->content_type = RAC_MESSAGE; state->current_message = &state->current_action->content.message; } + else if (strcmp(element_name, "end-context") == 0) + { + if (!state->in_action) + { + *error = g_error_new(1, 0, "Unexpected <end-context> element, it must be inside an action"); + return; + } + state->current_action->content_type = RAC_EXPIRE; + state->current_message = &state->current_action->content.message; + } } void -- 1.7.0.4