[syslog-ng] [PATCH] correlation: explicit context timeout action

[Balint Kovacs]

Hi,

I'm trying to use patterndb correlation to find matching firewall 
connection startup and ending log messages and emit a consolidated 
message. I found that with high-volume load, the memory usage of 
syslog-ng climbs rapidly and it would be beneficial to be able to kick 
already ended events from the memory. Please find a patch below (against 
3.4) that does this, I thought the easy place for the user would be the 
<action> part of the ending rule, this way it can even be specified when 
the context should be ended.

Balint

From: Balint Kovacs <blint at balabit.hu>
Date: Wed, 21 Sep 2011 11:46:30 +0200
Subject: [PATCH] correlation: add action to explicitly end context

If there is a message that can be associated with the end of a context,
the below action can explicitly end it, reducing the memory footprint.

Example:
<action>
<end-context />
</action>

Signed-off-by: Balint Kovacs <blint at balabit.hu>
---
  modules/dbparser/patterndb-int.h |    3 ++-
  modules/dbparser/patterndb.c     |   22 ++++++++++++++++++++++
  2 files changed, 24 insertions(+), 1 deletions(-)

diff --git a/modules/dbparser/patterndb-int.h 
b/modules/dbparser/patterndb-int.h
index 0434847..eff77e6 100644
--- a/modules/dbparser/patterndb-int.h
+++ b/modules/dbparser/patterndb-int.h
@@ -115,7 +115,8 @@ enum
  enum
  {
    RAC_NONE,
-  RAC_MESSAGE
+  RAC_MESSAGE,
+  RAC_EXPIRE
  };

  /* a rule may contain one or more actions to be performed */
diff --git a/modules/dbparser/patterndb.c b/modules/dbparser/patterndb.c
index 343ef6d..72e97e0 100644
--- a/modules/dbparser/patterndb.c
+++ b/modules/dbparser/patterndb.c
@@ -577,6 +577,18 @@ pdb_rule_run_actions(PDBRule *self, gint trigger, 
PatternDB *db, PDBContext *con
                    emit(genmsg, TRUE, emit_data);
                    log_msg_unref(genmsg);
                    break;
+                case RAC_EXPIRE:
+                  if (context)
+                    {
+                      msg_debug("Expiring patterndb correllation 
context as directed by action",
+                                 evt_tag_str("last_rule", 
context->rule->rule_id),
+                                 
evt_tag_long("remaining_context_count", 
g_hash_table_size(context->db->state)),
+                                 NULL);
+                      g_hash_table_remove(context->db->state, 
&context->key);
+                      timer_wheel_del_timer(db->timer_wheel, 
context->timer);
+                      pdb_context_unref(context);
+                    }
+                  break;
                  default:
                    g_assert_not_reached();
                    break;
@@ -921,6 +933,16 @@ pdb_loader_start_element(GMarkupParseContext 
*context, const gchar *element_name
        state->current_action->content_type = RAC_MESSAGE;
        state->current_message = &state->current_action->content.message;
      }
+  else if (strcmp(element_name, "end-context") == 0)
+    {
+      if (!state->in_action)
+        {
+          *error = g_error_new(1, 0, "Unexpected <end-context> element, 
it must be inside an action");
+          return;
+        }
+      state->current_action->content_type = RAC_EXPIRE;
+      state->current_message = &state->current_action->content.message;
+    }
  }

  void
-- 
1.7.0.4