[PATCH] basicfuncs: Implement a few numeric template functions.
From: Gergely Nagy <algernon@madhouse-project.org> Implement addition, substraction, multiplication, division and modulus template functions: $(+ N M), $(- N M), $(* N M), $(/ N M) and $(% N M), respectively. All of them take two numeric arguments, and log an error, if they receive less or more, or in case the number cannot be fully parsed as a number. Signed-off-by: Gergely Nagy <algernon@madhouse-project.org> --- modules/basicfuncs/basic-funcs.c | 106 ++++++++++++++++++++++++++++++++++++- 1 files changed, 103 insertions(+), 3 deletions(-) diff --git a/modules/basicfuncs/basic-funcs.c b/modules/basicfuncs/basic-funcs.c index a03e221..2be4584 100644 --- a/modules/basicfuncs/basic-funcs.c +++ b/modules/basicfuncs/basic-funcs.c @@ -23,7 +23,7 @@ tf_echo(LogMessage *msg, gint argc, GString *argv[], GString *result) TEMPLATE_FUNCTION_SIMPLE(tf_echo); static gboolean -tf_substr_parse_int(const gchar *s, long *d) +tf_parse_int(const gchar *s, long *d) { gchar *endptr; glong val; @@ -42,6 +42,101 @@ tf_substr_parse_int(const gchar *s, long *d) return TRUE; } +static gboolean +tf_num_parse(gint argc, GString *argv[], + const gchar *func_name, glong *n, glong *m) +{ + if (argc != 2) + { + msg_error("Template function requires two arguments.", + evt_tag_str("function", func_name), NULL); + return FALSE; + } + + if (!tf_parse_int(argv[0]->str, n)) + { + msg_error("Parsing failed, template function's first argument is not a number", + evt_tag_str("function", func_name), + evt_tag_str("arg1", argv[0]->str), NULL); + return FALSE; + } + + if (!tf_parse_int(argv[1]->str, m)) + { + msg_error("Parsing failed, template function's first argument is not a number", + evt_tag_str("function", func_name), + evt_tag_str("arg1", argv[1]->str), NULL); + return FALSE; + } + + return TRUE; +} + +static void +tf_num_plus(LogMessage *msg, gint argc, GString *argv[], GString *result) +{ + glong n, m; + + if (!tf_num_parse(argc, argv, "+", &n, &m)) + return; + + g_string_append_printf(result, "%li", n + m); +} + +TEMPLATE_FUNCTION_SIMPLE(tf_num_plus); + +static void +tf_num_minus(LogMessage *msg, gint argc, GString *argv[], GString *result) +{ + glong n, m; + + if (!tf_num_parse(argc, argv, "-", &n, &m)) + return; + + g_string_append_printf(result, "%li", n - m); +} + +TEMPLATE_FUNCTION_SIMPLE(tf_num_minus); + +static void +tf_num_multi(LogMessage *msg, gint argc, GString *argv[], GString *result) +{ + glong n, m; + + if (!tf_num_parse(argc, argv, "*", &n, &m)) + return; + + g_string_append_printf(result, "%li", n * m); +} + +TEMPLATE_FUNCTION_SIMPLE(tf_num_multi); + +static void +tf_num_div(LogMessage *msg, gint argc, GString *argv[], GString *result) +{ + glong n, m; + + if (!tf_num_parse(argc, argv, "/", &n, &m)) + return; + + g_string_append_printf(result, "%li", (glong)(n / m)); +} + +TEMPLATE_FUNCTION_SIMPLE(tf_num_div); + +static void +tf_num_mod(LogMessage *msg, gint argc, GString *argv[], GString *result) +{ + glong n, m; + + if (!tf_num_parse(argc, argv, "%", &n, &m)) + return; + + g_string_append_printf(result, "%li", n % m); +} + +TEMPLATE_FUNCTION_SIMPLE(tf_num_mod); + static void tf_substr(LogMessage *msg, gint argc, GString *argv[], GString *result) { @@ -64,14 +159,14 @@ tf_substr(LogMessage *msg, gint argc, GString *argv[], GString *result) return; /* get offset position from second argument */ - if (!tf_substr_parse_int(argv[1]->str, &start)) { + if (!tf_parse_int(argv[1]->str, &start)) { msg_error("$(substr) parsing failed, start could not be parsed", evt_tag_str("start", argv[1]->str), NULL); return; } /* if we were called with >2 arguments, third was desired length */ if (argc > 2) { - if (!tf_substr_parse_int(argv[2]->str, &len)) { + if (!tf_parse_int(argv[2]->str, &len)) { msg_error("$(substr) parsing failed, length could not be parsed", evt_tag_str("length", argv[2]->str), NULL); return; } @@ -282,6 +377,11 @@ static Plugin basicfuncs_plugins[] = TEMPLATE_FUNCTION_PLUGIN(tf_grep, "grep"), TEMPLATE_FUNCTION_PLUGIN(tf_if, "if"), TEMPLATE_FUNCTION_PLUGIN(tf_substr, "substr"), + TEMPLATE_FUNCTION_PLUGIN(tf_num_plus, "+"), + TEMPLATE_FUNCTION_PLUGIN(tf_num_minus, "-"), + TEMPLATE_FUNCTION_PLUGIN(tf_num_multi, "*"), + TEMPLATE_FUNCTION_PLUGIN(tf_num_div, "/"), + TEMPLATE_FUNCTION_PLUGIN(tf_num_mod, "%"), }; gboolean -- 1.7.2.5
Gergely Nagy <algernon@balabit.hu> writes:
From: Gergely Nagy <algernon@madhouse-project.org>
Implement addition, substraction, multiplication, division and modulus template functions: $(+ N M), $(- N M), $(* N M), $(/ N M) and $(% N M), respectively.
This is against 3.4, but should apply to 3.3 aswell, provided the $(substr) patch sent earlier is applied too, because this patch reuses the tf_substr_parse_int() introduced there. I chose short function names because, well... they're short, and for numbers, they made sense. Could've used $(add-int) and similar, but that seemed unnatural to me. As for the reasoning behind these functions - apart from getting the template function sub-language closer to Turing completeness - is to be able to fix priority numbers coming from sources that are off by one, for example (or going to destinations that expect it off by one). At least, that's for $(+). The rest is just for fun, because I was there. -- |8]
Gergely Nagy <algernon@balabit.hu> writes:
From: Gergely Nagy <algernon@madhouse-project.org>
Implement addition, substraction, multiplication, division and modulus template functions: $(+ N M), $(- N M), $(* N M), $(/ N M) and $(% N M), respectively.
Aand, while reviewing this code with Viktor Juhasz, we spotted a mistake in the code below:
+static void +tf_num_div(LogMessage *msg, gint argc, GString *argv[], GString *result) +{ + glong n, m; + + if (!tf_num_parse(argc, argv, "/", &n, &m)) + return; + + g_string_append_printf(result, "%li", (glong)(n / m)); +}
If m is 0, this will crash syslog-ng. I'll repost a version of the patch that guards against this later today. $(/ $FOO 0) will emit an msg_error and do nothing. -- |8]
Implement addition, substraction, multiplication, division and modulus template functions: $(+ N M), $(- N M), $(* N M), $(/ N M) and $(% N M), respectively. All of them take two numeric arguments, and log an error, if they receive less or more, or in case the number cannot be fully parsed as a number. Signed-off-by: Gergely Nagy <algernon@balabit.hu> --- modules/basicfuncs/basic-funcs.c | 120 +++++++++++++++++++++++++++++++++++++- tests/unit/test_template.c | 9 +++ 2 files changed, 126 insertions(+), 3 deletions(-) diff --git a/modules/basicfuncs/basic-funcs.c b/modules/basicfuncs/basic-funcs.c index a03e221..2cc1442 100644 --- a/modules/basicfuncs/basic-funcs.c +++ b/modules/basicfuncs/basic-funcs.c @@ -23,7 +23,7 @@ tf_echo(LogMessage *msg, gint argc, GString *argv[], GString *result) TEMPLATE_FUNCTION_SIMPLE(tf_echo); static gboolean -tf_substr_parse_int(const gchar *s, long *d) +tf_parse_int(const gchar *s, long *d) { gchar *endptr; glong val; @@ -42,6 +42,115 @@ tf_substr_parse_int(const gchar *s, long *d) return TRUE; } +static gboolean +tf_num_parse(gint argc, GString *argv[], + const gchar *func_name, glong *n, glong *m) +{ + if (argc != 2) + { + msg_error("Template function requires two arguments.", + evt_tag_str("function", func_name), NULL); + return FALSE; + } + + if (!tf_parse_int(argv[0]->str, n)) + { + msg_error("Parsing failed, template function's first argument is not a number", + evt_tag_str("function", func_name), + evt_tag_str("arg1", argv[0]->str), NULL); + return FALSE; + } + + if (!tf_parse_int(argv[1]->str, m)) + { + msg_error("Parsing failed, template function's first argument is not a number", + evt_tag_str("function", func_name), + evt_tag_str("arg1", argv[1]->str), NULL); + return FALSE; + } + + return TRUE; +} + +static void +tf_num_plus(LogMessage *msg, gint argc, GString *argv[], GString *result) +{ + glong n, m; + + if (!tf_num_parse(argc, argv, "+", &n, &m)) + return; + + g_string_append_printf(result, "%li", n + m); +} + +TEMPLATE_FUNCTION_SIMPLE(tf_num_plus); + +static void +tf_num_minus(LogMessage *msg, gint argc, GString *argv[], GString *result) +{ + glong n, m; + + if (!tf_num_parse(argc, argv, "-", &n, &m)) + return; + + g_string_append_printf(result, "%li", n - m); +} + +TEMPLATE_FUNCTION_SIMPLE(tf_num_minus); + +static void +tf_num_multi(LogMessage *msg, gint argc, GString *argv[], GString *result) +{ + glong n, m; + + if (!tf_num_parse(argc, argv, "*", &n, &m)) + return; + + g_string_append_printf(result, "%li", n * m); +} + +TEMPLATE_FUNCTION_SIMPLE(tf_num_multi); + +static void +tf_num_div(LogMessage *msg, gint argc, GString *argv[], GString *result) +{ + glong n, m; + + if (!tf_num_parse(argc, argv, "/", &n, &m)) + return; + + if (m == 0) + { + msg_error ("$(/): Division by zero in the template function", + NULL); + return; + } + + g_string_append_printf(result, "%li", (glong)(n / m)); +} + +TEMPLATE_FUNCTION_SIMPLE(tf_num_div); + +static void +tf_num_mod(LogMessage *msg, gint argc, GString *argv[], GString *result) +{ + glong n, m; + + if (!tf_num_parse(argc, argv, "%", &n, &m)) + return; + + if (m == 0) + { + msg_error ("$(%): Division by zero in the template function", + NULL); + return; + } + + g_string_append_printf(result, "%li", n % m); +} + +TEMPLATE_FUNCTION_SIMPLE(tf_num_mod); + static void tf_substr(LogMessage *msg, gint argc, GString *argv[], GString *result) { @@ -64,14 +173,14 @@ tf_substr(LogMessage *msg, gint argc, GString *argv[], GString *result) return; /* get offset position from second argument */ - if (!tf_substr_parse_int(argv[1]->str, &start)) { + if (!tf_parse_int(argv[1]->str, &start)) { msg_error("$(substr) parsing failed, start could not be parsed", evt_tag_str("start", argv[1]->str), NULL); return; } /* if we were called with >2 arguments, third was desired length */ if (argc > 2) { - if (!tf_substr_parse_int(argv[2]->str, &len)) { + if (!tf_parse_int(argv[2]->str, &len)) { msg_error("$(substr) parsing failed, length could not be parsed", evt_tag_str("length", argv[2]->str), NULL); return; } @@ -282,6 +391,11 @@ static Plugin basicfuncs_plugins[] = TEMPLATE_FUNCTION_PLUGIN(tf_grep, "grep"), TEMPLATE_FUNCTION_PLUGIN(tf_if, "if"), TEMPLATE_FUNCTION_PLUGIN(tf_substr, "substr"), + TEMPLATE_FUNCTION_PLUGIN(tf_num_plus, "+"), + TEMPLATE_FUNCTION_PLUGIN(tf_num_minus, "-"), + TEMPLATE_FUNCTION_PLUGIN(tf_num_multi, "*"), + TEMPLATE_FUNCTION_PLUGIN(tf_num_div, "/"), + TEMPLATE_FUNCTION_PLUGIN(tf_num_mod, "%"), }; gboolean diff --git a/tests/unit/test_template.c b/tests/unit/test_template.c index fca0076..aabc3db 100644 --- a/tests/unit/test_template.c +++ b/tests/unit/test_template.c @@ -246,6 +246,15 @@ main(int argc G_GNUC_UNUSED, char *argv[] G_GNUC_UNUSED) testcase(msg, "$(substr $HOST -1)", "p"); testcase(msg, "$(substr $HOST -2 1)", "r"); + testcase(msg, "$(+ $FACILITY_NUM 1)", "20"); + testcase(msg, "$(- $FACILITY_NUM 1)", "18"); + testcase(msg, "$(* $FACILITY_NUM 2)", "38"); + testcase(msg, "$(/ $FACILITY_NUM 2)", "9"); + testcase(msg, "$(% $FACILITY_NUM 3)", "1"); + testcase(msg, "$(/ $FACILITY_NUM 0)", ""); + testcase(msg, "$(% $FACILITY_NUM 0)", ""); + testcase(msg, "$(+ foo bar)", ""); + /* message refs */ testcase(msg, "$(echo ${HOST}@0 ${PID}@1)", "bzorp 23323"); testcase(msg, "$(echo $HOST $PID)@0", "bzorp 23323"); -- 1.7.2.5
Hi Gergely, Thanks for the patch, I see an immediate use-case for this. I applied the patch to the 3.4 tree, and at the same time, I've changed it slightly. On Tue, 2011-05-03 at 22:19 +0200, Gergely Nagy wrote:
Implement addition, substraction, multiplication, division and modulus template functions: $(+ N M), $(- N M), $(* N M), $(/ N M) and $(% N M), respectively.
All of them take two numeric arguments, and log an error,
instead of logging an error, always, insert a "NaN" into the output stream, which stands for Not-a-Number, commonly used to represent non-numbers in underflows. I know it's not that intuitive, but in production environments, generating loads of error messages such as these in the fast path (even more, since you can several invocation of these functions per message), would do no good. So, I've changed errors to debug (so if someone is running it with debugging enabled still get the messages), and in all error paths I've used NaN instead of the empty string. Also, tests didn't cover negative numbers, now I've added to cover them too. Also, while I was at it, I've changed all g_string_append_printf() calls to format_int32_padded(), which is much faster. g_string_append_printf() is double slow, because: 1) it uses printf, but to measure the length of the string, it runs printf without storing its results, and then 2) allocates a buffer to format this result, and then 3) moves the temp buffer to the end of the gstring 4) frees the buffer. So it is not even double, but quadraple slow. :)
if they receive less or more, or in case the number cannot be fully parsed as a number.
Signed-off-by: Gergely Nagy <algernon@balabit.hu>
This is the commit message, please let me know if you disagree with my changes. commit 89b5006f1fc015f1049ec3713638074434beba65 Author: Balazs Scheidler <bazsi@balabit.hu> Date: Tue May 3 23:01:09 2011 +0200 basicfuncs: Implement a few numeric template functions Implement addition, substraction, multiplication, division and modulus template functions: $(+ N M), $(- N M), $(* N M), $(/ N M) and $(% N M), respectively. All of them take two numeric arguments, and result in "NaN" (aka Not-a-Number) in case parameters cannot be parsed, or if a division by zero would occur. Signed-off-by: Gergely Nagy <algernon@balabit.hu> Signed-off-by: Balazs Scheidler <bazsi@balabit.hu> -- Bazsi
Balazs Scheidler <bazsi@balabit.hu> writes:
Hi Gergely,
Thanks for the patch, I see an immediate use-case for this.
I applied the patch to the 3.4 tree, and at the same time, I've changed it slightly.
On Tue, 2011-05-03 at 22:19 +0200, Gergely Nagy wrote:
Implement addition, substraction, multiplication, division and modulus template functions: $(+ N M), $(- N M), $(* N M), $(/ N M) and $(% N M), respectively.
All of them take two numeric arguments, and log an error,
instead of logging an error, always, insert a "NaN" into the output stream, which stands for Not-a-Number, commonly used to represent non-numbers in underflows. I know it's not that intuitive, but in production environments, generating loads of error messages such as these in the fast path (even more, since you can several invocation of these functions per message), would do no good.
Sounds reasonable, and this makes it easier to detect errors too. Thanks for the explanation & idea, I'll use a similar approach in future template functions =)
Also, while I was at it, I've changed all g_string_append_printf() calls to format_int32_padded(), which is much faster.
g_string_append_printf() is double slow, because:
1) it uses printf, but to measure the length of the string, it runs printf without storing its results, and then 2) allocates a buffer to format this result, and then 3) moves the temp buffer to the end of the gstring 4) frees the buffer.
So it is not even double, but quadraple slow. :)
Yikes.
This is the commit message, please let me know if you disagree with my changes.
I completely agree with your changes. Thanks! -- |8]
participants (2)
-
Balazs Scheidler
-
Gergely Nagy