[syslog-ng] syslog-ng anon patch
Roberto Nibali
ratz at tac.ch
Thu Jun 2 09:05:20 CEST 2005
ADDENDUM (hit the wrong button, sorry)
> I don't see the necessity to provide a keyword strip as a subset of replace.
> Please drop it, while referring to the equivalent lines below, written by you.
>
>
>> replace(ips,"0.0.0.0 <http://0.0.0.0>") <--- this is the same as
>>strip(ips)
>> replace(<regex>,"----") <--- this is the same as strip(regex)
That's the place.
>>+ This patch adds the capability to syslog-ng that allows you to strip
>>+ out any given regexp or all IP addresses from log messages before
>>+ they are written to disk. The goal is to give the system administrator
>>+ the means to implement site logging policies, by allowing them easy
>>+ control over exactly what data they retain in their logfiles,
>>+ regardless of what a particular daemon might think is best.
This can also be done with a match and a /dev/null destination. Please be
specific in what your patch achieves.
>>+ Data retention has become a hot legal topic for ISPs and other Online
>>+ Service Providers (OSPs). There are many instances where it is preferable
>>+ to keep less information on users than is collected by default on many
>>+ systems.
Over here it's more an issue of showing less information on users than is
collected. When you work for the state, for banks or insurances, you'll notice
that there the wind is blowing into the other direction. All, without loss, data
is to be stored; and this under penalty even. At least here in Switzerland. If
you lose a message while a potential "break-in" has occured or can be correlated
it might cost you your head :).
>>diff -uNr orig/syslog-ng-1.6.7/doc/syslog-ng-anon.conf new/syslog-ng-1.6.7/doc/syslog-ng-anon.conf
>>--- orig/syslog-ng-1.6.7/doc/syslog-ng-anon.conf 1969-12-31 18:00:00.000000000 -0600
>>+++ new/syslog-ng-1.6.7/doc/syslog-ng-anon.conf 2005-05-30 18:25:40.828858265 -0500
>>@@ -0,0 +1,243 @@
I don't think this sample file is needed.
>>+## sympa.log
>>+
>>+filter f_sympa { program("^(sympa|bounced|archived|task_manager)"); };
>>+destination d_sympa { file("/var/log/sympa.log"); };
>>+log {
>>+ source(s_all);
>>+ filter(f_sympa);
>>+ destination(d_sympa);
>>+ flags(final);
>>+};
>>+
>>+############################################################
>>+## wwsympa.log
>>+
>>+filter f_wwsympa { program("^wwsympa"); };
>>+destination d_wwsympa { file("/var/log/wwsympa.log"); };
>>+log {
>>+ source(s_all);
>>+ filter(f_wwsympa);
>>+ filter(f_strip);
>>+ destination(d_wwsympa);
>>+ flags(final);
>>+};
Too specific to be in a package config file as a skeleton but this is only my view.
Cynically I could argue that by skimming through your sample syslog-ng.conf file
you don't seem to have any of the daemons chroot()'d, yet you
>>+ | KW_STRIP '(' string ')' { $$ = make_filter_strip($3); free($3); }
remove
>>+ | KW_REPLACE '(' string string ')' { $$ = make_filter_replace($3,$4); free($3); free($4); }
>> | KW_FILTER '(' string ')' { $$ = make_filter_call($3); free($3); }
>> ;
>>
>>diff -uNr orig/syslog-ng-1.6.7/src/cfg-lex.l new/syslog-ng-1.6.7/src/cfg-lex.l
>>--- orig/syslog-ng-1.6.7/src/cfg-lex.l 2005-05-30 18:27:50.829842715 -0500
>>+++ new/syslog-ng-1.6.7/src/cfg-lex.l 2005-05-30 18:25:40.827858450 -0500
>>@@ -140,6 +140,8 @@
>> { "netmask", KW_NETMASK },
>> { "host", KW_HOST },
>> { "match", KW_MATCH },
>>+ { "strip", KW_STRIP },
remove
>>+struct filter_expr_node *make_filter_strip(const char *re)
>>+{
>>+ if (strcasecmp(re,"ips") == 0)
>>+ return make_filter_replace(re,"0.0.0.0");
>>+ else
>>+ return make_filter_replace(re,"----");
>>+}
>>+
remove
>>+#define FMIN(a,b) (a)<(b) ? (a):(b)
>>+
>>+static int do_filter_replace(struct filter_expr_node *c,
>>+ struct log_filter *rule UNUSED,
>>+ struct log_info *log)
>>+{
>>+ CAST(filter_expr_re, self, c);
>>+ char * buffer = log->msg->data;
>>+ int snippet_size;
>>+ regmatch_t pmatch;
>>+ char new_msg[2048];
>>+ char * new_msg_max = new_msg+2048;
>>+ char * new_msg_ptr = new_msg;
>>+ int replace_length = strlen(self->replace->data);
>>+
>>+ int error = regexec(&self->regex, buffer, 1, &pmatch, 0);
>>+ if (error != 0) return 1;
>>+ while (error==0) {
>>+ /* copy string snippet which preceeds matched text */
>>+ snippet_size = FMIN(pmatch.rm_so, new_msg_max-new_msg_ptr);
>>+ memcpy(new_msg_ptr, buffer, snippet_size);
>>+ new_msg_ptr += snippet_size;
>>+
>>+ /* copy replacement string */
>>+ snippet_size = FMIN(replace_length, new_msg_max-new_msg_ptr);
>>+ memcpy(new_msg_ptr, self->replace->data, snippet_size);
>>+ new_msg_ptr += snippet_size;
>>+
>>+ /* search for next match */
>>+ buffer += pmatch.rm_eo;
>>+ error = regexec (&self->regex, buffer, 1, &pmatch, REG_NOTBOL);
>>+ }
>>+ /* copy the rest of the old msg */
>>+ snippet_size = FMIN(strlen(buffer),new_msg_max-new_msg_ptr);
>>+ memcpy(new_msg_ptr, buffer, snippet_size);
>>+ new_msg_ptr += snippet_size;
>>+
>>+ ol_string_free(log->msg);
>>+ log->msg = c_format_cstring("%s", new_msg_ptr-new_msg,new_msg);
>>+ return 1;
>>+}
>>+
>>+struct filter_expr_node *make_filter_replace(const char *re, const char *replacement)
>>+{
>>+ int regerr;
>>+ NEW(filter_expr_re, self);
>>+ self->super.eval = do_filter_replace;
>>+ self->replace = format_cstring(replacement);
>>+
>>+ if (strcasecmp(re,"ips") == 0) {
>>+ re = "(25[0-5]|2[0-4][0-9]|[0-1]?[0-9]?[0-9])([\\.\\-](25[0-5]|2[0-4][0-9]|[0-1]?[0-9]?[0-9])){3}";
>>+ }
remove, also because not all IPs are logged in dotted decimals for example.
>>+ regerr = regcomp(&self->regex, re, REG_ICASE | REG_EXTENDED);
>>+ if (regerr) {
>>+ char errorbuf[256];
>>+ regerror(regerr, &self->regex, errorbuf, sizeof(errorbuf));
>>+ werror("Error compiling regular expression: \"%z\" (%z)\n", re, errorbuf);
>>+ KILL(self);
>>+ return NULL;
>>+ }
>>+
>>+ return &self->super;
>>+}
>>+
>> static int do_filter_prog(struct filter_expr_node *c,
>> struct log_filter *rule UNUSED,
>> struct log_info *log)
>>diff -uNr orig/syslog-ng-1.6.7/src/filters.h new/syslog-ng-1.6.7/src/filters.h
>>--- orig/syslog-ng-1.6.7/src/filters.h 2002-02-04 10:07:50.000000000 -0600
>>+++ new/syslog-ng-1.6.7/src/filters.h 2005-05-30 18:25:40.827858450 -0500
>>@@ -66,6 +66,8 @@
>> struct filter_expr_node *make_filter_netmask(const char *nm);
>> struct filter_expr_node *make_filter_host(const char *re);
>> struct filter_expr_node *make_filter_match(const char *re);
>>+struct filter_expr_node *make_filter_strip(const char *re);
remove
>>+struct filter_expr_node *make_filter_replace(const char *re, const char *replacement);
>> struct filter_expr_node *make_filter_call(const char *name);
Best regards,
Roberto Nibali, ratz
--
-------------------------------------------------------------
addr://Rathausgasse 31, CH-5001 Aarau tel://++41 62 823 9355
http://www.terreactive.com fax://++41 62 823 9356
-------------------------------------------------------------
terreActive AG Wir sichern Ihren Erfolg
-------------------------------------------------------------
More information about the syslog-ng
mailing list