filter logger tags from syslog
Hello, I've been using classic syslog for centralizing apache access logs from one server to a remote syslog server, the thing is syslog adds some nasty tags before the lines in the access logs and I cant get them off, ie: "Nov 25 21:25:37 server1 logger:" I would like to know if syslog-ng has the option to filter this kind of stuff, I just want to have the logs sent to the syslog server exactly like I was saving them in a local access.log file. Thanks in advance.
Hi Jose, Jose Sanchez írta:
Hello,
I've been using classic syslog for centralizing apache access logs from one server to a remote syslog server, the thing is syslog adds some nasty tags before the lines in the access logs and I cant get them off, ie:
"Nov 25 21:25:37 server1 logger:"
I would like to know if syslog-ng has the option to filter this kind of stuff, I just want to have the logs sent to the syslog server exactly like I was saving them in a local access.log file.
I don't understand you completely where you use syslog or syslog-ng on these hosts If you use syslog-ng then yes the syslog-ng can do it. There is an example of the possible solutions if both sides are syslog-ng: _client side:_ source s_file{file("/var/log/apache2/access.log" flags(no-parse) );}; destination d_tcp{tcp("10.30.0.32" port(666) template("$MSG\n") );}; log { source(s_file); destination(d_tcp); }; _server side:_ source s_tcp{tcp(port(666) flags(no-parse) );}; destination d_test { file("/var/log/test.log" template("$MSG\n") ); }; log { source (s_tcp); destination(d_test); }; if you use syslogd on client side and syslog-ng on the server side you need to use a config like this (but I am not sure in this case): source s_tcp{udp(port(514));}; destination d_test { file("/var/log/test.log" template("$MSG\n") ); }; log { source (s_tcp); destination(d_test); };
Thanks in advance.
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
Hello Pallagi, Thank you for the prompt response, very appreciated. Im running syslog at the client side and syslog-ng at the server side. Currently I have setup both and Im getting the following prefix into the log file on my syslog-ng server (this log is generated from apache on my client server), "logger: XXX.XXX.XXX.XXX - - [26/Nov/2009:11:11:36 -0600] \"GET... " The issue is how can I configure syslog-ng to remove the "logger:" prefix when logging. Basically I just want a clean log same way like if I had an access_log file configured on apache on the client. Thank you in advance. --- On Thu, 11/26/09, Pallagi Zoltán <pzolee@balabit.hu> wrote:
From: Pallagi Zoltán <pzolee@balabit.hu> Subject: Re: [syslog-ng] filter logger tags from syslog To: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu>, josesan311@yahoo.com Date: Thursday, November 26, 2009, 6:40 AM
Hi Jose,
Jose Sanchez írta:
Hello,
I've been using classic syslog for centralizing apache access logs from one server to a remote syslog server, the thing is syslog adds some nasty tags before the lines in the access logs and I cant get them off, ie:
"Nov 25 21:25:37 server1 logger:"
I would like to know if syslog-ng has the option to filter this kind of stuff, I just want to have the logs sent to the syslog server exactly like I was saving them in a local access.log file.
I don't understand you completely where you use syslog or syslog-ng on these hosts
If you use syslog-ng then yes the syslog-ng can do it. There is an example of the possible solutions if both sides are syslog-ng:
client side:
source s_file{file("/var/log/apache2/access.log"
flags(no-parse)
);};
destination d_tcp{tcp("10.30.0.32" port(666)
template("$MSG\n")
);};
log {
source(s_file);
destination(d_tcp);
};
server side:
source s_tcp{tcp(port(666)
flags(no-parse)
);};
destination d_test { file("/var/log/test.log"
template("$MSG\n")
); };
log {
source (s_tcp);
destination(d_test);
};
if you use syslogd on client side and syslog-ng on the server side you need to use a config like this (but I am not sure in this case):
source s_tcp{udp(port(514));};
destination d_test { file("/var/log/test.log"
template("$MSG\n")
); };
log {
source (s_tcp);
destination(d_test);
};
Thanks in advance.
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
Sorry, forgot to add, this is my syslog-ng.conf file, # syslog-ng configuration file. # # This should behave pretty much like the original syslog on RedHat. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (no); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); internal(); udp(ip(0.0.0.0) port(514)); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" sync(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; # vim:ft=syslog-ng:ai:si:ts=4:sw=4:et: source s_tcp{udp(port(514));}; destination d_test { file("/var/log/test.log" template("$MSG\n") ); }; log { source (s_tcp); destination(d_test); }; --- On Thu, 11/26/09, Jose Sanchez <josesan311@yahoo.com> wrote:
From: Jose Sanchez <josesan311@yahoo.com> Subject: Re: [syslog-ng] filter logger tags from syslog To: syslog-ng@lists.balabit.hu Date: Thursday, November 26, 2009, 11:16 AM Hello Pallagi,
Thank you for the prompt response, very appreciated. Im running syslog at the client side and syslog-ng at the server side. Currently I have setup both and Im getting the following prefix into the log file on my syslog-ng server (this log is generated from apache on my client server),
"logger: XXX.XXX.XXX.XXX - - [26/Nov/2009:11:11:36 -0600] \"GET... "
The issue is how can I configure syslog-ng to remove the "logger:" prefix when logging. Basically I just want a clean log same way like if I had an access_log file configured on apache on the client.
Thank you in advance.
--- On Thu, 11/26/09, Pallagi Zoltán <pzolee@balabit.hu> wrote:
From: Pallagi Zoltán <pzolee@balabit.hu> Subject: Re: [syslog-ng] filter logger tags from syslog To: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu>, josesan311@yahoo.com Date: Thursday, November 26, 2009, 6:40 AM
Hi Jose,
Jose Sanchez írta:
Hello,
I've been using classic syslog for centralizing apache access logs from one server to a remote syslog server, the thing is syslog adds some nasty tags before the lines in the access logs and I cant get them off, ie:
"Nov 25 21:25:37 server1 logger:"
I would like to know if syslog-ng has the option to filter this kind of stuff, I just want to have the logs sent to the syslog server exactly like I was saving them in a local access.log file.
I don't understand you completely where you use syslog or syslog-ng on these hosts
If you use syslog-ng then yes the syslog-ng can do it. There is an example of the possible solutions if both sides are syslog-ng:
client side:
source s_file{file("/var/log/apache2/access.log"
flags(no-parse)
);};
destination d_tcp{tcp("10.30.0.32" port(666)
template("$MSG\n")
);};
log {
source(s_file);
destination(d_tcp);
};
server side:
source s_tcp{tcp(port(666)
flags(no-parse)
);};
destination d_test { file("/var/log/test.log"
template("$MSG\n")
); };
log {
source (s_tcp);
destination(d_test);
};
if you use syslogd on client side and syslog-ng on the server side you need to use a config like this (but I am not sure in this case):
source s_tcp{udp(port(514));};
destination d_test { file("/var/log/test.log"
template("$MSG\n")
); };
log {
source (s_tcp);
destination(d_test);
};
Thanks in advance.
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
Jose Sanchez írta:
Hello Pallagi,
Thank you for the prompt response, very appreciated. Im running syslog at the client side and syslog-ng at the server side.
Ok, I've tested it with this combination
Currently I have setup both and Im getting the following prefix into the log file on my syslog-ng server (this log is generated from apache on my client server),
"logger: XXX.XXX.XXX.XXX - - [26/Nov/2009:11:11:36 -0600] \"GET... "
The issue is how can I configure syslog-ng to remove the "logger:" prefix when logging. Basically I just want a clean log same way like if I had an access_log file configured on apache on the client.
I think my solution remove it. original log in access.log: 127.0.0.1 - - [26/Nov/2009:19:40:14 +0100] "GET / HTTP/1.1" 200 56 "-" "Mozilla/5.0 (X11; U; Linux i686; hu-HU; rv:1.9.0.15) Gecko/2009102815 Ubuntu/9.04 (jaunty) Firefox/3.0.15" log on the server without $MSG template: Nov 26 20:02:25 127.0.0.1 logger[5645]: 127.0.0.1 - - [26/Nov/2009:20:02:25 +0100] "GET / HTTP/1.1" 200 56 "-" "Mozilla/5.0 (X11; U; Linux i686; hu-HU; rv:1.9.0.15) Gecko/2009102815 Ubuntu/9.04 (jaunty) Firefox/3.0.15" and the log on server with $MSG template (it was my solution): 127.0.0.1 - - [26/Nov/2009:20:19:47 +0100] "GET / HTTP/1.1" 200 56 "-" "Mozilla/5.0 (X11; U; Linux i686; hu-HU; rv:1.9.0.15) Gecko/2009102815 Ubuntu/9.04 (jaunty) Firefox/3.0.15" There is no more "logger" prefix. Your config file also seems to be correct (but you have duplicated udp sources). Didn't you forget to restart your syslog-ng? I don't really have more idea why it doesn't work for you.
Thank you in advance.
--- On Thu, 11/26/09, Pallagi Zoltán <pzolee@balabit.hu> wrote:
From: Pallagi Zoltán <pzolee@balabit.hu> Subject: Re: [syslog-ng] filter logger tags from syslog To: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu>, josesan311@yahoo.com Date: Thursday, November 26, 2009, 6:40 AM
Hi Jose,
Jose Sanchez írta:
Hello,
I've been using classic syslog for centralizing apache access logs from one server to a remote syslog server, the thing is syslog adds some nasty tags before the lines in the access logs and I cant get them off, ie:
"Nov 25 21:25:37 server1 logger:"
I would like to know if syslog-ng has the option to filter this kind of stuff, I just want to have the logs sent to the syslog server exactly like I was saving them in a local access.log file.
I don't understand you completely where you use syslog or syslog-ng on these hosts
If you use syslog-ng then yes the syslog-ng can do it. There is an example of the possible solutions if both sides are syslog-ng:
client side:
source s_file{file("/var/log/apache2/access.log"
flags(no-parse)
);};
destination d_tcp{tcp("10.30.0.32" port(666)
template("$MSG\n")
);};
log {
source(s_file);
destination(d_tcp);
};
server side:
source s_tcp{tcp(port(666)
flags(no-parse)
);};
destination d_test { file("/var/log/test.log"
template("$MSG\n")
); };
log {
source (s_tcp);
destination(d_test);
};
if you use syslogd on client side and syslog-ng on the server side you need to use a config like this (but I am not sure in this case):
source s_tcp{udp(port(514));};
destination d_test { file("/var/log/test.log"
template("$MSG\n")
); };
log {
source (s_tcp);
destination(d_test);
};
Thanks in advance.
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
Hello, I believe its working now, thanks a lot for everything. --- On Thu, 11/26/09, Zoltán Pallagi <pzolee@balabit.hu> wrote:
From: Zoltán Pallagi <pzolee@balabit.hu> Subject: Re: [syslog-ng] filter logger tags from syslog To: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu>, josesan311@yahoo.com Date: Thursday, November 26, 2009, 1:48 PM
Jose Sanchez írta:
Hello Pallagi,
Thank you for the prompt response, very appreciated. Im running syslog at the client side and syslog-ng at the server side.
Ok, I've tested it with this combination
Currently I have setup both and Im getting the following prefix into the log file on my syslog-ng server (this log is generated from apache on my client server),
"logger: XXX.XXX.XXX.XXX - - [26/Nov/2009:11:11:36 -0600] \"GET... "
The issue is how can I configure syslog-ng to remove the "logger:" prefix when logging. Basically I just want a clean log same way like if I had an access_log file configured on apache on the client.
I think my solution remove it.
original log in access.log:
127.0.0.1 - - [26/Nov/2009:19:40:14 +0100] "GET / HTTP/1.1" 200 56 "-" "Mozilla/5.0 (X11; U; Linux i686; hu-HU; rv:1.9.0.15) Gecko/2009102815 Ubuntu/9.04 (jaunty) Firefox/3.0.15"
log on the server without $MSG template:
Nov 26 20:02:25 127.0.0.1 logger[5645]: 127.0.0.1 - - [26/Nov/2009:20:02:25 +0100] "GET / HTTP/1.1" 200 56 "-" "Mozilla/5.0 (X11; U; Linux i686; hu-HU; rv:1.9.0.15) Gecko/2009102815 Ubuntu/9.04 (jaunty) Firefox/3.0.15"
and the log on server with $MSG template (it was my solution):
127.0.0.1 - - [26/Nov/2009:20:19:47 +0100] "GET / HTTP/1.1" 200 56 "-" "Mozilla/5.0 (X11; U; Linux i686; hu-HU; rv:1.9.0.15) Gecko/2009102815 Ubuntu/9.04 (jaunty) Firefox/3.0.15"
There is no more "logger" prefix. Your config file also seems to be correct (but you have duplicated udp sources).
Didn't you forget to restart your syslog-ng? I don't really have more idea why it doesn't work for you.
Thank you in advance.
--- On Thu, 11/26/09, Pallagi Zoltán <pzolee@balabit.hu> wrote:
From: Pallagi Zoltán <pzolee@balabit.hu> Subject: Re: [syslog-ng] filter logger tags from syslog To: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu>, josesan311@yahoo.com Date: Thursday, November 26, 2009, 6:40 AM
Hi Jose,
Jose Sanchez írta:
Hello,
I've been using classic syslog for centralizing apache access logs from one server to a remote syslog server, the thing is syslog adds some nasty tags before the lines in the access logs and I cant get them off, ie:
"Nov 25 21:25:37 server1 logger:"
I would like to know if syslog-ng has the option to filter this kind of stuff, I just want to have the logs sent to the syslog server exactly like I was saving them in a local access.log file.
I don't understand you completely where you use syslog or syslog-ng on these hosts
If you use syslog-ng then yes the syslog-ng can do it. There is an example of the possible solutions if both sides are syslog-ng:
client side:
source s_file{file("/var/log/apache2/access.log"
flags(no-parse)
);};
destination d_tcp{tcp("10.30.0.32" port(666)
template("$MSG\n")
);};
log {
source(s_file);
destination(d_tcp);
};
server side:
source s_tcp{tcp(port(666)
flags(no-parse)
);};
destination d_test { file("/var/log/test.log"
template("$MSG\n")
); };
log {
source (s_tcp);
destination(d_test);
};
if you use syslogd on client side and syslog-ng on the server side you need to use a config like this (but I am not sure in this case):
source s_tcp{udp(port(514));};
destination d_test { file("/var/log/test.log"
template("$MSG\n")
); };
log {
source (s_tcp);
destination(d_test);
};
Thanks in advance.
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
participants (3)
-
Jose Sanchez
-
Pallagi Zoltán
-
Zoltán Pallagi