[syslog-ng] possible memleak or bad configuration?
Andreas Sartori
andreas.sartori at fh-salzburg.ac.at
Mon Mar 15 17:03:45 CET 2010
today i compiled a 3.1.beta2 and its the same issue with the memory.
after a reboot in the morning, we are currently at 2gb mem.
i hope we can get that fixed!
-andy
On 3/15/10 8:45 AM, Andreas Sartori wrote:
> we were running 3.0.4 (self compiled with libdbi for oracle) (same
> problem) and then upgraded to 3.0.5 rhel5 from (directly from the website).
>
> the box itself is a vm on esxi4u1 with centos 5.4 x86_84.
>
> -andy
>
>
>
> On 3/13/10 7:03 PM, Martin Holste wrote:
>> The db parser code had a big memory leak in previous 3.1 versions but
>> was fixed a few months ago; what build are you running? We process 2
>> billion logs per day through db parser with no leaks at all using the
>> build from git commit 9ef6062c1cf72a3f7da880ac245f9ee080bea992.
>>
>> --Martin
>>
>> On Sat, Mar 13, 2010 at 2:22 AM, Andreas Sartori
>> <andreas.sartori at fh-salzburg.ac.at
>> <mailto:andreas.sartori at fh-salzburg.ac.at>> wrote:
>>
>> hello,
>>
>>
>> we have setup a central logging server. currently we are logging
>> firewalls
>> and some webserver / mailserver for testing purpose. the memory
>> usage on
>> the logging server is badly increasing. after 2 days of operation we are
>> at 6.8 gb ram usage.
>>
>> can someone help out, what information do you need to help?
>>
>> thanks in advance.
>>
>> -andy
>>
>> ------------
>>
>> @version:3.0
>> #
>> # configuration file for syslog-ng, customized for remote logging
>> #
>>
>> options {
>> owner("root");
>> group("root");
>> perm(0600);
>> dir_perm(0750);
>> create_dirs(yes);
>> log_fifo_size(10000);
>> };
>>
>>
>>
>> ################################################################################################
>> ######################### SOURCES
>> ##############################
>> ################################################################################################
>>
>> # Syslog internal logging
>> source s_internal { internal(); };
>> destination d_syslognglog { file("/var/log/syslog-ng.log"); };
>> log { source(s_internal); destination(d_syslognglog); };
>>
>>
>> # Remote logging
>> source s_remote {
>> tcp(ip(0.0.0.0) max-connections(20) port(514)
>> keep_hostname(yes));
>> udp(ip(0.0.0.0) port(514) use_dns(no) log_fetch_limit(500)
>> log_iw_size(1000));
>> };
>>
>>
>> ################################################################################################
>> ######################### FILTER
>> ##############################
>> ################################################################################################
>>
>> filter http-official { netmask(xxx.xxx.xxx.47/255.255.255.255
>> <http://255.255.255.255>) or
>> netmask(xxx.xxx.xxx.48/255.255.255.255<http://255.255.255.255>) or
>> netmask(xxx.xxx.xxx.167/255.255.255.255<http://255.255.255.255>) or
>> netmask(xxx.xxx.xxx.46/255.255.255.255<http://255.255.255.255>) or
>> netmask(xxx.xxx.xxx.52/255.255.255.255<http://255.255.255.255>) or
>> netmask(xxx.xxx.xxx.25/255.255.255.255<http://255.255.255.255>) or
>> netmask(xxx.xxx.xxx.26/255.255.255.255<http://255.255.255.255>); };
>>
>> filter mail-proxy-internal { netmask(10.10.9.20/255.255.255.255
>> <http://10.10.9.20/255.255.255.255>) and not
>> program("perdition"); };
>> filter mail-relay-internal { netmask(10.10.9.30/255.255.255.255
>> <http://10.10.9.30/255.255.255.255>); };
>>
>> filter mail-relay-alpha-external-out {
>> netmask(xxx.xxx.xxx.59/255.255.255.255<http://255.255.255.255>) and
>> facility(local1); };
>> filter mail-relay-beta-external-out {
>> netmask(xxx.xxx.xxx.60/255.255.255.255<http://255.255.255.255>) and
>> facility(local1); };
>> filter mail-relay-alpha-external-in {
>> netmask(xxx.xxx.xxx.59/255.255.255.255<http://255.255.255.255>) and
>> facility(mail); };
>> filter mail-relay-beta-external-in {
>> netmask(xxx.xxx.xxx.60/255.255.255.255<http://255.255.255.255>) and
>> facility(mail); };
>>
>> filter mail-proxy-node1-external {
>> netmask(xxx.xxx.xxx.18/255.255.255.255<http://255.255.255.255>)
>> and not program("perdition"); };
>> filter mail-proxy-node2-external {
>> netmask(xxx.xxx.xxx.22/255.255.255.255<http://255.255.255.255>)
>> and not program("perdition"); };
>>
>> filter vpn { netmask(10.20.40.0/255.255.255.0
>> <http://10.20.40.0/255.255.255.0>); };
>> filter fw-intern-all { netmask(10.10.20.1/255.255.255.255
>> <http://10.10.20.1/255.255.255.255>); };
>>
>> filter fw-intern-security {
>> netmask(10.10.20.1/255.255.255.255
>> <http://10.10.20.1/255.255.255.255>) and
>> match("security" value(".classifier.class")
>> type("string"));
>> };
>>
>> filter fw-intern-info {
>> netmask(10.10.20.1/255.255.255.255
>> <http://10.10.20.1/255.255.255.255>) and
>> match("informational" value(".classifier.class")
>> type("string"));
>> };
>>
>> filter fw-intern-rest {
>> netmask(10.10.20.1/255.255.255.255
>> <http://10.10.20.1/255.255.255.255>) and not
>> match("security" value(".classifier.class")
>> type("string")) and not
>> match("informational" value(".classifier.class")
>> type("string"));
>> };
>>
>>
>> filter fw-extern-all { netmask(10.80.11.20/255.255.255.255
>> <http://10.80.11.20/255.255.255.255>); };
>>
>> filter fw-extern-security {
>> netmask(10.80.11.20/255.255.255.255
>> <http://10.80.11.20/255.255.255.255>) and
>> match("security" value(".classifier.class")
>> type("string"));
>> };
>>
>> filter fw-extern-info {
>> netmask(10.80.11.20/255.255.255.255
>> <http://10.80.11.20/255.255.255.255>) and
>> match("informational" value(".classifier.class")
>> type("string"));
>> };
>>
>> filter fw-extern-rest {
>> netmask(10.80.11.20/255.255.255.255
>> <http://10.80.11.20/255.255.255.255>) and not
>> match("security" value(".classifier.class")
>> type("string")) and not
>> match("informational" value(".classifier.class")
>> type("string"));
>> };
>>
>> filter fw-extern-new { netmask(10.80.11.30/255.255.255.255
>> <http://10.80.11.30/255.255.255.255>); };
>>
>> ################################################################################################
>> ######################### PARSER
>> ##############################
>> ################################################################################################
>>
>> parser pattern_db_fwint {
>> db_parser(
>> file("/etc/syslog-ng/fw-int_patterndb.xml")
>> );
>> };
>>
>> parser pattern_db_fwext {
>> db_parser(
>> file("/etc/syslog-ng/fw-ext_patterndb.xml")
>> );
>> };
>>
>> ################################################################################################
>> ######################### DESTINATIONS
>> ##############################
>> ################################################################################################
>>
>> destination http-log { file("/logging/server/web/$HOST"
>> template("$MSGONLY\n") template-escape(no) owner("root") group("root")
>> perm(0644)); };
>>
>> destination mail-out {
>> file("/logging/server/mail/mail-out_$MONTH.log"); };
>> destination mail-in {
>> file("/logging/server/mail/mail-in_$MONTH.log"); };
>>
>> destination vpn {
>> file("/logging/network/vpn_$MONTH.log" flush_lines(10));
>> };
>>
>> destination fw-intern-all {
>> file("/logging/network/fw-intern_$MONTH.log" flush_lines(10));
>> };
>>
>> destination fw-extern-all {
>> file("/logging/network/fw-extern_$MONTH.log" flush_lines(10));
>> };
>>
>>
>> destination fw-extern-new {
>> file("/logging/network/fw-new_$MONTH.log" flush_lines(10));
>> };
>>
>>
>> destination dump {
>> file("/logging/network/dump.log" template
>> ("$R_YEAR-$R_MONTH-$R_DAY
>> $R_HOUR:$R_MIN:$R_SEC, $HOST, $FIREWALL_SEQ, $MSGHDR, 0, $FIREWALL_IO,
>> $FIREWALL_PROTO, $FIREWALL_SCR_LAN, $FIREWALL_SRC_IP,
>> $FIREWALL_SRC_PORT,
>> $FIREWALL_DST_LAN, $FIREWALL_DST_IP, $FIREWALL_DST_PORT,
>> $FIREWALL_NAT_SRC_IP, $FIREWALL_NAT_DST_IP, $FIREWALL_RULE,
>> $FIREWALL_REASON, $FIREWALL_DURATION\n"));
>> # file("/logging/network/dump.log" template ("$MSGHDR\n")
>> flush_lines(5));
>> };
>>
>>
>>
>> ################################################################################################
>> ######################### FINAL-LOGS
>> ##############################
>> ################################################################################################
>>
>> ##### TO FILE
>>
>> log { source(s_remote); filter(http-official); destination(http-log); };
>> log { source(s_remote); filter(mail-proxy-internal);
>> destination(mail-out); };
>> log { source(s_remote); filter(mail-relay-internal);
>> destination(mail-out); };
>> log { source(s_remote); filter(mail-relay-alpha-external-out);
>> destination(mail-out); };
>> log { source(s_remote); filter(mail-relay-beta-external-out);
>> destination(mail-out); };
>> log { source(s_remote); filter(mail-proxy-node1-external);
>> destination(mail-out); };
>> log { source(s_remote); filter(mail-proxy-node2-external);
>> destination(mail-out); };
>> log { source(s_remote); filter(mail-relay-alpha-external-in);
>> destination(mail-in); };
>> log { source(s_remote); filter(mail-relay-beta-external-in);
>> destination(mail-in); };
>> log { source(s_remote); filter(vpn); destination(vpn); };
>> log { source(s_remote); filter(fw-intern-all);
>> destination(fw-intern-all); };
>> log { source(s_remote); filter(fw-extern-new);
>> destination(fw-extern-new); };
>> log { source(s_remote); filter(fw-extern-all);
>> destination(fw-extern-all);
>> flags(final); };
>>
>>
>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation:
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.campin.net/syslog-ng/faq.html
>>
>>
>>
>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.campin.net/syslog-ng/faq.html
>>
>
--
___________________________________________
FACHHOCHSCHULE SALZBURG GmbH
Salzburg University of Applied Sciences
Andreas Sartori
Systems Engineer
IS - Information Services
Urstein Süd 1 | 5412 Puch/Salzburg | Austria
fon: +43 (0)50-2211-1655 | fax: -1699
web: www.fh-salzburg.ac.at
Gerichtsstand Salzburg | FN166054y
WELCOME TO YOUR FUTURE!
___________________________________________
More information about the syslog-ng
mailing list