- syslog-ng - lists.balabit.hu

set tag in log statement
by Patrick Hemmer 10 Jan '12

10 Jan '12

Would it be possible to set tags on a message when it is matched by a `log {}` statement? This would be useful for complex log configurations so you could filter out messages that were logged by previous `log {}` statements without having to write really long `filter {}`s. I'm not sure if this is possible or not. Would depend on the order of operations, like if a filter block in a latter log statement is evaluated at the same time as an earlier filter block, or if its evaluated when it reaches the `log {}`. Such a config would look like filter f_a { facility(mail); }; log { source(s_local); filter(f_a); destination(d_a); tags("tag1"); }; filter f_b { facility(local0); }; log { source(s_local); filter(f_b); destination(d_b); tags("tag1"); }; filter f_c { not tags("tag1"); }; log { source(s_local); filter(f_c); destination (d_c); };

4 8

Escape slashes etc in macro variables ($PROGRAM)
by Göran Sandahl 10 Jan '12

10 Jan '12

Howdy, Is there a option for escaping macro variables such as $PROGRAM? I've seen a couple of discussions about this on the list but they never seem to come to a conclusion. Case in point: I'm creating an output file macro like this: file("/var/log/$SOURCEIP.$PROGRAM.log" ... and Syslog-ng failes to write the file whenever $PROGRAM looks similar to "postfix/quemgr" since its then trying to "create" the directory postfix due to the slash. Of course, thats not what I want. Can this be circumvented somehow? Cheers! Goran

5 5

Re: [syslog-ng] Syslog-ng 3.2 connection timeout with firewall
by André Larose 10 Jan '12

10 Jan '12

Hi, I found the timeout issue. It turns out that the keep_alive is not generated by Syslog-NG but by the OS. I adjusted the following values: tcp_keepalive_time tcp_keepalive_intvl tcp_keepalive_probes First in the live Kernel with the command : sysctl -w net.ipv4.tcp_keepalive_time=180 net.ipv4.tcp_keepalive_intvl=180 net.ipv4.tcp_keepalive_probes=9 Then I made it permanent with adding to /etc/sysctl.conf: cat >>/etc/sysctl.conf <<EOF # # Keepalive parameters for the syslog-ng # net.ipv4.tcp_keepalive_time = 180 net.ipv4.tcp_keepalive_intvl = 180 net.ipv4.tcp_keepalive_probes = 9 EOF You can verify your work is done with the following command: sysctl net.ipv4.tcp_keepalive_time net.ipv4.tcp_keepalive_intvl net.ipv4.tcp_keepalive_probes net.ipv4.tcp_keepalive_time = 180 net.ipv4.tcp_keepalive_intvl = 180 net.ipv4.tcp_keepalive_probes = 9 I found the information from this URL: http://tldp.org/HOWTO/TCP-Keepalive-HOWTO/usingkeepalive.html Maybe it could be mentioned in the documentation that the so_keep-alive option can be tweaked at the kernel level. Regards, ------------------------------ Message: 8 Date: Tue, 3 Jan 2012 17:46:06 -0500 From: Andr? Larose <andre.larose(a)telus.com<mailto:andre.larose@telus.com>> Subject: [syslog-ng] Syslog-ng 3.2 connection timeout with firewall To: "syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>" <syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>> Message-ID: <BDB004F99074254BBCEE3AB94F31FDF5435439F706(a)WP40066.corp.ads<mailto:BDB004F99074254BBCEE3AB94F31FDF5435439F706@WP40066.corp.ads>> Content-Type: text/plain; charset="iso-8859-1" Hi, I have two syslog-ng 3.2, one client and one server. The two are separated by a firewall and a load balancer. I noticed that after some inactivity I was not able to receive logs from my client. So I started some tcpdump on both servers to check the traffic. From what I see the firewall will close the connection after some time, so when the client sends traffic it gets dropped. I added the keep-alive(yes) and so_keep-alive(yes) and the mark_freq(60) to the configs. But I still do not see keepalive packets with tcpdump. Am I missing other parameters to have "keepalive" traffic sent ? Thank you in advance.

1 0

program destination and init??
by Matt Zagrabelny 09 Jan '12

09 Jan '12

Hi, I've got a question regarding the program destination and how/when it spawns the program. I've got a few program destination directives in my syslog-ng config. When I start the syslog-ng daemon, the program's (of the program destinations) parents are the syslog-ng daemon. After some amount of time (perhaps 20 or 30 hours) I get more programs spawned. Some of the programs have parents that are 'init' (PID = 1). One of my programs is 'vpndb' - see the grep output for an example: $ ps -ef | grep vpndb root 610 1 41 Nov23 ? 2-01:29:55 perl /usr/bin/vpndb root 4036 605 0 06:44 ? 00:00:00 /bin/sh -c /usr/bin/vpndb root 4039 4036 0 06:44 ? 00:00:00 perl /usr/bin/vpndb root 5231 1 26 Nov24 ? 1-02:09:12 perl /usr/bin/vpndb root 5322 1 1 Nov27 ? 00:21:26 perl /usr/bin/vpndb root 5658 1381 0 08:33 pts/4 00:00:00 grep vpndb root 16167 1 12 Nov26 ? 06:19:10 perl /usr/bin/vpndb root 26542 1 19 Nov25 ? 14:12:19 perl /usr/bin/vpndb Above there are 5 vpndb programs with a parent of PID = 1. One of the vpndb's belongs to syslog-ng. Two of my other program destinations are "ssh" commands. They exhibit the same symptoms (multiple programs running with parents of PID = 1). Hence I don't think this problem is related to the specific program being executed. Does anyone have any idea why I am getting multiple programs spawned? (I presume that syslog-ng is spawning these programs.) I do have cron jobs that will 'reload' the syslog-ng daemon. Thanks for any hints. -matt zagrabelny

5 11

[PATCH] logrewrite: Correctly initialize the conditional rules.
by Gergely Nagy 09 Jan '12

09 Jan '12

In case where the condition of a rewrite used a filter reference instead of inlining the filter itself, the filter expression was never initialized. This resulted in the condition always evaluating to not-match, making the following construct completely useless: rewrite("new-value", value("MESSAGE"), condition(filter(f_my_filter))) The fix is to introduce a new function, log_rewrite_init_method(), which will initialize the condition filter if it exists, and has an init method. Then we bind this method to LogPipe's init callback during log_rewrite_init(). This way, when initializing the rewrite pipe, the condition will get initialised too. Reported-by: Thomas Wollner <tw(a)wollner-net.de> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- lib/logrewrite.c | 12 ++++++++++++ 1 files changed, 12 insertions(+), 0 deletions(-) diff --git a/lib/logrewrite.c b/lib/logrewrite.c index a578439..3ff245a 100644 --- a/lib/logrewrite.c +++ b/lib/logrewrite.c @@ -75,6 +75,17 @@ log_rewrite_free_method(LogPipe *s) log_process_pipe_free_method(s); } +static gboolean +log_rewrite_init_method(LogPipe *s) +{ + LogRewrite *self = (LogRewrite *) s; + + if (self->condition && self->condition->init) + self->condition->init(self->condition, log_pipe_get_config(s)); + + return TRUE; +} + static void log_rewrite_init(LogRewrite *self) { @@ -83,6 +94,7 @@ log_rewrite_init(LogRewrite *self) self->super.super.flags |= PIF_CLONE; self->super.super.free_fn = log_rewrite_free_method; self->super.super.queue = log_rewrite_queue; + self->super.super.init = log_rewrite_init_method; self->value_handle = LM_V_MESSAGE; } -- 1.7.7.3

2 1

Error while starting Syslog-ng
by Kareep Navas 03 Jan '12

03 Jan '12

Error creating AF_INET socket (Error 0) Error initializing configuration, exiting.

2 1

What is the latest version of Syslog-ng?
by Kareep Navas 03 Jan '12

03 Jan '12

Hi All, We are currently using syslog-ng 1.4.14 and i would like to use & know the latest version of this syslog-ng? What is the purpose of the latest version? How to download the latest version? What are all the dependencies? Please direct me.. Thank you,

2 1

missing IP information at the message part
by Oğuz Yarımtepe 01 Jan '12

01 Jan '12

I have syslog-ng and logzilla 2.9.9g installed on a Debian squeeze machine. I can see the logs at the web interface. I configured syslog-ng to collect apache error and access logs. The problem is that i don't see IP information at the message part, on web interface. For ex, one of the machine that is sending apache logs is "www" machine. The syslog-ng configuration is as below: # All messages send to a remote site # #log { source(s_src); destination(d_net); }; #manual edit options { log_msg_size(8192); }; ####### # sources # ####### source s_apache_access { # unix-stream("/var/log/httpd/apache_log.socket" # max-connections(512) # keep-alive(yes)); file ("/var/log/apache2/access.log" flags(no-parse)); }; source s_apache_error { # unix-stream("/var/log/httpd/apache_log.socket" # max-connections(512) # keep-alive(yes)); file ("/var/log/apache2/error.log" flags(no-parse)); }; ################ # destinations # ################ destination d_custom_access { file("/var/log/apache2/custom_access.log"); }; destination d_custom_error { file("/var/log/apache2/custom_error.log"); }; destination d_apache_tls { tcp("192.168.1.145" port(6514) tls(ca_dir("/etc/syslog-ng/etc/ca.d") key_file("/etc/syslog-ng/etc/key.d/debian4key.pem") cert_file("/etc/syslog-ng/etc/cert.d/debian4cert.pem"))); }; ########### # filters # ########### filter f_apache_access { message("GET|POST"); }; filter f_apache_error { message("error"); }; log { source(s_apache_access); # filter(f_apache_access); destination(d_apache_tls); }; log { source(s_apache_error); # filter(f_apache_access); destination(d_apache_tls); }; and # tail -n1 /var/log/apache2/access.log 78.185.240.170 - - [01/Jan/2012:13:35:06 +0200] "GET /images/rand/randpic11.jpg HTTP/1.1" 200 45670 "http://www.comu.edu.tr/" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.63 Safari/535.7" www:/home/oguz# For the www machine i don't see the 78.185.240.170 information at the web ui, but the rest of the line starting with -- Thogh another machine, "reverse", is sending apache logs with and additional field and i can see its log at the web ui without any missing information. The reverse has the same configuration except from the key information. # tail -n1 /var/log/apache2/other_vhosts_access.log www.beeseurope.eu:80 178.154.160.29 - - [01/Jan/2012:13:36:46 +0200] "GET /mod/forum/discuss.php?d=55&parent=88 HTTP/1.1" 200 34557 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)" At the web interface, I can see the IP information also at the message part. My syslog-ng configuration at the server is as below: options { long_hostnames(off); # doesn't actually help on Solaris, log(3) truncates at 1024 chars log_msg_size(8192); # buffer just a little for performance flush_lines(16384); # memory is cheap, buffer messages unable to write (like to loghost) log_fifo_size(16384); # Hosts we don't want syslog from #bad_hostname("^(ctld.|cmd|tmd|last)$"); # The time to wait before a dead connection is reestablished (seconds) time_reopen(10); #Use DNS so that our good names are used, not hostnames use_dns(yes); dns_cache(yes); #Use the whole DNS name use_fqdn(yes); keep_hostname(yes); chain_hostnames(no); #Read permission for everyone perm(0644); # The default action of syslog-ng 1.6.0 is to log a STATS line # to the file every 10 minutes. That's pretty ugly after a while. # Change it to every 12 hours so you get a nice daily update of # # how many messages syslog-ng missed (0). # stats(43200); log_msg_size(8192); }; source s_apache { tcp(ip(0.0.0.0) port(6514) tls( key_file("/etc/syslog-ng/etc/key.d/debian1key.pem") cert_file("/etc/syslog-ng/etc/cert.d/debian1cert.pem") ca_dir("/etc/syslog-ng/etc/ca.d"))); }; filter f_apache_access { message("GET|POST"); }; filter f_apache_error { message("error"); }; # Create destination to LogZilla destination d_logzilla { program("/opt/logzilla-2.9.9g/scripts/db_insert.pl" template("$HOST\t$FACILITY\t$PRIORITY\t$LEVEL\t$TAG\t$YEAR-$MONTH-$DAY\t$HOUR:$MIN:$SEC\t$PROGRAM\t$MSG\n") #template("$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC\t$HOST\t$PRI\t$PROGRAM\t$MSGONLY\n") ); }; # test purposes destination d_apache_test_access { file("/var/log/apache2/hosts/$HOST/$YEAR/$MONTH/$DAY/access.log" create_dirs(yes)); }; destination d_apache_test_error { file("/var/log/apache2/hosts/$HOST/$YEAR/$MONTH/$DAY/error.log" create_dirs(yes)); }; # Tell syslog-ng to log to our new destination log { source(s_apache); destination(d_logzilla); }; log { source(s_apache); filter(f_apache_error); destination(d_apache_test_error); }; log { source(s_apache); filter(f_apache_access); destination(d_apache_test_access); }; Any ides about how i can fix the missing IP information at the web ui of logzilla?

1 0

Re: [syslog-ng] syslog-ng Digest, Vol 80, Issue 33
by Anup Shetty 23 Dec '11

23 Dec '11

I cant execute those commands. Here's the error Unknown command Syntax: pdbtool <command> [options] Possible commands are: match Match a message against the pattern database dump Dump pattern datebase tree merge Merge pattern databases dictionary Dump pattern dictionary Version syslog-ng-premium-edition 3.2.1 Installer-Version: 3.2.1 > > > > > > ------------------------------ > > Message: 3 > Date: Thu, 22 Dec 2011 13:11:05 -0600 > From: Martin Holste <mcholste(a)gmail.com> > Subject: Re: [syslog-ng] Pattern matching. > To: "Syslog-ng users' and developers' mailing list" > <syslog-ng(a)lists.balabit.hu> > Message-ID: > <CANpnLHgau7bZrSP2ro0QY=a8ZcJZLyqJgAVegWufDuszOjuCMA(a)mail.gmail.com > > > Content-Type: text/plain; charset=ISO-8859-1 > > You can also include an example pattern as part of the actual rule like > this: > > <ruleset> > <program></program> > <rule id="2"> > <pattern>@ESTRING:user::@ Security Microsoft > Windows security auditing.: [Success Audit] A computer account was > changed. Subject: Security ID: S-1-5-7 Account Name: > ANONYMOUS LOGON Account Domain: NT AUTHORITY Logon ID: 0x3e6 > Computer Account That Was Changed: Security ID: @ESTRING:: > @Account Name: @ESTRING:ACC_NAME: @ Account Domain: WW002 > Changed Attributes: SAM Account Name: - Display Name: - User > Principal Name: - Home Directory: - Home Drive: - Script Path: > - Profile Path: - User Workstations: - Password Last Set: > @ESTRING:: @@ESTRING:: @ Account Expires: - Primary Group ID: - > AllowedToDelegateTo: - Old UAC Value: - New UAC Value: - User > Account Control: - User Parameters: - SID History: - Logon > Hours: - DNS Host Name: - Service Principal Names: - > Additional Information: Privileges: - (EventID 4742)</pattern> > <examples> > <example> > <test_message > program="Microsoft_Windows_security_auditing.[5784]">: Security > Microsoft Windows security auditing.: [Success Audit] A computer > account was changed. Subject: Security ID: S-1-5-7 Account > Name: ANONYMOUS LOGON Account Domain: NT AUTHORITY Logon ID: > 0x3e6 Computer Account That Was Changed: Security ID: > S-1-5-21-776561741-789336058-725345543-305444 Account Name: User1$ > Account Domain: TEST Changed Attributes: SAM Account Name: - > Display Name: - User Principal Name: - Home Directory: - Home > Drive: - Script Path: - Profile Path: - User Workstations: - > Password Last Set: 12/22/2011 3:38:32 AM Account Expires: - > Primary Group ID: - AllowedToDelegateTo: - Old UAC Value: - New > UAC Value: - User Account Control: - User Parameters: - SID > History: - Logon Hours: - DNS Host Name: - Service Principal > Names: - Additional Information: Privileges: - (EventID > 4742)</test_message> > <test_value > name="ACC_NAME">User1$</test_value> > </example> > </examples> > </rule> > </ruleset> > > Then you can test it more easily like this: > pdbtool test patterndb.xml > > On Thu, Dec 22, 2011 at 8:04 AM, Balazs Scheidler <bazsi(a)balabit.hu> > wrote: > > On Thu, 2011-12-22 at 14:31 +0530, Anup Shetty wrote: > >> Nope, no luck yet. Still blanks being spit out. > >> > >> > >> Here's the exact extract of the pattern matching and the log: > >> > >> > >> Pattern String > >> --------------------------- > >> > >> > >> @ESTRING:user::@ Security Microsoft Windows security auditing.: > >> [Success Audit] A computer account was changed. ? ?Subject: ? Security > >> ID: ?S-1-5-7 ? Account Name: ?ANONYMOUS LOGON ? Account Domain: ?NT > >> AUTHORITY ? Logon ID: ?0x3e6 ? ?Computer Account That Was Changed: > >> Security ID: ?@ESTRING:: ?@Account Name: ? @ESTRING:ACC_NAME: @ > >> Account Domain: ?WW002 ? ?Changed Attributes: ? SAM Account Name: - > >> Display Name: ?- ? User Principal Name: - ? Home Directory: ?- ? Home > >> Drive: ?- ? Script Path: ?- ? Profile Path: ?- ? User Workstations: - > >> Password Last Set: @ESTRING:: @@ESTRING:: @ ? Account Expires: ?- > >> Primary Group ID: - ? AllowedToDelegateTo: - ? Old UAC Value: ?- ? New > >> UAC Value: ?- ? User Account Control: - ? User Parameters: - ? SID > >> History: ?- ? Logon Hours: ?- ? DNS Host Name: ?- ? Service Principal > >> Names: - ? ?Additional Information: ? Privileges: ?- (EventID 4742) > >> > >> > >> Log > >> ------------------ > >> > >> > >> Dec 22 03:38:32 Server.zoom11.test.net > >> Microsoft_Windows_security_auditing.[5784]: : Security Microsoft > >> Windows security auditing.: [Success Audit] A computer account was > >> changed. ? ?Subject: ? Security ID: ?S-1-5-7 ? Account Name: > >> ?ANONYMOUS LOGON ? Account Domain: ?NT AUTHORITY ? Logon ID: ?0x3e6 > >> ?Computer Account That Was Changed: ? Security ID: > >> ?S-1-5-21-776561741-789336058-725345543-305444 ? Account Name: ?User1$ > >> Account Domain: ?TEST ? ?Changed Attributes: ? SAM Account Name: - > >> Display Name: ?- ? User Principal Name: - ? Home Directory: ?- ? Home > >> Drive: ?- ? Script Path: ?- ? Profile Path: ?- ? User Workstations: - > >> Password Last Set: 12/22/2011 3:38:32 AM ? Account Expires: ?- > >> Primary Group ID: - ? AllowedToDelegateTo: - ? Old UAC Value: ?- ? New > >> UAC Value: ?- ? User Account Control: - ? User Parameters: - ? SID > >> History: ?- ? Logon Hours: ?- ? DNS Host Name: ?- ? Service Principal > >> Names: - ? ?Additional Information: ? Privileges: ?- (EventID 4742) > >> > >> > > "pdbtool match" can be used to test patterns. > > > > pdbtool patch -p <path to xml file> -P '<appname>' -M '<msg>' --debug > --color-out > > > > This even colours the output so that the partial matches can be > > recognized. This is the best way to troubleshoot patterns. > > > > -- > > Bazsi > > > > > -- Thanks and regards, Anup

2 1

How to split logs in the default folders?
by tokie＠tiscali.it 23 Dec '11

23 Dec '11

Hi, I'm not yet able to store my log by ip address. Can someone help me? How can I redirect everything that comes from a unique ip address (from LAN) to a folder I created? eg.: 192.168.1.1 (host1) == logs ==> 192.168.1.2 (syslog-ng-server )--->/ var/log/hosts/192.168.1.1 Tks and merry xmas!!! Tokie E' nata indoona: chiama, videochiama e messaggia Gratis. Scarica indoona per iPhone, Android e PC: http://www.indoona.com/

2 1