- syslog-ng - lists.balabit.hu

documentation missing sql options
by Patrick Hemmer 10 Jan '12

10 Jan '12

The documentation (at least http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-3.3-guid… <http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-3.3-guid…>) is missing a couple of options for the sql destination driver * session-statements * retries Was just going through and noticed cause I need to use the session-statements one. <http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-3.3-guid…>

1 1

set tag in log statement
by Patrick Hemmer 10 Jan '12

10 Jan '12

Would it be possible to set tags on a message when it is matched by a `log {}` statement? This would be useful for complex log configurations so you could filter out messages that were logged by previous `log {}` statements without having to write really long `filter {}`s. I'm not sure if this is possible or not. Would depend on the order of operations, like if a filter block in a latter log statement is evaluated at the same time as an earlier filter block, or if its evaluated when it reaches the `log {}`. Such a config would look like filter f_a { facility(mail); }; log { source(s_local); filter(f_a); destination(d_a); tags("tag1"); }; filter f_b { facility(local0); }; log { source(s_local); filter(f_b); destination(d_b); tags("tag1"); }; filter f_c { not tags("tag1"); }; log { source(s_local); filter(f_c); destination (d_c); };

4 8

Escape slashes etc in macro variables ($PROGRAM)
by Göran Sandahl 10 Jan '12

10 Jan '12

Howdy, Is there a option for escaping macro variables such as $PROGRAM? I've seen a couple of discussions about this on the list but they never seem to come to a conclusion. Case in point: I'm creating an output file macro like this: file("/var/log/$SOURCEIP.$PROGRAM.log" ... and Syslog-ng failes to write the file whenever $PROGRAM looks similar to "postfix/quemgr" since its then trying to "create" the directory postfix due to the slash. Of course, thats not what I want. Can this be circumvented somehow? Cheers! Goran

5 5

Re: [syslog-ng] Syslog-ng 3.2 connection timeout with firewall
by André Larose 10 Jan '12

10 Jan '12

Hi, I found the timeout issue. It turns out that the keep_alive is not generated by Syslog-NG but by the OS. I adjusted the following values: tcp_keepalive_time tcp_keepalive_intvl tcp_keepalive_probes First in the live Kernel with the command : sysctl -w net.ipv4.tcp_keepalive_time=180 net.ipv4.tcp_keepalive_intvl=180 net.ipv4.tcp_keepalive_probes=9 Then I made it permanent with adding to /etc/sysctl.conf: cat >>/etc/sysctl.conf <<EOF # # Keepalive parameters for the syslog-ng # net.ipv4.tcp_keepalive_time = 180 net.ipv4.tcp_keepalive_intvl = 180 net.ipv4.tcp_keepalive_probes = 9 EOF You can verify your work is done with the following command: sysctl net.ipv4.tcp_keepalive_time net.ipv4.tcp_keepalive_intvl net.ipv4.tcp_keepalive_probes net.ipv4.tcp_keepalive_time = 180 net.ipv4.tcp_keepalive_intvl = 180 net.ipv4.tcp_keepalive_probes = 9 I found the information from this URL: http://tldp.org/HOWTO/TCP-Keepalive-HOWTO/usingkeepalive.html Maybe it could be mentioned in the documentation that the so_keep-alive option can be tweaked at the kernel level. Regards, ------------------------------ Message: 8 Date: Tue, 3 Jan 2012 17:46:06 -0500 From: Andr? Larose <andre.larose(a)telus.com<mailto:andre.larose@telus.com>> Subject: [syslog-ng] Syslog-ng 3.2 connection timeout with firewall To: "syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>" <syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>> Message-ID: <BDB004F99074254BBCEE3AB94F31FDF5435439F706(a)WP40066.corp.ads<mailto:BDB004F99074254BBCEE3AB94F31FDF5435439F706@WP40066.corp.ads>> Content-Type: text/plain; charset="iso-8859-1" Hi, I have two syslog-ng 3.2, one client and one server. The two are separated by a firewall and a load balancer. I noticed that after some inactivity I was not able to receive logs from my client. So I started some tcpdump on both servers to check the traffic. From what I see the firewall will close the connection after some time, so when the client sends traffic it gets dropped. I added the keep-alive(yes) and so_keep-alive(yes) and the mark_freq(60) to the configs. But I still do not see keepalive packets with tcpdump. Am I missing other parameters to have "keepalive" traffic sent ? Thank you in advance.

1 0

program destination and init??
by Matt Zagrabelny 09 Jan '12

09 Jan '12

Hi, I've got a question regarding the program destination and how/when it spawns the program. I've got a few program destination directives in my syslog-ng config. When I start the syslog-ng daemon, the program's (of the program destinations) parents are the syslog-ng daemon. After some amount of time (perhaps 20 or 30 hours) I get more programs spawned. Some of the programs have parents that are 'init' (PID = 1). One of my programs is 'vpndb' - see the grep output for an example: $ ps -ef | grep vpndb root 610 1 41 Nov23 ? 2-01:29:55 perl /usr/bin/vpndb root 4036 605 0 06:44 ? 00:00:00 /bin/sh -c /usr/bin/vpndb root 4039 4036 0 06:44 ? 00:00:00 perl /usr/bin/vpndb root 5231 1 26 Nov24 ? 1-02:09:12 perl /usr/bin/vpndb root 5322 1 1 Nov27 ? 00:21:26 perl /usr/bin/vpndb root 5658 1381 0 08:33 pts/4 00:00:00 grep vpndb root 16167 1 12 Nov26 ? 06:19:10 perl /usr/bin/vpndb root 26542 1 19 Nov25 ? 14:12:19 perl /usr/bin/vpndb Above there are 5 vpndb programs with a parent of PID = 1. One of the vpndb's belongs to syslog-ng. Two of my other program destinations are "ssh" commands. They exhibit the same symptoms (multiple programs running with parents of PID = 1). Hence I don't think this problem is related to the specific program being executed. Does anyone have any idea why I am getting multiple programs spawned? (I presume that syslog-ng is spawning these programs.) I do have cron jobs that will 'reload' the syslog-ng daemon. Thanks for any hints. -matt zagrabelny

5 11

[PATCH] logrewrite: Correctly initialize the conditional rules.
by Gergely Nagy 09 Jan '12

09 Jan '12

In case where the condition of a rewrite used a filter reference instead of inlining the filter itself, the filter expression was never initialized. This resulted in the condition always evaluating to not-match, making the following construct completely useless: rewrite("new-value", value("MESSAGE"), condition(filter(f_my_filter))) The fix is to introduce a new function, log_rewrite_init_method(), which will initialize the condition filter if it exists, and has an init method. Then we bind this method to LogPipe's init callback during log_rewrite_init(). This way, when initializing the rewrite pipe, the condition will get initialised too. Reported-by: Thomas Wollner <tw(a)wollner-net.de> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- lib/logrewrite.c | 12 ++++++++++++ 1 files changed, 12 insertions(+), 0 deletions(-) diff --git a/lib/logrewrite.c b/lib/logrewrite.c index a578439..3ff245a 100644 --- a/lib/logrewrite.c +++ b/lib/logrewrite.c @@ -75,6 +75,17 @@ log_rewrite_free_method(LogPipe *s) log_process_pipe_free_method(s); } +static gboolean +log_rewrite_init_method(LogPipe *s) +{ + LogRewrite *self = (LogRewrite *) s; + + if (self->condition && self->condition->init) + self->condition->init(self->condition, log_pipe_get_config(s)); + + return TRUE; +} + static void log_rewrite_init(LogRewrite *self) { @@ -83,6 +94,7 @@ log_rewrite_init(LogRewrite *self) self->super.super.flags |= PIF_CLONE; self->super.super.free_fn = log_rewrite_free_method; self->super.super.queue = log_rewrite_queue; + self->super.super.init = log_rewrite_init_method; self->value_handle = LM_V_MESSAGE; } -- 1.7.7.3

2 1

Error while starting Syslog-ng
by Kareep Navas 03 Jan '12

03 Jan '12

Error creating AF_INET socket (Error 0) Error initializing configuration, exiting.

2 1

What is the latest version of Syslog-ng?
by Kareep Navas 03 Jan '12

03 Jan '12

Hi All, We are currently using syslog-ng 1.4.14 and i would like to use & know the latest version of this syslog-ng? What is the purpose of the latest version? How to download the latest version? What are all the dependencies? Please direct me.. Thank you,

2 1

missing IP information at the message part
by Oğuz Yarımtepe 01 Jan '12

01 Jan '12

I have syslog-ng and logzilla 2.9.9g installed on a Debian squeeze machine. I can see the logs at the web interface. I configured syslog-ng to collect apache error and access logs. The problem is that i don't see IP information at the message part, on web interface. For ex, one of the machine that is sending apache logs is "www" machine. The syslog-ng configuration is as below: # All messages send to a remote site # #log { source(s_src); destination(d_net); }; #manual edit options { log_msg_size(8192); }; ####### # sources # ####### source s_apache_access { # unix-stream("/var/log/httpd/apache_log.socket" # max-connections(512) # keep-alive(yes)); file ("/var/log/apache2/access.log" flags(no-parse)); }; source s_apache_error { # unix-stream("/var/log/httpd/apache_log.socket" # max-connections(512) # keep-alive(yes)); file ("/var/log/apache2/error.log" flags(no-parse)); }; ################ # destinations # ################ destination d_custom_access { file("/var/log/apache2/custom_access.log"); }; destination d_custom_error { file("/var/log/apache2/custom_error.log"); }; destination d_apache_tls { tcp("192.168.1.145" port(6514) tls(ca_dir("/etc/syslog-ng/etc/ca.d") key_file("/etc/syslog-ng/etc/key.d/debian4key.pem") cert_file("/etc/syslog-ng/etc/cert.d/debian4cert.pem"))); }; ########### # filters # ########### filter f_apache_access { message("GET|POST"); }; filter f_apache_error { message("error"); }; log { source(s_apache_access); # filter(f_apache_access); destination(d_apache_tls); }; log { source(s_apache_error); # filter(f_apache_access); destination(d_apache_tls); }; and # tail -n1 /var/log/apache2/access.log 78.185.240.170 - - [01/Jan/2012:13:35:06 +0200] "GET /images/rand/randpic11.jpg HTTP/1.1" 200 45670 "http://www.comu.edu.tr/" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.63 Safari/535.7" www:/home/oguz# For the www machine i don't see the 78.185.240.170 information at the web ui, but the rest of the line starting with -- Thogh another machine, "reverse", is sending apache logs with and additional field and i can see its log at the web ui without any missing information. The reverse has the same configuration except from the key information. # tail -n1 /var/log/apache2/other_vhosts_access.log www.beeseurope.eu:80 178.154.160.29 - - [01/Jan/2012:13:36:46 +0200] "GET /mod/forum/discuss.php?d=55&parent=88 HTTP/1.1" 200 34557 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)" At the web interface, I can see the IP information also at the message part. My syslog-ng configuration at the server is as below: options { long_hostnames(off); # doesn't actually help on Solaris, log(3) truncates at 1024 chars log_msg_size(8192); # buffer just a little for performance flush_lines(16384); # memory is cheap, buffer messages unable to write (like to loghost) log_fifo_size(16384); # Hosts we don't want syslog from #bad_hostname("^(ctld.|cmd|tmd|last)$"); # The time to wait before a dead connection is reestablished (seconds) time_reopen(10); #Use DNS so that our good names are used, not hostnames use_dns(yes); dns_cache(yes); #Use the whole DNS name use_fqdn(yes); keep_hostname(yes); chain_hostnames(no); #Read permission for everyone perm(0644); # The default action of syslog-ng 1.6.0 is to log a STATS line # to the file every 10 minutes. That's pretty ugly after a while. # Change it to every 12 hours so you get a nice daily update of # # how many messages syslog-ng missed (0). # stats(43200); log_msg_size(8192); }; source s_apache { tcp(ip(0.0.0.0) port(6514) tls( key_file("/etc/syslog-ng/etc/key.d/debian1key.pem") cert_file("/etc/syslog-ng/etc/cert.d/debian1cert.pem") ca_dir("/etc/syslog-ng/etc/ca.d"))); }; filter f_apache_access { message("GET|POST"); }; filter f_apache_error { message("error"); }; # Create destination to LogZilla destination d_logzilla { program("/opt/logzilla-2.9.9g/scripts/db_insert.pl" template("$HOST\t$FACILITY\t$PRIORITY\t$LEVEL\t$TAG\t$YEAR-$MONTH-$DAY\t$HOUR:$MIN:$SEC\t$PROGRAM\t$MSG\n") #template("$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC\t$HOST\t$PRI\t$PROGRAM\t$MSGONLY\n") ); }; # test purposes destination d_apache_test_access { file("/var/log/apache2/hosts/$HOST/$YEAR/$MONTH/$DAY/access.log" create_dirs(yes)); }; destination d_apache_test_error { file("/var/log/apache2/hosts/$HOST/$YEAR/$MONTH/$DAY/error.log" create_dirs(yes)); }; # Tell syslog-ng to log to our new destination log { source(s_apache); destination(d_logzilla); }; log { source(s_apache); filter(f_apache_error); destination(d_apache_test_error); }; log { source(s_apache); filter(f_apache_access); destination(d_apache_test_access); }; Any ides about how i can fix the missing IP information at the web ui of logzilla?

1 0

Re: [syslog-ng] syslog-ng Digest, Vol 80, Issue 33
by Anup Shetty 23 Dec '11

23 Dec '11

I cant execute those commands. Here's the error Unknown command Syntax: pdbtool <command> [options] Possible commands are: match Match a message against the pattern database dump Dump pattern datebase tree merge Merge pattern databases dictionary Dump pattern dictionary Version syslog-ng-premium-edition 3.2.1 Installer-Version: 3.2.1 > > > > > > ------------------------------ > > Message: 3 > Date: Thu, 22 Dec 2011 13:11:05 -0600 > From: Martin Holste <mcholste(a)gmail.com> > Subject: Re: [syslog-ng] Pattern matching. > To: "Syslog-ng users' and developers' mailing list" > <syslog-ng(a)lists.balabit.hu> > Message-ID: > <CANpnLHgau7bZrSP2ro0QY=a8ZcJZLyqJgAVegWufDuszOjuCMA(a)mail.gmail.com > > > Content-Type: text/plain; charset=ISO-8859-1 > > You can also include an example pattern as part of the actual rule like > this: > > <ruleset> > <program></program> > <rule id="2"> > <pattern>@ESTRING:user::@ Security Microsoft > Windows security auditing.: [Success Audit] A computer account was > changed. Subject: Security ID: S-1-5-7 Account Name: > ANONYMOUS LOGON Account Domain: NT AUTHORITY Logon ID: 0x3e6 > Computer Account That Was Changed: Security ID: @ESTRING:: > @Account Name: @ESTRING:ACC_NAME: @ Account Domain: WW002 > Changed Attributes: SAM Account Name: - Display Name: - User > Principal Name: - Home Directory: - Home Drive: - Script Path: > - Profile Path: - User Workstations: - Password Last Set: > @ESTRING:: @@ESTRING:: @ Account Expires: - Primary Group ID: - > AllowedToDelegateTo: - Old UAC Value: - New UAC Value: - User > Account Control: - User Parameters: - SID History: - Logon > Hours: - DNS Host Name: - Service Principal Names: - > Additional Information: Privileges: - (EventID 4742)</pattern> > <examples> > <example> > <test_message > program="Microsoft_Windows_security_auditing.[5784]">: Security > Microsoft Windows security auditing.: [Success Audit] A computer > account was changed. Subject: Security ID: S-1-5-7 Account > Name: ANONYMOUS LOGON Account Domain: NT AUTHORITY Logon ID: > 0x3e6 Computer Account That Was Changed: Security ID: > S-1-5-21-776561741-789336058-725345543-305444 Account Name: User1$ > Account Domain: TEST Changed Attributes: SAM Account Name: - > Display Name: - User Principal Name: - Home Directory: - Home > Drive: - Script Path: - Profile Path: - User Workstations: - > Password Last Set: 12/22/2011 3:38:32 AM Account Expires: - > Primary Group ID: - AllowedToDelegateTo: - Old UAC Value: - New > UAC Value: - User Account Control: - User Parameters: - SID > History: - Logon Hours: - DNS Host Name: - Service Principal > Names: - Additional Information: Privileges: - (EventID > 4742)</test_message> > <test_value > name="ACC_NAME">User1$</test_value> > </example> > </examples> > </rule> > </ruleset> > > Then you can test it more easily like this: > pdbtool test patterndb.xml > > On Thu, Dec 22, 2011 at 8:04 AM, Balazs Scheidler <bazsi(a)balabit.hu> > wrote: > > On Thu, 2011-12-22 at 14:31 +0530, Anup Shetty wrote: > >> Nope, no luck yet. Still blanks being spit out. > >> > >> > >> Here's the exact extract of the pattern matching and the log: > >> > >> > >> Pattern String > >> --------------------------- > >> > >> > >> @ESTRING:user::@ Security Microsoft Windows security auditing.: > >> [Success Audit] A computer account was changed. ? ?Subject: ? Security > >> ID: ?S-1-5-7 ? Account Name: ?ANONYMOUS LOGON ? Account Domain: ?NT > >> AUTHORITY ? Logon ID: ?0x3e6 ? ?Computer Account That Was Changed: > >> Security ID: ?@ESTRING:: ?@Account Name: ? @ESTRING:ACC_NAME: @ > >> Account Domain: ?WW002 ? ?Changed Attributes: ? SAM Account Name: - > >> Display Name: ?- ? User Principal Name: - ? Home Directory: ?- ? Home > >> Drive: ?- ? Script Path: ?- ? Profile Path: ?- ? User Workstations: - > >> Password Last Set: @ESTRING:: @@ESTRING:: @ ? Account Expires: ?- > >> Primary Group ID: - ? AllowedToDelegateTo: - ? Old UAC Value: ?- ? New > >> UAC Value: ?- ? User Account Control: - ? User Parameters: - ? SID > >> History: ?- ? Logon Hours: ?- ? DNS Host Name: ?- ? Service Principal > >> Names: - ? ?Additional Information: ? Privileges: ?- (EventID 4742) > >> > >> > >> Log > >> ------------------ > >> > >> > >> Dec 22 03:38:32 Server.zoom11.test.net > >> Microsoft_Windows_security_auditing.[5784]: : Security Microsoft > >> Windows security auditing.: [Success Audit] A computer account was > >> changed. ? ?Subject: ? Security ID: ?S-1-5-7 ? Account Name: > >> ?ANONYMOUS LOGON ? Account Domain: ?NT AUTHORITY ? Logon ID: ?0x3e6 > >> ?Computer Account That Was Changed: ? Security ID: > >> ?S-1-5-21-776561741-789336058-725345543-305444 ? Account Name: ?User1$ > >> Account Domain: ?TEST ? ?Changed Attributes: ? SAM Account Name: - > >> Display Name: ?- ? User Principal Name: - ? Home Directory: ?- ? Home > >> Drive: ?- ? Script Path: ?- ? Profile Path: ?- ? User Workstations: - > >> Password Last Set: 12/22/2011 3:38:32 AM ? Account Expires: ?- > >> Primary Group ID: - ? AllowedToDelegateTo: - ? Old UAC Value: ?- ? New > >> UAC Value: ?- ? User Account Control: - ? User Parameters: - ? SID > >> History: ?- ? Logon Hours: ?- ? DNS Host Name: ?- ? Service Principal > >> Names: - ? ?Additional Information: ? Privileges: ?- (EventID 4742) > >> > >> > > "pdbtool match" can be used to test patterns. > > > > pdbtool patch -p <path to xml file> -P '<appname>' -M '<msg>' --debug > --color-out > > > > This even colours the output so that the partial matches can be > > recognized. This is the best way to troubleshoot patterns. > > > > -- > > Bazsi > > > > > -- Thanks and regards, Anup

2 1