- syslog-ng - lists.balabit.hu

unofficial syslog-ng 3.7.1 Debian packages
by Budai, László 17 Aug '15

17 Aug '15

Hi, I've just created my unofficial OBS repo[1] for syslog-ng 3.7.1 series (current version: 3.7.1-1 ). List of supported OSs (i386/amd64): * Debian 7.0 * Debian 8.0 * Ubuntu 12.04 * Ubuntu 14.04 * Ubuntu 14.10 * Ubuntu 15.04 Install ------- example: Debian 8.0 1. get release key wget -qO - http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… | sudo apt-key add - 2. add repo to APT sources eg.: /etc/apt/sources.list.d/syslog-ng-obs.list deb http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… ./ Then `apt-get update` and `apt-get install syslog-ng-core=3.7.1-1` You can replace Debian_8.0 to * Debian_7.0 * xUbuntu_12.04, * xUbuntu_14.04 * xUbuntu_14.10 * xUbuntu_15.04 (`x` before Ubuntu is not a typo ;-) ) Available packages: * syslog-ng-core * syslog-ng-dbg * syslog-ng-dev * syslog-ng-mod-stomp * syslog-ng-mod-amqp * syslog-ng-mod-elastic * syslog-ng-mod-geoip * syslog-ng-mod-graphite * syslog-ng-mod-hdfs * syslog-ng-mod-http * syslog-ng-mod-java * syslog-ng-mod-java-common-lib * syslog-ng-mod-journal * syslog-ng-mod-json * syslog-ng-mod-kafka * syslog-ng-mod-mongodb * syslog-ng-mod-python * syslog-ng-mod-redis * syslog-ng-mod-riemann * syslog-ng-mod-smtp * syslog-ng-mod-sql * syslog-ng-mod-stomp Notes to the java destinations ------------------------------ About the ``` Error opening plugin module; module='mod-java', error='libjvm.so: cannot open shared object file: No such file or directory' ``` problem. libmod-java.so is linked to the libjvm.so library. If your java installation doesn't create a symlink in /usr/lib directory to libjvm.so then you have to modify your ld.so.conf or expand your LD_LIBRARY_PATH with the containing library before starting syslog-ng. eg.: LD_LIBRARY_PATH=/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server:$LD_LIBRARY_PATH Links -------- [1] https://build.opensuse.org/project/show/home:laszlo_budai:syslog-ng [2] https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.7.1 regards, Laszlo Budai

1 0

Error in pattern ?
by Thanh Dat 17 Aug '15

17 Aug '15

2 2

There is only last field of host ip.
by Jacek Drewniak 11 Aug '15

11 Aug '15

Hi, I have problem, the host field contains only last field of client ip. For example if my client ip is 192.168.1.234 on server in host field I only have 234. There is my server config: http://pastebin.com/hwgNXCQK How to get whole ip? -- *Jacek Drewniak* R&D *email*: jacek.drewniak(a)oort.in *mobile*: *+**48 696 151 670* *website*: www.oort.in AWARDS Bluetooth Breakthrough Award Finalist CES 2015 Envisioneering Innovation & Design Award Winner Tech Trailblazers Awards Winner Most exciting company at Bluetooth Media Event in New York 2014 Polish Agency for Enterprise Development Award Winner

1 0

Re: [syslog-ng] Log rotation in Syslog-ng
by jrhendri＠roadrunner.com 11 Aug '15

11 Aug '15

Agree about doing rotation separately. I create the logs by date (using macros within syslog-ng destinations) then run a couple cron jobs to manage retention. You should tweak this to your own preferences: Here are some pertinent options: options { check_hostname(yes); # check if the hostname contains valid characters use_dns(no); # do not resolve names for speed dns_cache(no); # no DNS cache since we do not resolve names keep_hostname(yes); # keep hostnames to enable related macros chain_hostnames(no); # do not track / forward syslog forwarder chain # options related to file and directories dir_owner("root"); dir_group("logadmin"); owner("root"); group("logadmin"); perm(0640); dir_perm(0750); create_dirs(yes); }; and an example destination: destination d_separatedbyhosts { file("/data/syslog-ng/$YEAR/$MONTH/$DAY/$HOST_FROM/$HOST_FROM.$FACILITY.$PRIORITY.$YEAR.$MONTH.$DAY"); }; This way each filename uniquely identifies the source and date (in case you need to send copies somewhere, they aren't all named "messages"...) This is an example command that compresses after 1 day (run nightly) /bin/find /data/syslog-ng -type f -mtime +1 ! -name \*.gz -exec /bin/gzip --best {} \; And this deletes after 90 days (also run nightly) /bin/find /data/syslog-ng -type f -name \*.gz -mtime +90 -exec /bin/rm {} \; Good luck, Jim ---- Justin Kala <justinkala(a)gmail.com> wrote: > Can we control the log retention/rotation in syslog-ng.conf .Syslog-ng > server version is 3.4 OSE Solaris 10. > > If so, how do we do it ?? Highly appreciate if you can share the examples > /links > -- > Kaladhar

1 0

Log rotation in Syslog-ng
by Justin Kala 10 Aug '15

10 Aug '15

Can we control the log retention/rotation in syslog-ng.conf .Syslog-ng server version is 3.4 OSE Solaris 10. If so, how do we do it ?? Highly appreciate if you can share the examples /links -- Kaladhar

2 1

Issue with filtering messages and sending them
by Cosmin Neagu 10 Aug '15

10 Aug '15

Hello everyone, I have an issue with syslog-ng (detailed below) and I tend to incline is related to syslog-ng, an I hope will find some answers here. I set up a "solution" so that when someone fails to login to a ssh linux server, I receive an email with info about that. The idea is like this: Syslog-ng should look for any messages containing info about login failures, and when it sees those messages, it should pass them to a little script that emails them. Syslog-ng relevant config for that: filter f_ssh {program (sshd) and match("Failed password" value("MESSAGE")); }; destination d_sshalert { program("/home/cosmin/sshalert.sh"); }; #this is the script that is sendind the emails destination d_sshfile { file("/home/cosmin/LOGS/sshdfailed.log"); }; log { source(s_src); filter(f_ssh); destination(d_sshfile);}; #log { source(s_src); filter(f_ssh); destination(d_sshalert); flags(final); }; log { source(s_src); filter(f_ssh); destination(d_sshalert);}; And the script that is sending the emails: cosmin@srv:~$ cat sshalert.sh #!/bin/bash email="someone(a)gmail.com" while read event; do echo -e "Subject:*** SSH failed attempt on domain.ro ***\nFrom:SSH Watcher <admin(a)domain.ro>\nTo:someone@gmail.com\n\n####### WARNING #######\n\nA failed SSH attempt has been logged:\n${event}\n\n*** Required actions***\n1. Check ip owner and location:\nEx: #curl ipinfo.io/X.Y.Z.T\n2. Ban the source IP address in iptables.rules\n***********************\n\nFor any questions contact: admin(a)domain.ro\nHave a nice day\n" | /usr/sbin/sendmail -f admin(a)domain.ro ${email} done Everything is working as expected, I receive mails like this when failed attempts exists: ####### WARNING ####### A failed SSH attempt has been logged: *Aug 6 15:49:47 srv sshd[18236]: Failed password for someone from 173.XX.220.XX port 59004 ssh2* *** Required actions*** 1. Check ip owner and location: Ex: #curl ipinfo.io/X.Y.Z.T 2. Ban the source IP address in iptables.rules *********************** For any questions contact:admin@domain.ro Have a nice day Please not that the log inserted is the right one: Aug 6 15:49:47 srv sshd[18236]: Failed password for someone from 173.XX.220.XX port 59004 ssh2 My problem is that form time to time (did not manage to discover a time pattern), I also receive some strange mails like the following: ####### WARNING ####### A failed SSH attempt has been logged: *Aug 6 16:09:47 srv -- MARK --* *** Required actions*** 1. Check ip owner and location: Ex: #curl ipinfo.io/X.Y.Z.T 2. Ban the source IP address in iptables.rules *********************** For any questions contact:admin@domain.ro Have a nice day As you can see, the log inserted in the mail is strange: *Aug 6 16:09:47 srv -- MARK --*. I assume that somehow, syslog-ng is the one that is sending this log to my script which is why I configured another destination to a local file to test my hypothesis (d_sshfile) but the message is not inserted in that local file Now, my question for you is: is there a way to determine if syslog-ng is the one responsible for sending that strange message and why? Am I doing something wrong with syslog-ng config (maybe the filter is not right?) Any help would be appreciated. Thanks -- Best Regards Cosmin Neagu

3 5

cloudinit
by Amar Johal 09 Aug '15

09 Aug '15

Hi, We are moving from rsyslog to syslog-ng, we know that rsyslog as /etc/rsyslog.d/21-cloudinit.conf for cloud init logs, once we remove rsyslog for syslog-ng what would happen to our cloudinit logs? Thanks!

3 2

3.7 rpm-s from git HEAD (without Java support)
by Czanik, Péter 08 Aug '15

08 Aug '15

Hi, I just updated my 3.7 packages to git head. Along the way support for key value pairs was added and support for Java was removed. Compiling Java modules requires gradle, which is not available for openSUSE / SLES12 as a package. Gradle is available for Fedora22, but compiling java modules fails. Unfortunately compiling Java support independent of Java modules is not possible right now... openSUSE / SLES12: https://build.opensuse.org/project/show/home:czanik:syslog-ng37 Fedora / EPEL7: https://copr.fedoraproject.org/coprs/czanik/syslog-ng37/ Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

Re: [syslog-ng] Failed to compile syslog-ng-incubator with 3.7.0beta2
by Jacek Drewniak 05 Aug '15

05 Aug '15

Ok, I understand, thank You. But now I have other compilation error: *(...)* Download https://repo1.maven.org/maven2/org/apache/directory/api/api-asn1-api/1.0.0-… Download https://repo1.maven.org/maven2/org/apache/directory/api/api-util/1.0.0-M20/… Download https://repo1.maven.org/maven2/javax/xml/stream/stax-api/1.0-2/stax-api-1.0… Download https://repo1.maven.org/maven2/javax/activation/activation/1.1/activation-1… Download https://repo1.maven.org/maven2/org/apache/httpcomponents/httpclient/4.2.5/h… :hdfs:processResources UP-TO-DATE :hdfs:classes :hdfs:jar :hdfs:assemble :hdfs:compileTestJava UP-TO-DATE :hdfs:processTestResources UP-TO-DATE :hdfs:testClasses UP-TO-DATE :hdfs:test UP-TO-DATE :hdfs:check UP-TO-DATE :hdfs:build :http:compileJava *FAILURE*: Build failed with an exception. * What went wrong: Could not resolve all dependencies for configuration ':http:compile'. *> Could not find :syslog-ng-common:.* Searched in the following locations: file:/home/jacek/syslog-ng-master/modules/java/syslog-ng-core/libs/syslog-ng-common-.jar file:/home/jacek/syslog-ng-master/modules/java/syslog-ng-core/libs/syslog-ng-common.jar file:/home/jacek/syslog-ng-master/modules/java-modules/common/http/libs/syslog-ng-common-.jar file:/home/jacek/syslog-ng-master/modules/java-modules/common/http/libs/syslog-ng-common.jar file:/usr/lib/syslog-ng-java-module-dependency-jars/jars/syslog-ng-common-.jar file:/usr/lib/syslog-ng-java-module-dependency-jars/jars/syslog-ng-common.jar Required by: java-modules:http:unspecified * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. *BUILD FAILED* Total time: 46.271 secs make[2]: *** [java-modules] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 -- *Jacek Drewniak* R&D *email*: jacek.drewniak(a)oort.in *mobile*: *+**48 696 151 670* *website*: www.oort.in AWARDS Bluetooth Breakthrough Award Finalist CES 2015 Envisioneering Innovation & Design Award Winner Tech Trailblazers Awards Winner Most exciting company at Bluetooth Media Event in New York 2014 Polish Agency for Enterprise Development Award Winner 2015-08-05 14:30 GMT+02:00 Fabien Wernli <wernli(a)in2p3.fr>: > On Wed, Aug 05, 2015 at 02:01:15PM +0200, Jacek Drewniak wrote: > > Yes I have oracle-jave8 and grandle2 but still elasticsearch doesnt > appear > > in configure summary. > > > > Here I paste the configuration http://pastie.org/10331217 > > it won't appear in config.log > it appears as: > > java : yes > >

2 1

Failed to compile syslog-ng-incubator with 3.7.0beta2
by Jacek Drewniak 05 Aug '15

05 Aug '15

Hi, I would like to compile incubator to use elasticsearch destination. Unfortunately there is an error during compilation. :( syslog-ng Incubator 0.4.1 configured ------------------------------------ Debugging: no Modules: basicfuncs-plus yes kafka yes logmongource: yes lua: yes (lua5.1) monitor-source: yes (lua5.1) perl: no python: yes (python) rss yes trigger-source yes tfgetent yes zmq no grok-parser no date-parser yes java yes jacek@uwbTestField:~/syslog-ng-incubator$ make /usr/local/share/tools/lex-rules.am:14: warning: overriding commands for target `.y.c' make: warning: ignoring old commands for target `.y.c' GEN modules/trigger-source/trigger-source-grammar.y YACC modules/trigger-source/trigger-source-grammar.c GEN modules/monitor-source/monitor-source-grammar.y YACC modules/monitor-source/monitor-source-grammar.c GEN modules/rss/rss-grammar.y YACC modules/rss/rss-grammar.c GEN modules/lua/lua-grammar.y YACC modules/lua/lua-grammar.c GEN modules/python/python-grammar.y YACC modules/python/python-grammar.c GEN modules/kafka/kafka-grammar.y YACC modules/kafka/kafka-grammar.c GEN modules/zmq/zmq-grammar.y YACC modules/zmq/zmq-grammar.c GEN modules/java/native/java-grammar.y YACC modules/java/native/java-grammar.c GEN modules/java/org/syslog_ng/InternalMessageSender.class GEN modules/java/org/syslog_ng/SyslogNgClassLoader.class GEN modules/java/org/syslog_ng/LogMessage.class GEN modules/java/org/syslog_ng/LogTemplate.class GEN modules/java/org/syslog_ng/LogPipe.class GEN modules/java/org/syslog_ng/LogDestination.class GEN modules/java/org/syslog_ng/TextLogDestination.class GEN modules/java/org/syslog_ng/StructuredLogDestination.class GEN modules/java/org/syslog_ng/DummyTextDestination.class GEN modules/java/org/syslog_ng/DummyStructuredDestination.class GEN modules/java/LogDestination.h GEN modules/java/LogMessage.h GEN modules/java/LogTemplate.h GEN modules/java/InternalMessageSender.h GEN modules/java/SyslogNg.jar added manifest adding: org/(in = 0) (out= 0)(stored 0%) adding: org/syslog_ng/(in = 0) (out= 0)(stored 0%) adding: org/syslog_ng/LogPipe.class(in = 1461) (out= 825)(deflated 43%) adding: org/syslog_ng/SyslogNgClassLoader.class(in = 4154) (out= 2224)(deflated 46%) adding: org/syslog_ng/DummyTextDestination.class(in = 1273) (out= 675)(deflated 46%) adding: org/syslog_ng/LogMessage.class(in = 576) (out= 353)(deflated 38%) adding: org/syslog_ng/DummyStructuredDestination.class(in = 1692) (out= 905)(deflated 46%) adding: org/syslog_ng/SyslogNgClassLoader$1.class(in = 1305) (out= 648)(deflated 50%) adding: org/syslog_ng/TextLogDestination.class(in = 491) (out= 334)(deflated 31%) adding: org/syslog_ng/InternalMessageSender.class(in = 875) (out= 448)(deflated 48%) adding: org/syslog_ng/LogDestination.class(in = 908) (out= 516)(deflated 43%) adding: org/syslog_ng/LogTemplate.class(in = 785) (out= 477)(deflated 39%) adding: org/syslog_ng/StructuredLogDestination.class(in = 649) (out= 406)(deflated 37%) GEN modules/date/date-parser-grammar.y YACC modules/date/date-parser-grammar.c make all-am make[1]: Entering directory `/home/jacek/syslog-ng-incubator' /usr/local/share/tools/lex-rules.am:14: warning: overriding commands for target `.y.c' make[1]: warning: ignoring old commands for target `.y.c' CC modules/trigger-source/modules_trigger_source_libtrigger_source_la-trigger-source-grammar.lo CC modules/trigger-source/modules_trigger_source_libtrigger_source_la-trigger-source.lo modules/trigger-source/trigger-source.c: In function 'trigger_source_new': modules/trigger-source/trigger-source.c:167:27: error: too few arguments to function 'log_source_set_options' owner->super.super.id, NULL, FALSE, FALSE); ^ In file included from modules/trigger-source/trigger-source.h:28:0, from modules/trigger-source/trigger-source.c:24: /usr/local/include/syslog-ng/logsource.h:97:6: note: declared here void log_source_set_options(LogSource *self, LogSourceOptions *options, gint stats_level, gint stats_source, const gchar *stats_id, const gchar *stats_instance, gboolean threaded, gboolean pos_tracked, LogExprNode *expr_node); ^ make[1]: *** [modules/trigger-source/modules_trigger_source_libtrigger_source_la-trigger-source.lo] Error 1 make[1]: Leaving directory `/home/jacek/syslog-ng-incubator' make: *** [all] Error 2 -- *Jacek Drewniak* R&D *email*: jacek.drewniak(a)oort.in *mobile*: *+**48 696 151 670* *website*: www.oort.in AWARDS Bluetooth Breakthrough Award Finalist CES 2015 Envisioneering Innovation & Design Award Winner Tech Trailblazers Awards Winner Most exciting company at Bluetooth Media Event in New York 2014 Polish Agency for Enterprise Development Award Winner

2 4