- syslog-ng - lists.balabit.hu

Issue with filtering messages and sending them
by Cosmin Neagu 10 Aug '15

10 Aug '15

Hello everyone, I have an issue with syslog-ng (detailed below) and I tend to incline is related to syslog-ng, an I hope will find some answers here. I set up a "solution" so that when someone fails to login to a ssh linux server, I receive an email with info about that. The idea is like this: Syslog-ng should look for any messages containing info about login failures, and when it sees those messages, it should pass them to a little script that emails them. Syslog-ng relevant config for that: filter f_ssh {program (sshd) and match("Failed password" value("MESSAGE")); }; destination d_sshalert { program("/home/cosmin/sshalert.sh"); }; #this is the script that is sendind the emails destination d_sshfile { file("/home/cosmin/LOGS/sshdfailed.log"); }; log { source(s_src); filter(f_ssh); destination(d_sshfile);}; #log { source(s_src); filter(f_ssh); destination(d_sshalert); flags(final); }; log { source(s_src); filter(f_ssh); destination(d_sshalert);}; And the script that is sending the emails: cosmin@srv:~$ cat sshalert.sh #!/bin/bash email="someone(a)gmail.com" while read event; do echo -e "Subject:*** SSH failed attempt on domain.ro ***\nFrom:SSH Watcher <admin(a)domain.ro>\nTo:someone@gmail.com\n\n####### WARNING #######\n\nA failed SSH attempt has been logged:\n${event}\n\n*** Required actions***\n1. Check ip owner and location:\nEx: #curl ipinfo.io/X.Y.Z.T\n2. Ban the source IP address in iptables.rules\n***********************\n\nFor any questions contact: admin(a)domain.ro\nHave a nice day\n" | /usr/sbin/sendmail -f admin(a)domain.ro ${email} done Everything is working as expected, I receive mails like this when failed attempts exists: ####### WARNING ####### A failed SSH attempt has been logged: *Aug 6 15:49:47 srv sshd[18236]: Failed password for someone from 173.XX.220.XX port 59004 ssh2* *** Required actions*** 1. Check ip owner and location: Ex: #curl ipinfo.io/X.Y.Z.T 2. Ban the source IP address in iptables.rules *********************** For any questions contact:admin@domain.ro Have a nice day Please not that the log inserted is the right one: Aug 6 15:49:47 srv sshd[18236]: Failed password for someone from 173.XX.220.XX port 59004 ssh2 My problem is that form time to time (did not manage to discover a time pattern), I also receive some strange mails like the following: ####### WARNING ####### A failed SSH attempt has been logged: *Aug 6 16:09:47 srv -- MARK --* *** Required actions*** 1. Check ip owner and location: Ex: #curl ipinfo.io/X.Y.Z.T 2. Ban the source IP address in iptables.rules *********************** For any questions contact:admin@domain.ro Have a nice day As you can see, the log inserted in the mail is strange: *Aug 6 16:09:47 srv -- MARK --*. I assume that somehow, syslog-ng is the one that is sending this log to my script which is why I configured another destination to a local file to test my hypothesis (d_sshfile) but the message is not inserted in that local file Now, my question for you is: is there a way to determine if syslog-ng is the one responsible for sending that strange message and why? Am I doing something wrong with syslog-ng config (maybe the filter is not right?) Any help would be appreciated. Thanks -- Best Regards Cosmin Neagu

3 5

cloudinit
by Amar Johal 09 Aug '15

09 Aug '15

Hi, We are moving from rsyslog to syslog-ng, we know that rsyslog as /etc/rsyslog.d/21-cloudinit.conf for cloud init logs, once we remove rsyslog for syslog-ng what would happen to our cloudinit logs? Thanks!

3 2

3.7 rpm-s from git HEAD (without Java support)
by Czanik, Péter 08 Aug '15

08 Aug '15

Hi, I just updated my 3.7 packages to git head. Along the way support for key value pairs was added and support for Java was removed. Compiling Java modules requires gradle, which is not available for openSUSE / SLES12 as a package. Gradle is available for Fedora22, but compiling java modules fails. Unfortunately compiling Java support independent of Java modules is not possible right now... openSUSE / SLES12: https://build.opensuse.org/project/show/home:czanik:syslog-ng37 Fedora / EPEL7: https://copr.fedoraproject.org/coprs/czanik/syslog-ng37/ Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

Re: [syslog-ng] Failed to compile syslog-ng-incubator with 3.7.0beta2
by Jacek Drewniak 05 Aug '15

05 Aug '15

Ok, I understand, thank You. But now I have other compilation error: *(...)* Download https://repo1.maven.org/maven2/org/apache/directory/api/api-asn1-api/1.0.0-… Download https://repo1.maven.org/maven2/org/apache/directory/api/api-util/1.0.0-M20/… Download https://repo1.maven.org/maven2/javax/xml/stream/stax-api/1.0-2/stax-api-1.0… Download https://repo1.maven.org/maven2/javax/activation/activation/1.1/activation-1… Download https://repo1.maven.org/maven2/org/apache/httpcomponents/httpclient/4.2.5/h… :hdfs:processResources UP-TO-DATE :hdfs:classes :hdfs:jar :hdfs:assemble :hdfs:compileTestJava UP-TO-DATE :hdfs:processTestResources UP-TO-DATE :hdfs:testClasses UP-TO-DATE :hdfs:test UP-TO-DATE :hdfs:check UP-TO-DATE :hdfs:build :http:compileJava *FAILURE*: Build failed with an exception. * What went wrong: Could not resolve all dependencies for configuration ':http:compile'. *> Could not find :syslog-ng-common:.* Searched in the following locations: file:/home/jacek/syslog-ng-master/modules/java/syslog-ng-core/libs/syslog-ng-common-.jar file:/home/jacek/syslog-ng-master/modules/java/syslog-ng-core/libs/syslog-ng-common.jar file:/home/jacek/syslog-ng-master/modules/java-modules/common/http/libs/syslog-ng-common-.jar file:/home/jacek/syslog-ng-master/modules/java-modules/common/http/libs/syslog-ng-common.jar file:/usr/lib/syslog-ng-java-module-dependency-jars/jars/syslog-ng-common-.jar file:/usr/lib/syslog-ng-java-module-dependency-jars/jars/syslog-ng-common.jar Required by: java-modules:http:unspecified * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. *BUILD FAILED* Total time: 46.271 secs make[2]: *** [java-modules] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 -- *Jacek Drewniak* R&D *email*: jacek.drewniak(a)oort.in *mobile*: *+**48 696 151 670* *website*: www.oort.in AWARDS Bluetooth Breakthrough Award Finalist CES 2015 Envisioneering Innovation & Design Award Winner Tech Trailblazers Awards Winner Most exciting company at Bluetooth Media Event in New York 2014 Polish Agency for Enterprise Development Award Winner 2015-08-05 14:30 GMT+02:00 Fabien Wernli <wernli(a)in2p3.fr>: > On Wed, Aug 05, 2015 at 02:01:15PM +0200, Jacek Drewniak wrote: > > Yes I have oracle-jave8 and grandle2 but still elasticsearch doesnt > appear > > in configure summary. > > > > Here I paste the configuration http://pastie.org/10331217 > > it won't appear in config.log > it appears as: > > java : yes > >

2 1

Failed to compile syslog-ng-incubator with 3.7.0beta2
by Jacek Drewniak 05 Aug '15

05 Aug '15

Hi, I would like to compile incubator to use elasticsearch destination. Unfortunately there is an error during compilation. :( syslog-ng Incubator 0.4.1 configured ------------------------------------ Debugging: no Modules: basicfuncs-plus yes kafka yes logmongource: yes lua: yes (lua5.1) monitor-source: yes (lua5.1) perl: no python: yes (python) rss yes trigger-source yes tfgetent yes zmq no grok-parser no date-parser yes java yes jacek@uwbTestField:~/syslog-ng-incubator$ make /usr/local/share/tools/lex-rules.am:14: warning: overriding commands for target `.y.c' make: warning: ignoring old commands for target `.y.c' GEN modules/trigger-source/trigger-source-grammar.y YACC modules/trigger-source/trigger-source-grammar.c GEN modules/monitor-source/monitor-source-grammar.y YACC modules/monitor-source/monitor-source-grammar.c GEN modules/rss/rss-grammar.y YACC modules/rss/rss-grammar.c GEN modules/lua/lua-grammar.y YACC modules/lua/lua-grammar.c GEN modules/python/python-grammar.y YACC modules/python/python-grammar.c GEN modules/kafka/kafka-grammar.y YACC modules/kafka/kafka-grammar.c GEN modules/zmq/zmq-grammar.y YACC modules/zmq/zmq-grammar.c GEN modules/java/native/java-grammar.y YACC modules/java/native/java-grammar.c GEN modules/java/org/syslog_ng/InternalMessageSender.class GEN modules/java/org/syslog_ng/SyslogNgClassLoader.class GEN modules/java/org/syslog_ng/LogMessage.class GEN modules/java/org/syslog_ng/LogTemplate.class GEN modules/java/org/syslog_ng/LogPipe.class GEN modules/java/org/syslog_ng/LogDestination.class GEN modules/java/org/syslog_ng/TextLogDestination.class GEN modules/java/org/syslog_ng/StructuredLogDestination.class GEN modules/java/org/syslog_ng/DummyTextDestination.class GEN modules/java/org/syslog_ng/DummyStructuredDestination.class GEN modules/java/LogDestination.h GEN modules/java/LogMessage.h GEN modules/java/LogTemplate.h GEN modules/java/InternalMessageSender.h GEN modules/java/SyslogNg.jar added manifest adding: org/(in = 0) (out= 0)(stored 0%) adding: org/syslog_ng/(in = 0) (out= 0)(stored 0%) adding: org/syslog_ng/LogPipe.class(in = 1461) (out= 825)(deflated 43%) adding: org/syslog_ng/SyslogNgClassLoader.class(in = 4154) (out= 2224)(deflated 46%) adding: org/syslog_ng/DummyTextDestination.class(in = 1273) (out= 675)(deflated 46%) adding: org/syslog_ng/LogMessage.class(in = 576) (out= 353)(deflated 38%) adding: org/syslog_ng/DummyStructuredDestination.class(in = 1692) (out= 905)(deflated 46%) adding: org/syslog_ng/SyslogNgClassLoader$1.class(in = 1305) (out= 648)(deflated 50%) adding: org/syslog_ng/TextLogDestination.class(in = 491) (out= 334)(deflated 31%) adding: org/syslog_ng/InternalMessageSender.class(in = 875) (out= 448)(deflated 48%) adding: org/syslog_ng/LogDestination.class(in = 908) (out= 516)(deflated 43%) adding: org/syslog_ng/LogTemplate.class(in = 785) (out= 477)(deflated 39%) adding: org/syslog_ng/StructuredLogDestination.class(in = 649) (out= 406)(deflated 37%) GEN modules/date/date-parser-grammar.y YACC modules/date/date-parser-grammar.c make all-am make[1]: Entering directory `/home/jacek/syslog-ng-incubator' /usr/local/share/tools/lex-rules.am:14: warning: overriding commands for target `.y.c' make[1]: warning: ignoring old commands for target `.y.c' CC modules/trigger-source/modules_trigger_source_libtrigger_source_la-trigger-source-grammar.lo CC modules/trigger-source/modules_trigger_source_libtrigger_source_la-trigger-source.lo modules/trigger-source/trigger-source.c: In function 'trigger_source_new': modules/trigger-source/trigger-source.c:167:27: error: too few arguments to function 'log_source_set_options' owner->super.super.id, NULL, FALSE, FALSE); ^ In file included from modules/trigger-source/trigger-source.h:28:0, from modules/trigger-source/trigger-source.c:24: /usr/local/include/syslog-ng/logsource.h:97:6: note: declared here void log_source_set_options(LogSource *self, LogSourceOptions *options, gint stats_level, gint stats_source, const gchar *stats_id, const gchar *stats_instance, gboolean threaded, gboolean pos_tracked, LogExprNode *expr_node); ^ make[1]: *** [modules/trigger-source/modules_trigger_source_libtrigger_source_la-trigger-source.lo] Error 1 make[1]: Leaving directory `/home/jacek/syslog-ng-incubator' make: *** [all] Error 2 -- *Jacek Drewniak* R&D *email*: jacek.drewniak(a)oort.in *mobile*: *+**48 696 151 670* *website*: www.oort.in AWARDS Bluetooth Breakthrough Award Finalist CES 2015 Envisioneering Innovation & Design Award Winner Tech Trailblazers Awards Winner Most exciting company at Bluetooth Media Event in New York 2014 Polish Agency for Enterprise Development Award Winner

2 4

Syslog-ng cross compile
by Jacek Drewniak 04 Aug '15

04 Aug '15

Hello, I am trying to cross compile syslog-ng with yocto distro. I have problem with --prefix. I would like to install syslog-ng to some custom folder and than copy all files into my board. But prefix hadrcode files path into syslog-ng. It is very hard to copy compilled files from build directory (without --prefix and make install, to not install into my desktop). It is possible to --prefix installation without hardcode paths to files? In the other hand I configure syslog-ng without json but config still needs it. I have to delete *cim* to go further. *Error parsing config, Error compiling template (Unknown template function "format-json") in /usr/local/share/include/scl/cim/template.conf at line 23, column 32:* * included from /usr/local/etc/scl.conf line 32, column 1* * included from /usr/local/etc/syslog-ng.conf line 8, column 1* *template-function "format-cim" "$(format-json --pair @timestamp='${R_ISODATE}' --pair @message='${MSG}' --key .cim.* --shift 5 --key _* --key .* --replace-prefix .=_ --key *.*)\n";* * ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^* -- *Jacek Drewniak* R&D *email*: jacek.drewniak(a)oort.in *mobile*: *+**48 696 151 670* *website*: www.oort.in AWARDS Bluetooth Breakthrough Award Finalist CES 2015 Envisioneering Innovation & Design Award Winner Tech Trailblazers Awards Winner Most exciting company at Bluetooth Media Event in New York 2014 Polish Agency for Enterprise Development Award Winner

3 4

csv-parser deactivate default_flags strip_whitespaces
by mona 30 Jul '15

30 Jul '15

Dear all, How do I deactivate in the csv-parsers that the default flag setting includes stripping whitespace in Line 37? https://github.com/balabit/syslog-ng/blob/syslog-ng-3.7.0beta2/modules/csvp… <https://github.com/balabit/syslog-ng/blob/syslog-ng-3.7.0beta2/modules/csvp…> Kind regards, Mona

1 1

Errors compiling syslog-ng 3.6.4 in CentOS 7.1
by C. L. Martinez 29 Jul '15

29 Jul '15

Hi all, I am trying to update my syslog-ng 3.5.x release to syslog-ng 3.6.4. When I run autogen.sh, the following error appears: [root@c071pubwww syslog-ng-syslog-ng-3.6.4]# ./autogen.sh Running autogen in 'lib/ivykis'... Don't know how to bootstrap submodule 'lib/ivykis' Any idea??

3 5

Syslog-ng v3.5.3 - Core dump from a certain incoming message?
by Clayton Dukes 28 Jul '15

28 Jul '15

Has anyone seen this or know what may be causing it? When I run a stack trace, I can see that this host causes syslog-ng to crash every time it sends a message I've run a couple of tcpdumps and it *seems* to be caused by an ARP request from a Cyclades box. This seems very odd to me of course. 10 130.085308 Cyclades_01:be:4b SuperMic_9a:58:be ARP 60 Who has x.x.188.52? Tell x.x.188.11 The *only* other packets from that host are repeated so they don't seem to be the cause: 4 1.000259 x.x.188.11 x.x.188.52 Syslog 257 LOCAL0.NOTICE: Jul 23 11:04:05 src_dev_log@ACS-01 Buffering: S12.Server-Farm-6509-01 [Jul 23 11:04:03.267 EDT: %MCAST-SP-3-QUERY_INT_MISMATCH: Snooping Querier received a non-matching query interval (125000 msec),]\n [pid 28379] recvfrom(9, "<133>Jul 23 10:19:58 src_dev_log"..., 8192, 0, {sa_family=AF_INET, sin_port=htons(3284), sin_addr=inet_addr("x.x.188.11")}, [16]) = 181 [pid 28379] stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3519, ...}) = 0 [pid 28379] mprotect(0x7f956c346000, 12288, PROT_READ|PROT_WRITE) = 0 [pid 28379] write(2, "**\nERROR:../../lib/logmsg.c:535:"..., 114) = 114 [pid 28379] rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0 [pid 28379] tgkill(28374, 28379, SIGABRT) = 0 [pid 28379] --- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=28374, si_uid=0} --- [pid 28379] +++ killed by SIGABRT (core dumped) +++ [pid 28434] +++ killed by SIGABRT (core dumped) +++ [pid 28428] +++ killed by SIGABRT (core dumped) +++ +++ killed by SIGABRT (core dumped) +++ ______________________________________________________________ Clayton Dukes ______________________________________________________________

3 2

syslog-ng GSoC IRC event
by Budai, László 20 Jul '15

20 Jul '15

Hi, after midterm, we will have an IRC meeting. Participants: mainly students and mentors, and of course anyone else who are interested in :-) Topics: * Where are we now? Status updates on projects. * Talking about on how to share your results with community. (online demos, blogs, whatever...) DATE: Tue 7/21/15 4:00 PM – 5:00 PM Time zone: Budapest regards, Laszlo Budai

1 0