- syslog-ng - lists.balabit.hu

Syslog-ng 3.8.1 supported on aix6.1?
by Sumana Veni 23 Nov '16

23 Nov '16

Hi, Can you please help me understand if syslog-ng 3.8.1 OSE is compatible with AIX 6.1. Couldn't locate the binary anywhere. Or is it only available in the PE. Thanks in advance. Regards, Veni

2 1

Unofficial Debian/Ubuntu syslog-ng 3.7 and 3.8 packages: updated
by Laszlo Budai 19 Nov '16

19 Nov '16

Hi, I've updated the syslog-ng-3.7 and syslog-ng-3.8 packages in my OBS repository. On my test environment I was able to install the packages without any issues. Please, test and feedback. regards, Laszlo Budai

2 2

mysql destination on el6, with syslog-ng 3.8
by Glen Johnson 17 Nov '16

17 Nov '16

It appears the syslog-ng38epel6 on COPR uses libdbi 0.9.0 from el7. When attempting to install syslog-ng from here, on an CentOS 6 instance, and then install libdbi-dbd-mysql, yum is unable to find appropriate version of libdbi-dbd-mysql to install. On el6, only libdbi 0.8.3 appears to be available, which yum does not think is compatible with libdbi 0.9.0. Not sure how to use syslog-ng's mysql destination on CentOS/RHEL 6 due to this. Any insight how to solve this, without creating a separate copr/obs build, would be appreciated ! czanik.blogs.balabit.com/2016/08/syslog-ng-3-8-what-changed/ copr.fedorainfracloud.org/coprs/czanik/syslog-ng38epel6/ Glen

3 4

unable to install syslog-ng ubuntu 16.04
by Simon Smith 14 Nov '16

14 Nov '16

Hi All has anyone been able to install syslog-ng 3.8 successfully on ubuntu 16.04 ? I’m using the link from the mailing list a few months ago but again no luck - http://lists.balabit.hu/pipermail/syslog-ng/2016-August/023150.html <http://lists.balabit.hu/pipermail/syslog-ng/2016-August/023150.html> no matter what packages i try to install it just won’t work any ideas? root@syslog:~# apt-get install syslog-ng Reading package lists... Done Building dependency tree Reading state information... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: syslog-ng : Depends: syslog-ng-core (>= 3.8.1) but it is not going to be installed Depends: syslog-ng-mod-sql but it is not going to be installed Depends: syslog-ng-mod-mongodb but it is not going to be installed Depends: syslog-ng-mod-json but it is not going to be installed Recommends: syslog-ng-mod-journal but it is not going to be installed Recommends: syslog-ng-mod-smtp but it is not going to be installed Recommends: syslog-ng-mod-amqp but it is not going to be installed Recommends: syslog-ng-mod-geoip but it is not going to be installed Recommends: syslog-ng-mod-redis but it is not going to be installed Recommends: syslog-ng-mod-stomp but it is not going to be installed Recommends: syslog-ng-mod-riemann but it is not going to be installed Recommends: syslog-ng-mod-graphite but it is not going to be installed Recommends: syslog-ng-mod-python but it is not going to be installed Recommends: syslog-ng-mod-java but it is not going to be installed Recommends: syslog-ng-mod-java-common-lib but it is not going to be installed Recommends: syslog-ng-mod-elastic but it is not going to be installed Recommends: syslog-ng-mod-hdfs but it is not going to be installed Recommends: syslog-ng-mod-kafka but it is not going to be installed Recommends: syslog-ng-mod-http but it is not going to be installed Recommends: syslog-ng-mod-curl but it is not going to be installed Recommends: syslog-ng-mod-kvformat but it is not going to be installed Recommends: syslog-ng-mod-add-contextual-data but it is not going to be installed E: Unable to correct problems, you have held broken packages. root@syslog:~# apt-get install syslog-ng-core Reading package lists... Done Building dependency tree Reading state information... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: syslog-ng-core : Depends: syslog-ng-mod-journal but it is not going to be installed E: Unable to correct problems, you have held broken packages. root@syslog:~# apt-get install syslog-ng-mod-journal Reading package lists... Done Building dependency tree Reading state information... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: syslog-ng-mod-journal : Depends: syslog-ng-abi-3.8-0 but it is not installable E: Unable to correct problems, you have held broken packages. root@syslog:~# apt-get install syslog-ng syslog-ng-core syslog-ng-mod-sql syslog-ng-mod-mongodb syslog-ng-mod-json syslog-ng-mod-journal Reading package lists... Done Building dependency tree Reading state information... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: syslog-ng-mod-journal : Depends: syslog-ng-abi-3.8-0 but it is not installable syslog-ng-mod-json : Depends: syslog-ng-abi-3.8-0 but it is not installable syslog-ng-mod-mongodb : Depends: syslog-ng-abi-3.8-0 but it is not installable syslog-ng-mod-sql : Depends: syslog-ng-abi-3.8-0 but it is not installable E: Unable to correct problems, you have held broken packages. root@syslog:~# regards Simon - Hestor Ltd

9 31

Insider 2016-11: Enrichment; Elasticsearch; LISA; FreeBSD; All Things Open;
by Czanik, Péter 11 Nov '16

11 Nov '16

Dear syslog-ng users, This is the 52nd issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Enriching log messages ---------------------- Log messages already contain a wealth of information about what is happening in your systems. Still, logs can be enriched with additional information – like the geolocation belonging to an IP address – which can improve the usability of logs considerably. Read how you can enrich log messages in real-time using syslog-ng at https://www.balabit.com/blog/enriching-log-messages-with-additional-informa… What is new in syslog-ng PE and SSB ----------------------------------- Join us at our next webinar on November 15, where you can learn about new syslog-ng PE and SSB features like content-based alerting, multiple log spaces, key=value parser and Big Data destinations. You can register at http://go.balabit.com/q9VU0A0F5l0E7Q03Z0zY00T Logging to Elasticsearch simplified ----------------------------------- Elasticsearch is gaining the momentum as the ultimate destination for log messages. A great part of it is Kibana, but also that it is a scalable storage of arbitrary name-value pairs. Read, how the use of syslog-ng can greatly simplify how logs reach Elasticsearch: https://www.balabit.com/blog/logging-to-elasticsearch-made-simple-with-sysl… Coming up: LISA --------------- You can learn about syslog-ng at the Balabit booth #145 at LISA (Large Installation System Administration) conference in Boston. Expo days are on December 7 and 8: https://www.usenix.org/conference/lisa16/expo syslog-ng with Java support on FreeBSD -------------------------------------- Java destinations on FreeBSD are not supported by the syslog-ng team and for technical reasons (it downloads JAR files from the Internet) it is also marked broken in FreeBSD ports. Luckily it does not prevent it from working: some people even use it it production. Learn, how to get started at https://www.balabit.com/blog/test-the-latest-freebsd-with-syslog-ng-3-8-1/ All Things Open --------------- Last week I visited All Things Open, one of the largest open source conferences of the US East Coast. I was presenting syslog-ng in the Operations track of the conference, but luckily I had a chance to stay for the full two days of the event. Read my experiences at: TBD Your feedback and news, or tips about the next issue are welcome at documentation(a)balabit.com. To read this newsletter online, visit: https://syslog-ng.org/ Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik

1 1

Syslog-ng 3.7.2 install fails on U12.04
by Varugis Kurien 10 Nov '16

10 Nov '16

When installing from the repository here (http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/xUb…) the install fails to install syslog-ng3.7.2 with the message: rm_conffile voodoo unsupported for package syslog-ng-mod-python! -- This appears to be coming when installing syslog-ng-mod-python due to a failure in a preinstallation script. How can I fix this?

1 1

Possible elementary question
by Varugis Kurien 10 Nov '16

10 Nov '16

I'm having some conniptions doing central logging using syslog-ng. Here is the scenario: 1. I generate logs using syslog-ng on local servers and then send them to a log-destination server. Here is an example of some of the conf files on the generating server: more iptables.log.conf source s_var_log_iptables.log { file("/var/log/iptables.log" program-override(var_log_iptables.log) flags(no-parse)); }; log { source(s_var_log_iptables.log); destination(d_midfin_logger_2); }; more midfin_logger_2.conf template t_sdx_tagged { template("$ISODATE $HOST $PROGRAM $MSG "); template_escape(no); }; destination d_midfin_logger_2 { tcp( "10.4.16.161" port(514) template(t_sdx_tagged)); }; What I am trying to do on the log host is to: 1. Segregate at the top level by host. For each such host: * Split the logs into different directories so that for example, I'll have one directory for the audit log from that host, another for iptables etc - so that under each host I'll have /var/log/iptables.log etc. I've been unable to grok the documentation to figure out quite how to do this and would really appreciate some help. thanks vk ==

3 3

linux-audit-parser not found on stock install of Ubuntu 16.04
by Varugis Kurien 02 Nov '16

02 Nov '16

When using a freshly installed version of U16.04, I found that the linux audit log parser does not seem to be installed. For example: syslog-ng --module-registry|grep audit <-- returns nothing == 1. This started because I experienced a failure when trying to enable the linux audit parser: Error parsing parser expression, parser plugin linux-audit-parser not found in /etc/syslog-ng/conf.d/auditd.log.conf at line 3, column 8: included from /etc/syslog-ng/syslog-ng.conf line 164, column 1 linux-audit-parser(prefix(".auditd.")); 2. The corresponding conf file is below: source s_var_log_audit.log { file("/var/log/audit/audit.log" flags(no-parse)); }; parser p_audit_syslog_parser { linux-audit-parser(prefix(".auditd.")); }; log { source(s_var_log_audit.log); parser(p_audit_syslog_parser); destination(d_midfin_logger_2); }; The officially supported version of syslog-ng in ubuntu is 3.5.1. == syslog-ng 3.5.6 Installer-Version: 3.5.6 Revision: 3.5.6-2.1 [@416d315] (Ubuntu/16.04) Compile-Date: Oct 24 2015 03:49:19 Available-Modules: json-plugin,csvparser,system-source,tfgeoip,afsocket-notls,afamqp,basicfuncs,affile,afsocket-tls,dbparser,afmongodb,cryptofuncs,afsmtp,linux-kmsg-format,afuser,redis,afsocket,afstomp,sysl\ ogformat,confgen,afprog,afsql Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: on Enable-Linux-Caps: on Enable-Pcre: on Do I need to install an unofficial version or is there some elementary mistake that I am making. thanks Varugis

2 1

Feedback test
by Scheidler, Balázs 27 Oct '16

27 Oct '16

Hi, The README.md file visible on github advertises this email address that is forwarded to the mailing list. This is a test and shows if the forwarding works properly -- Bazsi

1 0

Fwd: syslog-ng 3.7.3 crashing randomly
by Soumyadip Das Mahapatra 25 Oct '16

25 Oct '16

Hi, We have installed syslog-ng 3.7.3 on RHEL 6.6 from COPR repo: >From dmesg: [Thu Sep 1 02:24:08 2016] syslog-ng[10374]: segfault at 1a116c10 ip 000000001a116c10 sp 00007f26eebfa928 error 15 [Sun Sep 4 20:10:47 2016] syslog-ng[27797]: segfault at 0 ip (null) sp 00007f998b5fb928 error 14 in syslog-ng[400000+3000] [Mon Sep 5 19:04:15 2016] syslog-ng[12727]: segfault at 36aaa42080 ip 00000039b8618df0 sp 00007f996fffc878 error 7 in libglib-2.0.so.0.2800.8[ 39b8600000+115000] [Tue Sep 6 03:01:21 2016] syslog-ng[3827]: segfault at 15cb09d0 ip 0000000015cb09d0 sp 00007f99aebfa928 error 15 [Fri Sep 9 05:03:41 2016] syslog-ng[28275] general protection ip:39b8618df0 sp:7f29c57f8878 error:0 in libglib-2.0.so.0.2800.8[ 39b8600000+115000] [Sun Sep 11 00:48:01 2016] syslog-ng[14479]: segfault at 0 ip (null) sp 00007f2a03caf928 error 14 in syslog-ng[400000+3000] [Sun Sep 11 10:15:37 2016] syslog-ng[2012]: segfault at 0 ip 00000036aaa3cc1c sp 00007feed2bfa880 error 6 in libsyslog-ng-3.7.so.0.0.0[ 36aaa00000+a2000] Core 1: (gdb) bt full #0 0x00000039b8618df0 in g_atomic_int_add () from /lib64/libglib-2.0.so.0 No symbol table info available. #1 0x00000036aaa3cbe4 in stats_counter_add (self=0xabe68ea0, thread_id=5) at lib/stats/stats-counter.h:39 No locals. #2 log_queue_fifo_move_input_unlocked (self=0xabe68ea0, thread_id=5) at lib/logqueue-fifo.c:193 queue_len = 1755882337 #3 0x00000036aaa3cd6e in log_queue_fifo_move_input (user_data=0xabe68ea0) at lib/logqueue-fifo.c:215 self = 0xabe68ea0 thread_id = 5 __PRETTY_FUNCTION__ = "log_queue_fifo_move_input" #4 0x00000036aaa456ca in main_loop_worker_invoke_batch_callbacks () at lib/mainloop-worker.c:270 cb = 0xabe690e0 lh = 0xabe690e0 lh2 = 0xabe690e0 #5 0x00000036aaa6fa2a in iv_work_thread_do_work (_thr=0x9d6bd530) at iv_work.c:118 work = 0x6fb160 thr = 0x9d6bd530 pool = 0x6e48c0 last_seq = 1477269410 #6 0x00000036aaa6ed4a in iv_run_tasks (st=0x7f29b44d3c00) at iv_task.c:48 t = <value optimized out> tasks = {next = 0x7f29c57f89a0, prev = 0x7f29c57f89a0} #7 0x00000036aaa7100c in iv_main () at iv_main_posix.c:106 to = {tv_sec = 10, tv_nsec = 0} st = 0x7f29b44d3c00 #8 0x00000036aaa6f841 in iv_work_thread (_thr=0x9d6bd530) at iv_work.c:200 thr = 0x9d6bd530 pool = 0x6e48c0 #9 0x00000036aaa71a1f in iv_thread_handler (_thr=0x75242f90) at iv_thread_posix.c:142 __clframe = {__cancel_routine = 0x36aaa71a80 <iv_thread_cleanup_handler>, __cancel_arg = 0x75242f90, __do_it = 1, __cancel_type = 0} thr = 0x75242f90 #10 0x00000039b6e079d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #11 0x00000039b6ae88fd in clone () from /lib64/libc.so.6 No symbol table info available. Core 2: (gdb) bt full #0 0x0000000000000000 in ?? () No symbol table info available. #1 0x00000036aaa456ca in main_loop_worker_invoke_batch_callbacks () at lib/mainloop-worker.c:270 cb = 0x87e96b10 lh = 0x87e96b10 lh2 = 0x0 #2 0x00000036aaa6fa2a in iv_work_thread_do_work (_thr=0x7034ffd0) at iv_work.c:118 work = 0x6fb390 thr = 0x7034ffd0 pool = 0x6e48b0 last_seq = 1323426114 #3 0x00000036aaa6ed4a in iv_run_tasks (st=0x7f29d4360200) at iv_task.c:48 t = <value optimized out> tasks = {next = 0x7f2a03caf9a0, prev = 0x7f2a03caf9a0} #4 0x00000036aaa7100c in iv_main () at iv_main_posix.c:106 to = {tv_sec = 10, tv_nsec = 0} st = 0x7f29d4360200 #5 0x00000036aaa6f841 in iv_work_thread (_thr=0x7034ffd0) at iv_work.c:200 thr = 0x7034ffd0 pool = 0x6e48b0 #6 0x00000036aaa71a1f in iv_thread_handler (_thr=0x426b490) at iv_thread_posix.c:142 __clframe = {__cancel_routine = 0x36aaa71a80 <iv_thread_cleanup_handler>, __cancel_arg = 0x426b490, __do_it = 1, __cancel_type = 0} thr = 0x426b490 #7 0x00000039b6e079d1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #8 0x00000039b6ae88fd in clone () from /lib64/libc.so.6 No symbol table info available. So looks like segfault is happening at random part of the code. # cat /etc/sysconfig/syslog-ng #--- # Syslog-ng command line options # See syslog-ng(8) for more details #--- SYSLOGNG_PID="/var/run/syslog-ng.pid" SYSLOGNG_OPTIONS="-p $SYSLOGNG_PID --fd-limit 30000" SYSLOGNG_COMPAT_PID="/var/run/syslogd.pid" # cat /etc/syslog-ng/syslog-ng.conf @version:3.6 options { flush_lines (0); time_reopen (10); log_fifo_size (20000); use_dns (yes); use_fqdn (yes); create_dirs (yes); dir_group("wheel"); dir_owner("nobody"); dir_perm(0755); owner("nobody"); group("nobody"); perm(0644); threaded(yes); keep_hostname (yes); chain_hostnames(yes); bad_hostname("[^[:print:]]"); dns_cache(yes); dns_cache_expire(300); dns_cache_expire_failed(30); dns_cache_size(1000); stats_freq(3600); flush_lines(0); }; source s_sys { file ("/proc/kmsg" program_override("kernel: ")); unix-stream ("/dev/log"); # udp(ip(0.0.0.0) port(514)); }; source s_tcp { network(transport("tcp") port(514) so_rcvbuf(8388608) max-connections(200) log-iw-size(20000)); }; source s_udp { network(transport("udp") port(514) so_rcvbuf(8388608) log-iw-size(20000)); }; source s_net { network(transport("udp") port(515) so_rcvbuf(8388608) log-iw-size(20000) tags("source_net")); network(transport("tcp") port(515) so_rcvbuf(8388608) max-connections(200) log-iw-size(20000) tags("source_net")); }; source s_windows { network(transport("tcp") port(516) so_rcvbuf(8388608) max-connections(200) log-iw-size(20000) tags("windows")); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; destination d_userx { file("/u1/syslog/$HOST/$YEAR$MONTH$DAY/user.log"); }; destination d_kernx { file("/u1/syslog/$HOST/$YEAR$MONTH$DAY/kern.log"); }; destination d_mailx { file("/u1/syslog/$HOST/$YEAR$MONTH$DAY/mail.log"); }; destination d_mp { file("/u1/syslog/$HOST/$YEAR$MONTH$DAY/mp.log"); }; destination d_daemonx { file("/u1/syslog/$HOST/$YEAR$MONTH$DAY/daemon.log"); }; destination d_authx { file("/u1/syslog/$HOST/$YEAR$MONTH$DAY/auth.log"); }; destination d_lprx { file("/u1/syslog/$HOST/$YEAR$MONTH$DAY/lpr.log"); }; destination d_cronx { file("/u1/syslog/$HOST/$YEAR$MONTH$DAY/cron.log"); }; destination d_messagesx { file("/u1/syslog/$HOST/$YEAR$MONTH$DAY/messages.log"); }; destination d_networkx { file("/u1/syslog/$HOST/$YEAR$MONTH$DAY/network.log"); }; destination d_firewallx { file("/u1/syslog/$HOST/$YEAR$MONTH$DAY/firewall.log"); }; destination d_local0 { file("/u1/syslog/$HOST/$YEAR$MONTH$DAY/sp.log"); }; destination d_local1 { file("/u1/syslog/$HOST/$YEAR$MONTH$DAY/app.log"); }; destination d_localx { file("/u1/syslog/$HOST/$YEAR$MONTH$DAY/local.log"); }; destination d_local6 { file("/u1/syslog/$HOST/$YEAR$MONTH$DAY/httpd.log"); }; destination d_dnslogs { file("/u1/syslog/$HOST/$YEAR$MONTH$DAY/dnslogs.log"); }; destination d_logstash { tcp("127.0.0.1" port(5140) template("$R_ISODATE <$FACILITY_NUM.$LEVEL_NUM> $HOST $MSGHDR $MESSAGE\n") frac_digits(3) template-escape(no)); pipe("/var/log/logstash-pipe" template("$R_ISODATE <$FACILITY_NUM.$LEVEL_NUM> $HOST $MSGHDR $MESSAGE\n") frac_digits(3) template-escape(no)); }; destination d_userauth { tcp("abc.xyz.com" port(13456)); }; destination d_otto { tcp("abc.xyz.com" port(13456)); tcp("abc.xyz.com" port(13456)); }; destination d_notifier { tcp("abc.xyz.com" port(6000)); tcp("10.255.xx.yy" port(1248)); }; destination d_graylog { syslog("abc.xyz.com" transport("tcp") port(9514)); pipe("/var/log/graylog-pipe" ts-format(iso)); }; destination d_gtsadfeed { syslog("abc.xyz.com" transport("udp") port(514)); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)) and not message("Syslog connection (accepted|closed)"); }; filter f_auth { facility(auth,authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron,solaris-cron); }; filter f_user { facility(user); }; filter f_daemon { facility(daemon); }; filter f_lpr { facility(lpr); }; filter f_dnslogs { program("mydns_logger"); }; filter f_local0 { facility(local0); }; filter f_local1 { facility(local1) and not program("mydns_logger"); }; filter f_local2 { facility(local2); }; filter f_local3 { facility(local3); }; filter f_local4 { facility(local4); }; filter f_local5 { facility(local5) or tags("source_net"); }; # local5 is network, so is port 515 filter f_local6 { facility(local6); }; filter f_local7 { facility(local7); }; filter f_alert { level(alert); }; filter f_crit { level(crit); }; filter f_err { level(err); }; filter f_warn { level(warn); }; filter f_notice { level(notice); }; filter f_info { level(info); }; filter f_debug { level(debug); }; filter f_nonet { not facility(local4) and not facility(local5) and not tags("source_net"); }; filter f_nocf3 { not program("cf3"); }; filter f_nodhcpd { not program("dhcpd"); }; filter f_windows { tags("windows"); }; filter f_snmptrapd { program("snmptrapd"); }; filter f_userauth { not facility(local3); }; filter f_notifier_filter { not match("ASA-4-302015|ASA-4- 302013|TRAFFIC|permitted|Deny|Denied|denied", value("MESSAGE")); }; filter f_gtsadfeed_filter { host("*-adc*.abc.xyz.com" type(glob)); }; rewrite f_rewrite_name { set("$FULLHOST_FROM", value("HOST") condition(not tags("windows") and not match("REMOTELOG", value("MESSAGE")))); }; rewrite r_rewrite_name_windows { set("$FULLHOST_FROM", value("HOST") condition(not match("REMOTELOG", value("MESSAGE")))); }; rewrite r_snmptrapd { subst("^([^ ]+) (.*)", "${2}", value("MESSAGE")); set("${1}", value("HOST")); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; log { source(s_sys); filter(f_snmptrapd); rewrite(r_snmptrapd); destination(d_networkx); destination(d_logstash); log { filter(f_notifier_filter); destination(d_notifier); }; }; log { source(s_net); source(s_udp); source(s_tcp); source(s_sys); source(s_windows); log { filter(f_userauth); destination(d_userauth); destination(d_otto); }; log { rewrite(f_rewrite_name); log { destination(d_logstash); }; log { filter(f_nocf3); filter(f_nodhcpd); destination(d_graylog); }; log { filter(f_user); destination(d_userx); }; log { filter(f_kernel); destination(d_kernx); }; log { filter(f_mail); destination(d_mailx); }; log { filter(f_daemon); destination(d_daemonx); }; log { filter(f_auth); destination(d_authx); }; log { filter(f_lpr); destination(d_lprx); }; log { filter(f_cron); destination(d_cronx); }; log { filter(f_local0); destination(d_local0); }; log { filter(f_local1); destination(d_local1); }; log { filter(f_local2); destination(d_authx); }; log { filter(f_dnslogs); destination(d_dnslogs); }; log { filter(f_local4); destination(d_firewallx); log { filter(f_notifier_filter); destination(d_notifier); }; }; log { filter(f_local5); destination(d_networkx); log { filter(f_notifier_filter); destination(d_notifier); }; }; log { filter(f_local6); destination(d_local6); }; log { filter(f_local7); destination(d_mp); }; log { filter(f_windows); filter(f_gtsadfeed_filter); rewrite(r_rewrite_name_windows); destination(d_gtsadfeed); }; }; }; Could you please advise? Regards, Soumyadip

2 1

All surpassed messages are from Dec 31
by Scot Needy 22 Oct '16

22 Oct '16

This is strange. All suppressed messages I see have the Msg date re-written as "Dec 31 23:59:58”. Date never changes. Msg: Dec 31 23:59:58 192.168.100.73 Fdm[31436]: Last message '[FFE56B90 verbose 'P' repeated 1 times, suppressed by syslog-ng on

2 5

syslog-ng: list of changes
by Budai, László 21 Oct '16

21 Oct '16

Hi, today we merged some pull requests ( https://github.com/balabit/syslog-ng ). Changes: - Java destination updates - Support HDFS 2.7.3 - Support Elasticsearch 2.4 - release workflow - a new release branch created - debian package scripts are available only on that branch - Merged group unset <https://github.com/balabit/syslog-ng/pull/1235> feature - Fixed riemann configuration parse error <https://github.com/balabit/syslog-ng/issues/1212> - Fixed DNS cache list corruption <https://github.com/balabit/syslog-ng/pull/1218> regards, Laszlo Budai

1 0

AMQP Module not present?
by Thomas Polliard 20 Oct '16

20 Oct '16

The documentation seems to imply that the amqp driver should be available in the OSE version of syslog. I am running on a RHEL 7 systems with syslog-ng installed from EPEL. syslog-ng.x86_64 3.5.6-3.el7 @epel syslog-ng-devel.x86_64 3.5.6-3.el7 epel syslog-ng-geoip.x86_64 3.5.6-3.el7 epel syslog-ng-json.x86_64 3.5.6-3.el7 epel syslog-ng-libdbi.x86_64 3.5.6-3.el7 epel syslog-ng-mongodb.x86_64 3.5.6-3.el7 epel syslog-ng-redis.x86_64 3.5.6-3.el7 epel syslog-ng-smtp.x86_64 3.5.6-3.el7 epel When I try and load the amqp driver I get Error parsing destination, destination plugin amqp not found in /etc/syslog-ng/conf.d/51-ampq.conf at line 3, column 2: included from /etc/syslog-ng/syslog-ng.conf line 68, column 1 amqp( ^^^^ Any help would be appreciated. I am assuming I am either, missing a package that I should have installed or there is some missing information in the documentation. The document I was reading was https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-… Thanks again -- *Thomas J. Polliard* -- -------------------------------------------------- This e-mail transmission contains information intended only for the use of the recipient(s) named above. Further, it contains information that may be privileged and confidential. If you are not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this message (including any attachments) is strictly prohibited. If you have received this e-mail in error, please notify the sender by reply e-mail and then delete this message from your mail system. Thank you for your compliance.

2 1

syslog-ng fails logging to existing sqlite database
by Andrea Laini 19 Oct '16

19 Oct '16

Hello everybody, I would like to ask you a question about libdbi (version 0.8.4.1 with libdbi-drivers 0.8.3). I'm using syslog-ng 3.5.4.1 on my Yocto Poky 1.7 Linux distro. I need to log my data into a sqlite3 database and everything works fine, but the database has not be written: no log data stored. According to debug infos, it seems that syslog-ng recognizes that the database exists, connects to it, found (with the sql statement "SELECT * FROM tableName WHERE 0=1) that the table named "tableName" exists but it gets an error when verifying columns' names; so it tries to add a column with that name to the table, but the operation fails since this column already exists. I've googled and search on the web, but I've found nothing interesting. Trying syslog-ng 3.6.4 doesn't change anything, while using libdbi 0.9.0 returns an error when compiling due to mismatch between function signature in the source and header file. Have you any suggestions? Thanks in advance, Regards Andrea Please be informed that our offices will be closed for National Holidays from Monday 31/10/16 to Tuesday 01/11/16 included. We will be back on Wednesday 02/11/16. Vi ricordiamo che saremo chiusi per festività i giorni lunedì 31/10/16 e martedì 01/11/16. Saremo nuovamente operativi da mercoledì 02/11/16. Andrea Laini Email: andrea.laini(a)claypaky.it<mailto:andrea.laini@claypaky.it> _________________________________ Clay Paky S.p.A. Via Pastrengo, 3/b - 24068 Seriate (BG) - ITALY Phone: +39 - 035.654.311 - Fax: +39 - 035.30.18.76 Web: www.claypaky.it<http://www.claypaky.it/> _________________________________ Read our Disclaimer @ www.claypaky.it/disclaimer<http://www.claypaky.it/disclaimer/index.html>

2 4

add-contextual-data()
by Scot Needy 19 Oct '16

19 Oct '16

Could I leverage this type of CSV to add VLAN ID and VLAN description tags if I already have filter f_192_168_1_0 { netmask(192.168.1.0/24);}; VLAN_Descriptions.csv VLANID, VLAN Description, subnet/24 43, Database_#14141, 192.168.1.1/24 filter f_192_168_1_0 { netmask(192.168.1.0/24); add-contextual-data( selector($IP is in f_192_168_1_0 or something specific?????/) database(“/opt/syslog-ng/etc/VLAN_Descriptions.csv") default-selector("unknown-hostname") ); };

2 2

Openstack is a chatty $#!^
by Scot Needy 13 Oct '16

13 Oct '16

I’m quickly finding out that I need to divide my syslog traffic to different ES indexes or I’m missing a better way. I already have filters by subnet but can you have multiple ES destinations defined ? How do you recommend breaking up syslog-ng traffic for ES destinations ? destination d_os { elasticsearch2( index(“openstack_${YEAR}.${MONTH}.${DAY}") type("syslog") # Description: The type of the index. For example, type("test") template("$(format-json --scope rfc3164 --scope nv-pairs --exclude R_DATE --key ISODATE)\n”) …. } destination d_es { elasticsearch2( index(“syslog-ng_${YEAR}.${MONTH}.${DAY}") type("syslog") # Description: The type of the index. For example, type("test") template("$(format-json --scope rfc3164 --scope nv-pairs --exclude R_DATE --key ISODATE)\n”) …. } filter f_192_168_1_0 { netmask(192.168.1.0/24);}; filter f_192_168_4_0 { netmask(192.168.4.0/29);}; filter f_192_168_4_8 { netmask(192.168.4.8/29);}; log { source(s_net); parser(pattern_db); filter(f_192_168_1_0); filter(f_192_168_4_0); destination (d_es); }; log { source(s_net); parser(pattern_db); filter(f_192_168_4_8); destination (d_os); };

2 2

FreeBSD and Java destinations
by Czanik, Péter 11 Oct '16

11 Oct '16

Hi, FreeBSD 11.0 was released yesterday. I collected my favorite new features in a blog post where I also described how to get started Java destinations on FreeBSD. As an example I used the Elasticsearch destination. The blog is available at https://www.balabit.com/blog/test-the-latest-freebsd-with-syslog-ng-3-8-1/ Happy testing, and if you have any questions, problems or success stories, let me know! Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

conf file structure best practice for patterns
by Scot Needy 08 Oct '16

08 Oct '16

I have some code that can pull subnet info from IPplan or Solarwinds to generate 3 conf files. dynamic-dest.conf destination d_192_168_1_0 { file(/opt/syslog-ng/logs/192_168_1_0/$YEAR$MONTH$DAY-$HOUR-$HOST.log);}; dynamic-filter.conf filter f_192_168_1_0 { netmask(192.168.1.0/24);}; dynamic-logs.conf log { source(s_net); filter(f_192_168_1_0); destination(d_192_168_1_0);}; Many but not all of these subnets a specific to an application like “Cisco ASA, VMware or server" This works well for flat file archiving but when using a pattern database would it be best to have one single large patterndb or define unique ones for each area when splitting that data stream to ES ? log { source(s_net); parser(pattern_db); destination(d_es);}; OR log { source(s_net); filter(f_192_168_1_0); parser(ESXpattern_db) ;destination(d_es);}; log { source(s_net); filter(f_192_168_2_0); parser(ASApattern_db) ;destination(does);}; …

2 1

ES destination node or SLB ?
by Scot Needy 06 Oct '16

06 Oct '16

If I can put a 3 node ES cluster behind an SLB does it make sense to do that for a syslog-ng destination?

1 0

Elastic Search destination with x-pack
by Scot Needy 06 Oct '16

06 Oct '16

Hi, Installed the x-pack to secure my ES instance and Kibana but noticed there does not seem to be any docs on how to handle ES authentication in from syslog-ng.conf. I might be missing it. Looking at https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-… <https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-…> Thanks

2 6

Ubuntu 16 unmet dependencies - won't install
by Clayton Dukes 03 Oct '16

03 Oct '16

Hi Guys, Did something change recently with http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/xUb… (or am I just overworked and missing something obvious?) I am unable to install now from a fresh OS: root@foo[~]: # apt-get install syslog-ng-core # (also tried syslog-ng syslog-ng-core here) Reading package lists... Done Building dependency tree Reading state information... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: syslog-ng : Depends: syslog-ng-mod-sql but it is not going to be installed Depends: syslog-ng-mod-mongodb but it is not going to be installed Depends: syslog-ng-mod-json but it is not going to be installed Recommends: syslog-ng-mod-journal but it is not going to be installed Recommends: syslog-ng-mod-smtp but it is not going to be installed Recommends: syslog-ng-mod-amqp but it is not going to be installed Recommends: syslog-ng-mod-geoip but it is not going to be installed Recommends: syslog-ng-mod-redis but it is not going to be installed Recommends: syslog-ng-mod-stomp but it is not going to be installed Recommends: syslog-ng-mod-riemann but it is not going to be installed Recommends: syslog-ng-mod-graphite but it is not going to be installed Recommends: syslog-ng-mod-python but it is not going to be installed Recommends: syslog-ng-mod-java but it is not going to be installed Recommends: syslog-ng-mod-java-common-lib but it is not going to be installed Recommends: syslog-ng-mod-elastic but it is not going to be installed Recommends: syslog-ng-mod-hdfs but it is not going to be installed Recommends: syslog-ng-mod-kafka but it is not going to be installed Recommends: syslog-ng-mod-http but it is not going to be installed Recommends: syslog-ng-mod-curl but it is not going to be installed Recommends: syslog-ng-mod-kvformat but it is not going to be installed Recommends: syslog-ng-mod-add-contextual-data but it is not going to be installed syslog-ng-core : Depends: syslog-ng-mod-journal but it is not going to be installed E: Unable to correct problems, you have held broken packages. [cid:image001.png@01D21B13.4E86C770] Clayton Dukes / Founder and CTO http://www.logzilla.net<http://www.logzilla.net/>

3 4

ERROR:lib/dnscache.c:112:dns_cache_key_hash: code should not be reached
by Claus Albøge 01 Oct '16

01 Oct '16

Hi, I have noticed that syslog-ng exits from time to time, with the following error: ERROR:lib/dnscache.c:112:dns_cache_key_hash: code should not be reached Trying to run syslog-ng from gdb show the following: # gdb --args /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-80.el7 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/sbin/syslog-ng...Reading symbols from /usr/lib/debug/usr/sbin/syslog-ng.debug...done. done. (gdb) run Starting program: /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". [New Thread 0x7ffff1e8a700 (LWP 18743)] [New Thread 0x7ffff1689700 (LWP 18748)] [New Thread 0x7ffff0e88700 (LWP 18893)] [New Thread 0x7fffe3fff700 (LWP 18956)] [New Thread 0x7fffe37fe700 (LWP 18959)] [New Thread 0x7fffe2ffd700 (LWP 18960)] [New Thread 0x7fffe27fc700 (LWP 18961)] [New Thread 0x7fffe1ffb700 (LWP 18962)] [Thread 0x7ffff1689700 (LWP 18748) exited] [Thread 0x7fffe27fc700 (LWP 18961) exited] [Thread 0x7fffe3fff700 (LWP 18956) exited] [New Thread 0x7fffe3fff700 (LWP 19114)] [New Thread 0x7fffe27fc700 (LWP 19136)] [New Thread 0x7ffff1689700 (LWP 19137)] [Thread 0x7fffe27fc700 (LWP 19136) exited] [Thread 0x7ffff1e8a700 (LWP 18743) exited] [New Thread 0x7ffff1e8a700 (LWP 19338)] [New Thread 0x7fffe27fc700 (LWP 19339)] [Thread 0x7fffe27fc700 (LWP 19339) exited] [Thread 0x7ffff1e8a700 (LWP 19338) exited] [New Thread 0x7ffff1e8a700 (LWP 19566)] [New Thread 0x7fffe27fc700 (LWP 19567)] [Thread 0x7ffff1e8a700 (LWP 19566) exited] [Thread 0x7fffe37fe700 (LWP 18959) exited] [New Thread 0x7ffff1e8a700 (LWP 19731)] [New Thread 0x7fffe37fe700 (LWP 19732)] [Thread 0x7fffe27fc700 (LWP 19567) exited] [Thread 0x7fffe1ffb700 (LWP 18962) exited] [New Thread 0x7fffe1ffb700 (LWP 19984)] [New Thread 0x7fffe27fc700 (LWP 19985)] ** ERROR:lib/dnscache.c:112:dns_cache_key_hash: code should not be reached Program received signal SIGABRT, Aborted. [Switching to Thread 0x7ffff1689700 (LWP 19137)] 0x00007ffff5fb05f7 in raise () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-13.el7.x86_64 elfutils-libelf-0.163-3.el7.x86_64 elfutils-libs-0.163-3.el7.x86_64 eventlog-0.2.13-4.el7.x86_64 glib2-2.42.2-5.el7.x86_64 glibc-2.17-106.el7_2.8.x86_64 ivykis-0.36.2-2.el7.x86_64 json-c-0.11-4.el7_0.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.13.2-12.el7_2.x86_64 libattr-2.4.46-12.el7.x86_64 libcap-2.22-8.el7.x86_64 libcom_err-1.42.9-7.el7.x86_64 libgcc-4.8.5-4.el7.x86_64 libgcrypt-1.5.3-12.el7_1.1.x86_64 libgpg-error-1.12-3.el7.x86_64 libnet-1.1.6-7.el7.x86_64 libselinux-2.2.2-6.el7.x86_64 libuuid-2.23.2-26.el7_2.3.x86_64 openssl-libs-1.0.1e-51.el7_2.5.x86_64 pcre-8.32-15.el7_2.1.x86_64 systemd-libs-219-19.el7_2.13.x86_64 tcp_wrappers-libs-7.6-77.el7.x86_64 xz-libs-5.1.2-12alpha.el7.x86_64 zlib-1.2.7-15.el7.x86_64 (gdb) (gdb) bt #0 0x00007ffff5fb05f7 in raise () from /lib64/libc.so.6 #1 0x00007ffff5fb1ce8 in abort () from /lib64/libc.so.6 #2 0x00007ffff70426d5 in g_assertion_message () from /lib64/libglib-2.0.so.0 #3 0x00007ffff704276a in g_assertion_message_expr () from /lib64/libglib-2.0.so.0 #4 0x00007ffff7b62cc4 in dns_cache_key_hash (e=<optimized out>) at lib/dnscache.c:112 #5 0x00007ffff700c4fc in g_hash_table_remove_internal () from /lib64/libglib-2.0.so.0 #6 0x00007ffff7b67584 in resolve_sockaddr_to_inet_or_inet6_hostname (host_resolve_options=0x651754, saddr=0x7fffec3840d0, result_len=0x7ffff1686370) at lib/host-resolve.c:332 #7 resolve_sockaddr_to_hostname (result_len=result_len@entry=0x7ffff1686370, saddr=0x7fffec3840d0, host_resolve_options=0x651754) at lib/host-resolve.c:343 #8 0x00007ffff7b6c03a in log_source_mangle_hostname (self=self@entry=0x1cdf400, msg=msg@entry=0x7fffe4ec6da0) at lib/logsource.c:156 #9 0x00007ffff7b6c280 in log_source_queue (s=0x1cdf400, msg=0x7fffe4ec6da0, path_options=0x7ffff1686550, user_data=<optimized out>) at lib/logsource.c:266 #10 0x00007ffff7b6ca0a in log_pipe_queue (path_options=<optimized out>, msg=<optimized out>, s=<optimized out>) at lib/logpipe.h:339 #11 log_pipe_forward_msg (self=<optimized out>, path_options=<optimized out>, msg=<optimized out>) at lib/logpipe.h:302 #12 log_pipe_queue (path_options=<optimized out>, msg=<optimized out>, s=<optimized out>) at lib/logpipe.h:343 #13 log_pipe_forward_msg (self=<optimized out>, path_options=<optimized out>, msg=<optimized out>) at lib/logpipe.h:302 #14 log_pipe_queue (path_options=<optimized out>, msg=0x7fffe4ec6da0, s=0x7fffe4ec6da0) at lib/logpipe.h:343 #15 log_source_post (self=self@entry=0x1cdf400, msg=msg@entry=0x7fffe4ec6da0) at lib/logsource.c:249 #16 0x00007ffff7b6b179 in log_reader_handle_line (aux=0x7ffff1686600, length=<optimized out>, line=<optimized out>, self=0x1cdf400) at lib/logreader.c:320 #17 log_reader_fetch_log (self=self@entry=0x1cdf400) at lib/logreader.c:381 #18 0x00007ffff7b6b299 in log_reader_work_perform (s=0x1cdf400) at lib/logreader.c:88 #19 0x00007ffff7b7307d in _work (self=<optimized out>) at lib/mainloop-io-worker.c:52 #20 0x00007ffff655dc67 in iv_work_thread_do_work () from /lib64/libivykis.so.0 #21 0x00007ffff655cfc3 in iv_run_tasks () from /lib64/libivykis.so.0 #22 0x00007ffff655f7dc in iv_main () from /lib64/libivykis.so.0 #23 0x00007ffff655dab4 in iv_work_thread () from /lib64/libivykis.so.0 #24 0x00007ffff6560270 in iv_thread_handler () from /lib64/libivykis.so.0 #25 0x00007ffff6344dc5 in start_thread () from /lib64/libpthread.so.0 #26 0x00007ffff6071ced in clone () from /lib64/libc.so.6 # grep dns /etc/syslog-ng/syslog-ng.conf use_dns (no); Not sure what its trying to resolve. # syslog-ng -V syslog-ng 3.8.1 Installer-Version: 3.8.1 Revision: Module-Directory: //usr/lib64/syslog-ng Module-Path: //usr/lib64/syslog-ng Available-Modules: disk-buffer,sdjournal,afstomp,json-plugin,cryptofuncs,graphite,cef,kvformat,add-contextual-data,dbparser,pseudofile,csvparser,syslogformat,confgen,afsocket,afuser,date,linux-kmsg-format,system-source,basicfuncs,afamqp,affile,afprog Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: on Enable-Linux-Caps: off Please let me know, if more info is needed. /Claus A

4 9

Converting multiline text input to concatanated single-line syslog format
by Newport, Brendan (Contractor - Security Operations - Development & Support) 30 Sep '16

30 Sep '16

Classification: Public Hi! My first posting on the mailing list. I've run into a snag trying to get multiline logs concatenated onto one line and written as syslog-format messages. Actually the second part is working fine; I can get the first line identified and incorporated into a syslog message, but all subsequent lines aren't included. Just for testing I've some simple input; 'IMP-386: ORACLE error 386 encountered ORA-01017: invalid username/password; logon deniedUsername: Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production With the Partitioning, OLAP and Data Mining options Export file created by EXPORT:V10.02.01 via conventional path Warning: the objects were exported by FALCON, not by you import done in WE8ISO8859P1 character set and AL16UTF16 NCHAR character set IMP-00034: Warning: FromUser "FALBOS" not found in export file' What I'm aiming to do is log only the Oracle errors, commencing at the string 'IMP' and ending only when the next line is found with 'IMP' (this log only sees errors) So in syslog-ng.conf (version 5.0.6b PE) source s_table { file("/path to Table.log" multi-line-prefix("IMP") multi-line-garbage("set$") flags(no-p arse) flags(no-multi-line) program_override("Table") default-facility(local5) default-priority(info)); }; Just to try to get things working, I've hard-coded the final string present in my sample input - 'set' The output is scheduled to go off-server, but for the moment I want to see the transformed messages in their own file; destination d_table { file("/auditsox/table.log"); }; log { source(s_table); destination(d_table); }; What I get from the above (in d_table) is; Sep 29 14:07:15 p14425dev022 Table: IMP-386: ORACLE error 386 encountered < Ok, first line transformed into syslog messages ORA-01017: invalid username/password; logon deniedUsername: Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production With the Partitioning, OLAP and Data Mining options Export file created by EXPORT:V10.02.01 via conventional path Warning: the objects were exported by FALCON, not by you import done in WE8ISO8859P1 character set and AL16UTF16 NCHAR character Sep 29 14:07:26 p14425dev022 Table: IMP-00034: Warning: FromUser "FALBOS" not found in export file _____________________ The snag is, the lines after the first line in the input aren't concatenated, but are rather individual lines, with an LF. So I'm doing something wrong. With this source; source s_table { file("/path to Table.log " flags(no-multi-line) flags(no-parse) program_override(" Table") default-facility(local5) default-priority(info)); }; I can get every line of input to transform into a syslog message. Not much use though! Can anyone provide a pointer as to what I'm doing wonky? Thanks Brendan Lloyds Banking Group plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in Scotland no. SC95000. Telephone: 0131 225 4555. Lloyds Bank plc. Registered Office: 25 Gresham Street, London EC2V 7HN. Registered in England and Wales no. 2065. Telephone 0207626 1500. Bank of Scotland plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in Scotland no. SC327000. Telephone: 03457 801 801. Cheltenham & Gloucester plc. Registered Office: Barnett Way, Gloucester GL4 3RL. Registered in England and Wales 2299428. Telephone: 0345 603 1637 Lloyds Bank plc, Bank of Scotland plc are authorised by the Prudential Regulation Authority and regulated by the Financial Conduct Authority and Prudential Regulation Authority. Cheltenham & Gloucester plc is authorised and regulated by the Financial Conduct Authority. Halifax is a division of Bank of Scotland plc. Cheltenham & Gloucester Savings is a division of Lloyds Bank plc. HBOS plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in Scotland no. SC218813. This e-mail (including any attachments) is private and confidential and may contain privileged material. If you have received this e-mail in error, please notify the sender and delete it (including any attachments) immediately. You must not copy, distribute, disclose or use any of the information in it or any attachments. Telephone calls may be monitored or recorded.

6 7

Converting multiline text input to concatanated single-line syslog format
by Newport, Brendan (Contractor - Security Operations - Development & Support) 30 Sep '16

30 Sep '16

Classification: Public Ha! I think I might have been trying to achieve the impossible - getting syslog-ng to covert multiline log messages into concatenated syslog messages. That doesn't appear to work. But I defined syslogd instance to direct local5.info messatges received from my syslog-ng client into a log file and all works fine! more local5.log Sep 30 14:45:35 <hostname> local5:info <hostname> Table: IMP-386: ORACLE error 386 encountered ORA-01017: invalid username/passwo rd; logon deniedUsername: Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production Sep 30 14:45:35 <hostname> local5:info <hostname> Table: IMP-00034: Warning: FromUser "FALBOS" not found in export file With the P artitioning, OLAP and Data Mining options Export file created by EXPORT:V10.02.01 via conventional path Sep 30 14:45:35 <hostname> local5:info <hostname> Table: IMP-386: ORACLE error 386 encountered Warning: the objects were exported by FALCON, not by you import done in WE8ISO8859P1 character set and AL16UTF16 NCHAR character set Using this config is syslog-ng.conf; source s_table { file("<path to input multi-line log file>" multi-line-prefix("IMP") flags(no-parse) flags(no-multi-line) program_override("Table") default-facility(local5) default-priority(info)); }; filter f_local5_info { facility(local5) and level(info); }; and with the destination defined as a udp IP address to port 514; log { source(s_table); filter(f_local5_info); destination(d_syslog_udp); }; As the desire is to get multiline non-syslog logs off-host to Qradar as syslog messages, that's the POC passed fine! Lloyds Banking Group plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in Scotland no. SC95000. Telephone: 0131 225 4555. Lloyds Bank plc. Registered Office: 25 Gresham Street, London EC2V 7HN. Registered in England and Wales no. 2065. Telephone 0207626 1500. Bank of Scotland plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in Scotland no. SC327000. Telephone: 03457 801 801. Cheltenham & Gloucester plc. Registered Office: Barnett Way, Gloucester GL4 3RL. Registered in England and Wales 2299428. Telephone: 0345 603 1637 Lloyds Bank plc, Bank of Scotland plc are authorised by the Prudential Regulation Authority and regulated by the Financial Conduct Authority and Prudential Regulation Authority. Cheltenham & Gloucester plc is authorised and regulated by the Financial Conduct Authority. Halifax is a division of Bank of Scotland plc. Cheltenham & Gloucester Savings is a division of Lloyds Bank plc. HBOS plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in Scotland no. SC218813. This e-mail (including any attachments) is private and confidential and may contain privileged material. If you have received this e-mail in error, please notify the sender and delete it (including any attachments) immediately. You must not copy, distribute, disclose or use any of the information in it or any attachments. Telephone calls may be monitored or recorded.

2 1

syslog-ng forwarding and processing issue
by Denis Dolinský 30 Sep '16

30 Sep '16

Hi guys, I have following setup in place: remote server - 192.168.1.10 pseudomizer - syslog-ng PE in client mode - 192.168.2.10 SIEM - 192.168.3.10 So I am sending syslog logs from remote server to pseudomizer: source src { internal()}; destination dst { udp ("192.168.2.10) port (514);}; log { source(src); destination (dst); this is very old config from syslog v4 Then on pseudomizer - syslog-ng LTS 6.0.1 PE, I am collecting the logs, processing them - removing private data, putting pseudonyms instead and forwarding them to SIEM. source s_net_udp514 { syslog( ip(192.168.2.10) ip-protocol(4) transport("udp") so_rcvbuf(2097152) ); }; source src { internal(); unix-dgram("/dev/log"); system (); }; destination d_net_udp514 { syslog ( "192.168.3.10" port(514) transport(udp) spoof_source(yes) mark_mode(periodical));}; rewrite r_rewrite { subst("admin", "pseudonym000001", value("MESSAGE"), flags("global")); log { source(s_net_udp514); source (src); rewrite(r_rewrite); # do the pseudomizing destination(d_net_udp514); }; On SIEM device, I can see only pseudomizer internal logs (src), not processed logs from remote server. Any advice ? Many thanks. Denis

5 10