- syslog-ng - lists.balabit.hu

unknown symbol errors in jounalctl -xe
by vijay amruth 26 Jul '17

26 Jul '17

Hello All, hope everyone is doing great, I am unable to start syslog-ng, I see this messages... Any help is appreciated. Jul 25 11:36:14 server-1systemd[1]: Unit syslog.socket entered failed state. Jul 25 11:36:14 server-1systemd[1]: Unit syslog-ng.service entered failed state. Jul 25 11:36:14 server-1systemd[1]: syslog-ng.service failed. Jul 25 11:36:18 server-1kernel: dccp_diag: Unknown symbol dccp_hashinfo (err 0) Jul 25 11:36:18 server-1kernel: dccp_diag: Unknown symbol dccp_hashinfo (err 0) Jul 25 11:36:48 server-1systemd[1]: Listening on Syslog Socket. [root@server-1 tmp]# systemctl status syslog-ng.service ● syslog-ng.service - System Logger Daemon Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled) Active: activating (start) since Tue 2017-07-25 11:44:20 GMT+7; 36s ago Docs: man:syslog-ng(8) Main PID: 21822 (syslog-ng) CGroup: /system.slice/syslog-ng.service └─21822 /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid --fd-limit 50000 Jul 25 11:44:20 lvsinfosecsplunkcollector-1137760 systemd[1]: Starting System Logger Daemon... [root@server-1 tmp]# syslog-ng -V syslog-ng 3.9.1 Installer-Version: 3.9.1 Revision: Module-Directory: /usr/local/lib/syslog-ng Module-Path: /usr/local/lib/syslog-ng Available-Modules: date,json-plugin,syslogformat,add-contextual-data,affile,afprog,confgen,cryptofuncs,pseudofile,sdjournal,csvparser,graphite,cef,afmongodb,basicfuncs,system-source,afuser,dbparser,afstomp,afsocket,disk-buffer,kvformat,afamqp,linux-kmsg-format Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off -- Thanks, Vijay Amrut.

2 2

Windows Events multi-line
by Smith, Paul (Sr. Admin-InfoSec) 25 Jul '17

25 Jul '17

Hi Nagy, Thanks for checking that. That was probably one of the reasons it did not work. That is too bad since Windows servers using the syslog-ng agent are the source. I would have thought that would be working since bot products are from Balabit. Windows logs also send a lot of garbage with Information lines that it would also be good to remove since that is just going to fill up the log files with junk. I am getting 8-10 gigs per day from a single Windows Domain Controller and I need to add in about 25 more. The log file would be immense. I tested out using the syslog driver and was poking around with the multi-line-prefix and multi-line-garbage and they don't give any syntax errors, as long as I don't use the multi-line-mode option. Does syslog-ng ignore these options if the multi-line-mode option is not specified? Basically, do all the multi-line-xxx options not work like this? It seems like the only option that works for syslog or network is the flags(no-multi-line) flag. Is that the case? Are there any other alternatives? I would hate to write the syslogs to a file and then parse that file again to get the logs I want. That is double work for the server. But that seems like that might be the only option I have. Hmmm...maybe I can log to a VM syslog-ng and have that one parse the data first using the file driver and then I can relay to my main systems. Paul C Smith -----Original Message----- From: syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of syslog-ng-request(a)lists.balabit.hu Sent: Thursday, July 13, 2017 7:00 AM To: syslog-ng(a)lists.balabit.hu Subject: syslog-ng Digest, Vol 147, Issue 14 Send syslog-ng mailing list submissions to syslog-ng(a)lists.balabit.hu To subscribe or unsubscribe via the World Wide Web, visit https://lists.balabit.hu/mailman/listinfo/syslog-ng or, via email, send a message with subject or body 'help' to syslog-ng-request(a)lists.balabit.hu You can reach the person managing the list at syslog-ng-owner(a)lists.balabit.hu When replying, please edit your Subject line so it is more specific than "Re: Contents of syslog-ng digest..." Today's Topics: 1. Re: Windows Events multi-line (Nagy) ---------------------------------------------------------------------- Message: 1 Date: Thu, 13 Jul 2017 10:50:03 +0200 From: Nagy, Gábor <gabor.nagy(a)balabit.com> To: "Syslog-ng users' and developers' mailing list" <syslog-ng(a)lists.balabit.hu> Subject: Re: [syslog-ng] Windows Events multi-line Message-ID: <CAETAYnCft3KAyWF_5pB3g4+jodrn6+SpXTYJqqyfnFoUFzzaEw(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hello, Checking your configuration I see that you are using multi-line mode for a network source driver. Unfortunately, currently this is not supported, only the file() and pipe() source drivers support multi-line messages. I have checked the documentation and found that although it is mentioned which source drivers support multi-line mode (under flags option "no-multi-line"), the multi-line-* options are included in the network source driver options page. This is a documentation bug, which will be fixed soon. Sorry for the inconveniences. Best Regards, Gabor On Mon, Jul 10, 2017 at 8:15 PM, Smith, Paul (Sr. Admin-InfoSec) < Paul.C.Smith(a)snapon.com> wrote: > Hello, > > > > I’m having problems getting Windows events on a single line on > syslog-ng OSE. I’ve scoured the interwebs and not found what I need to > get this exact. I am guessing this is not an uncommon problem but I > can’t seem to find quite what I need. I am guessing I am just missing > some simple thing here. > > > > Here are my details. > > > > Using syslog-ng OSE 3.9.1 > > > > Have syslog-ng Windows Agent 6.0.6 running on Windows 2012 server > > > > Have a tcp source and that writes direct to the log file. Works fine > with no options set. > > > > Getting multiple lines per event. > > > > I’ve added what I think are the correct settings for multi-line, it > does not work. I don’t think it is the regex syntax, but something else? > > > > ___________ > > Syslog.ng.conf > > > > source s_nettcp_win { network(ip(x.x.x.x) port(601) transport("tcp") > > multi-line-mode(regexp) > > multi-line-prefix("^[0-9]{3,5} > \s<[0-9]{2,3}>1\s([0-9]{4})-(0[1-9]|1[0-2])-(0[1-9]|[1-2][ > 0-9]|3[0-1])T([0-9]{2}:[0-9]{2}:[0-9]{2}-[0-9]{2}:[0-9]{2})") > > flags(no-parse)); }; > > > > ___________ > > Here is the error when trying to start syslog-ng or run syslog-ng –s: > > > > Error parsing afsocket, syntax error, unexpected LL_IDENTIFIER, > expecting KW_NORMALIZE_HOSTNAMES or KW_USE_DNS or KW_USE_FQDN or > KW_DNS_CACHE in /etc/syslog-ng/syslog-ng.conf at line 38, column 76: > > > > source s_nettcp_win { network(ip(x.x.x.x) port(601) transport("tcp") > multi-line-mode(regexp) multi-line-prefix("^[0-9]{3,5} > \s<[0-9]{2,3}>1\s([0-9]{4})-(0[1-9]|1[0-2])-(0[1-9]|[1-2][ > 0-9]|3[0-1])T([0-9]{2}:[0-9]{2}:[0-9]{2}-[0-9]{2}:[0-9]{2})") > flags(no-parse)); }; > > > ^^^^^^^^^^^^^^^ > > ___________ > > Sample of the event log part I am matching regex on: > > > > Jul 10 12:11:19 x.x.x.x 912 <133>1 2017-07-10T12:11:18-05:00 > computername Microsoft_Windows_security_auditing. 6260 - [win(a)18372.4 > EVENT_CATEGORY="Logoff" EVENT_FACILITY="16" EVENT_ID="4634" EVENT_LEVEL="0" > EVENT_NAME="Security" EVENT_REC_NUM="573516592" EVENT_SID="N/A" > EVENT_SOURCE="Microsoft Windows security auditing." EVENT_TASK="Logoff" > EVENT_TYPE="Success Audit" EVENT_USERNAME="domain\\userid"][meta > sequenceId="10817278" sysUpTime="-198876"] domain\userid: Security > Microsoft Windows security auditing.: [Success Audit] An account was > logged off. > > > > ____________________________________________________________ > __________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/? > product=syslog-ng > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170713/826dd963/a…> ------------------------------ Subject: Digest Footer _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng ------------------------------ End of syslog-ng Digest, Vol 147, Issue 14 ******************************************

2 1

positional arguments for blocks
by Fabien Wernli 21 Jul '17

21 Jul '17

Hi, Many native blocks have positional arguments, e.g. network(): network( "127.0.0.1", … ) Is it possible to set positional arguments to custom blocks?

2 1

Re: [syslog-ng] syslog-ng 3.10 fedora/rhel/centos packages
by alex 19 Jul '17

19 Jul '17

Hello, this is taken from syslog-ng 3.8.1 change log: "Minimal version of libdbi is set to 0.9.0" Correct me if I am wrong, but latest version of libdbi on CentOS 6.X is 0.8.3. How is syslog-ng 3.9 the latest compatible version with older CentOS distro? *From:* syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu] *On Behalf Of *Czanik, Péter *Sent:* Wednesday, July 19, 2017 1:42 PM *To:* Syslog-ng users' and developers' mailing list < syslog-ng(a)lists.balabit.hu> *Subject:* Re: [syslog-ng] syslog-ng 3.10 fedora/rhel/centos packages Just tested, 3.10.1 does not compile on RHEL 6 (well, in practice the latest CentOS 6.X). It fails with: CCLD modules/dbparser/pdbtool/pdbtool ./lib/.libs/libsyslog-ng.so: undefined reference to `g_atomic_pointer_add' collect2: ld returned 1 exit status make[2]: *** [modules/dbparser/pdbtool/pdbtool] Error 1 make[1]: *** [all-recursive] Error 1 So, currently 3.9 is the latest syslog-ng compiling on old distros. Making sure, that syslog-ng compiles on old distros is on the roadmap of the team. Compiling on FreeBSD was fixed right after the 3.10 release, I hope RHEL 6 / SLES 11 will also compile again soon. Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik On Mon, Jul 17, 2017 at 6:47 PM, Czanik, Péter <peter.czanik(a)balabit.com> wrote: Hi, I'll check RHEL 6 later this week. Personally I tested on SLES 11, as (open)SUSE is my primary testing platform. RHEL 6 might work, at least the later versions, as glib was updated. See https://github.com/balabit/ syslog-ng/issues/1403 for some background info. Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik On Mon, Jul 17, 2017 at 5:12 PM, Evan Rempel <erempel(a)uvic.ca> wrote: RHEL 6 still has 2 years in the normal support lifecycle and even more if a extended support is purchased. IMHO syslog-ng should strive to have a current release that still compiles on these platforms until at least the end of standard support (2020). Evan. On 07/17/2017 07:36 AM, Czanik, Péter wrote: hi, Until now my unofficial syslog-ng 3.10 packages for Fedora / RHEL 7 / CentOS 7 were available from my "githead" repo. Today I created a dedicated 3.10 repository, as I plan to use the githead repo for creating builds from git HEAD again. It is available at https://copr.fedorainfracloud. org/coprs/czanik/syslog-ng310/ Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik

2 1

syslog-ng 3.10 fedora/rhel/centos packages
by Czanik, Péter 19 Jul '17

19 Jul '17

hi, Until now my unofficial syslog-ng 3.10 packages for Fedora / RHEL 7 / CentOS 7 were available from my "githead" repo. Today I created a dedicated 3.10 repository, as I plan to use the githead repo for creating builds from git HEAD again. It is available at https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng310/ Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik

2 3

Java Stack traces and the dreaded multi-line
by Clayton Dukes 17 Jul '17

17 Jul '17

Does anyone have good config examples for parsing incoming Java events from both Network and Files? [cid:image001.png@01D2FEE8.2D247F00]

1 0

in-list syntax
by Damian Bell 17 Jul '17

17 Jul '17

Hello, I’ve previously had a very simple regex list configured for Cisco ASA devices to deliver email alerts on routing changes in our network, which was configured thus:- filter f_sev1 { match("%ASA-3-622001"); }; …which works just fine. Wanting to expand this list a bit, and aware that scaling regex matching up is going to incur performance issues, I’ve thought that an “in-list” expression file might have been a better approach, and as such have configured the following:- filter f_sev1 { in-list("/etc/syslog-ng/email-match-list.list", value("PROGRAM")); }; …where “PROGRAM” is (I believe) the applicable part of the code that matches to the Cisco "%ASA-3-622001" part of the message. The whitelist has the following entries (as an example):- %DUAL-5-NBRCHANGE %ASA-3-622001 %SPANTREE-5-TOPOTRAP %SPANTREE-5-ROOTCHANGE %ASA-5-111010 I am not getting any matches here, however. What am I missing? Thanks very much in advance, Damian Damian Bell Infrastructure Engineer | Support | H Clarkson & Co Ltd T: +44 20 7334 5483 Email: Damian.Bell(a)clarksons.com<mailto:Damian.Bell@clarksons.com> Group Email: infrastructure(a)clarksons.com Clarksons Platou TM Commodity Quay, St. Katharine Docks | London E1W 1BF | United Kingdom www.clarksons.com<http://www.clarksons.com> Please consider the environment before printing this e-mail ________________________________ This message is private and confidential. If you have received it in error, you are on notice of its status. Please notify us immediately by reply email and then delete this message from your system. Please do not copy it or use it for any purposes, or disclose its contents to any other person: to do so could be a breach of confidence. Emails may be monitored. Details of Clarkson group companies and their regulators (where applicable) can be found at this url: Disclosure<http://www.clarksons.com/disclosure/> ________________________________

3 2

rpm to do upgrade on isolated network centos6 server?
by craig bowser 17 Jul '17

17 Jul '17

I'm sure this has been asked before, but I have not yet found a reference or faq so if someone can point me there, I'd appreciate it... After some googling and searching the email list history, I have not found a way to download the actual rpm for syslog-ng so I can burn it to disc and install it on a server on an isolated network. The page below indicates that getting actual rpm files only come with the commercial version and other than that you need to download from the repo? https://www.balabit.com/blog/rpm-packages-from-syslog-ng-git-head/ Also, I didn't see any current rpms on on rpmfind.net for centOS6..... So I assume I have to download the source, move it over, then install from that source? If that's the case, is there a list of other dependencies I need? Did I miss a link or instruction? Thanks. Craig L Bowser ____________________________ This email is measured by size. Bits and bytes may have settled during transport.

4 4

Windows Events multi-line
by Smith, Paul (Sr. Admin-InfoSec) 13 Jul '17

13 Jul '17

Hello, I'm having problems getting Windows events on a single line on syslog-ng OSE. I've scoured the interwebs and not found what I need to get this exact. I am guessing this is not an uncommon problem but I can't seem to find quite what I need. I am guessing I am just missing some simple thing here. Here are my details. Using syslog-ng OSE 3.9.1 Have syslog-ng Windows Agent 6.0.6 running on Windows 2012 server Have a tcp source and that writes direct to the log file. Works fine with no options set. Getting multiple lines per event. I've added what I think are the correct settings for multi-line, it does not work. I don't think it is the regex syntax, but something else? ___________ Syslog.ng.conf source s_nettcp_win { network(ip(x.x.x.x) port(601) transport("tcp") multi-line-mode(regexp) multi-line-prefix("^[0-9]{3,5}\s<[0-9]{2,3}>1\s([0-9]{4})-(0[1-9]|1[0-2])-(0[1-9]|[1-2][0-9]|3[0-1])T([0-9]{2}:[0-9]{2}:[0-9]{2}-[0-9]{2}:[0-9]{2})") flags(no-parse)); }; ___________ Here is the error when trying to start syslog-ng or run syslog-ng -s: Error parsing afsocket, syntax error, unexpected LL_IDENTIFIER, expecting KW_NORMALIZE_HOSTNAMES or KW_USE_DNS or KW_USE_FQDN or KW_DNS_CACHE in /etc/syslog-ng/syslog-ng.conf at line 38, column 76: source s_nettcp_win { network(ip(x.x.x.x) port(601) transport("tcp") multi-line-mode(regexp) multi-line-prefix("^[0-9]{3,5}\s<[0-9]{2,3}>1\s([0-9]{4})-(0[1-9]|1[0-2])-(0[1-9]|[1-2][0-9]|3[0-1])T([0-9]{2}:[0-9]{2}:[0-9]{2}-[0-9]{2}:[0-9]{2})") flags(no-parse)); }; ^^^^^^^^^^^^^^^ ___________ Sample of the event log part I am matching regex on: Jul 10 12:11:19 x.x.x.x 912 <133>1 2017-07-10T12:11:18-05:00 computername Microsoft_Windows_security_auditing. 6260 - [win(a)18372.4 EVENT_CATEGORY="Logoff" EVENT_FACILITY="16" EVENT_ID="4634" EVENT_LEVEL="0" EVENT_NAME="Security" EVENT_REC_NUM="573516592" EVENT_SID="N/A" EVENT_SOURCE="Microsoft Windows security auditing." EVENT_TASK="Logoff" EVENT_TYPE="Success Audit" EVENT_USERNAME="domain\\userid"][meta sequenceId="10817278" sysUpTime="-198876"] domain\userid: Security Microsoft Windows security auditing.: [Success Audit] An account was logged off.

2 1

Problems with set() / condition() ?
by Jorge Pereira 13 Jul '17

13 Jul '17

Hi List, I am trying to use the below set() + condition() rewrite { set("$(* $(/ $R_MIN 5) 5)", value("min_slice")); set("0${min_slice}", value("min5"), condition("min_slice" <= "9")); set("${min_slice}", value("min5"), condition("min_slice" >= "10")); # final macro, e.g: www.mytest.com_20170417-1001PM-ng_lua.log.gz set("${YEAR}${MONTH}${DAY}-${HOUR12}${min5}${AMPM}", value("logfmt_file")); set("${YEAR}${MONTH}${DAY}/${HOUR12}${min5}${AMPM}", value("logfmt_dir")); }; But, does not work with R_MIN values around 1 ~ 9 then never execute the second set(... *condition("min_slice" <= "9")*) statement. Has somebody idea? p.s: I am using the syslog-ng 3.7.3 -- Jorge Pereira

2 3