October 2017 - syslog-ng - lists.balabit.hu

collecting SAR data
by Jim Hendrick 17 Oct '17

17 Oct '17

Hi, Does anyone have experience pulling performance data from sysstat (sar) with syslog-ng (or even *cough* rsyslog...) Looking to automate the performance metrics and it seems like it should be straightforward. What I would like is to pull the text output that is generated every ~10 minutes by default without having to run a different cron across all the systems. (that is - I get that I could probably modify the "sa1" script or write another one to dump the data to a socket or something for syslog to pick up. I am hoping to just do something to monitor the data that is automatically generated. - something like using any file in /var/log/sa/sar?? as a source) The end system (for me at the moment) would be splunk so parsing the different format data *could* be pushed to that end or could be done with patterndb or something else on the syslog-ng side. Thoughts? Thanks in advance!! Jim

1 0

Re: [syslog-ng] [syslog
by james.r.hendrick 17 Oct '17

17 Oct '17

Do you mean the internal stats facility?I would look into the elasticsearch native destination if you are not already. Jim Sent from my Verizon, Samsung Galaxy smartphone -------- Original message --------From: Evan Rempel <erempel(a)uvic.ca> Date: 10/16/17 11:01 AM (GMT-05:00) To: syslog-ng(a)lists.balabit.hu Subject: Re: [syslog I have a perl script that collects some stats and logs them to syslog again. The syslog stream gets sent to ES, so they end up there, but as a syslog line, not a specific statistic item for things like grafana. On 10/15/2017 05:57 PM, Scot wrote: > Hi, > > Looked around for a few hours and didn't see anything. > > Has anyone worked on sending syslog-ng stats to ES ? > I see several ways I could but wondering if anyone has already. A push method directly from syslog-ng would be awesome. > > Scot > ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

3 2

Re: [syslog-ng] [syslog
by james.r.hendrick 17 Oct '17

17 Oct '17

Yeah, probably separate source / dest and maybe parsed for better formatting.Send it in json perhaps. Sent from my Verizon, Samsung Galaxy smartphone -------- Original message --------From: Scot <scotrn(a)gmail.com> Date: 10/16/17 6:24 PM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Subject: Re: [syslog-ng] [syslog Yep but how do you get that to a d_es destination without adding an external tool ? My understanding of the purpose is that stats may stop getting updated if the pipeline fails so It needs to be a separate "thread" but seems like that would be handled in separate src and destination definitions ? i.e. Keep your stats/monitoring separate from the data stream. On Mon, Oct 16, 2017 at 11:43 AM, james.r.hendrick <james.r.hendrick(a)gmail.com> wrote: Do you mean the internal stats facility?I would look into the elasticsearch native destination if you are not already. Jim Sent from my Verizon, Samsung Galaxy smartphone -------- Original message --------From: Evan Rempel <erempel(a)uvic.ca> Date: 10/16/17 11:01 AM (GMT-05:00) To: syslog-ng(a)lists.balabit.hu Subject: Re: [syslog I have a perl script that collects some stats and logs them to syslog again. The syslog stream gets sent to ES, so they end up there, but as a syslog line, not a specific statistic item for things like grafana. On 10/15/2017 05:57 PM, Scot wrote: > Hi, > > Looked around for a few hours and didn't see anything. > > Has anyone worked on sending syslog-ng stats to ES ? > I see several ways I could but wondering if anyone has already. A push method directly from syslog-ng would be awesome. > > Scot > ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

1 0

$FULLHOST doesn't work
by Thomas Haupt 16 Oct '17

16 Oct '17

Hi, I'm trying to use the hostname from inside the syslog message as filename but it doesn't work for me: https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-… my config: @version: 3.9 options { keep_hostname(yes); use-dns(no); }; source s_system { udp(ip("0.0.0.0") port(514)); }; destination d_all { file("/tmp/${FULLHOST}.log"); }; log { source(s_system); destination(d_all); }; The result is my ip address instead of "my-hostname", but I expect "my-hostname": syslog-ng -Fevd .... [2017-10-16T10:32:15.188058] Incoming log entry; line='<14>1 2017-10-16T10:32:15.151157+02:00 my-hostname lampelogtest 54321 111111 - Test syslog message' [2017-10-16T10:32:15.188327] Initializing destination file writer; template='/tmp/${FULLHOST}.log', filename='/tmp/10.3.25.4.log' Some more information: syslog-ng --version syslog-ng 3.9.1 Installer-Version: 3.9.1 Revision: Module-Directory: /usr/lib64/syslog-ng Module-Path: /usr/lib64/syslog-ng Available-Modules: add-contextual-data,afamqp,affile,afprog,afsocket,afstomp,afuser,basicfuncs,cef,confgen,cryptofuncs,csvparser,date,dbparser,disk-buffer,graphite,json-plugin,kvformat,linux-kmsg-format,pseudofile,syslogformat,system-source,tfgetent Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: on Enable-Linux-Caps: off The system is a SLES11SP4 Regards, Thomas

4 4

in-list filter with MESSAGE
by Gopi Joshi 15 Oct '17

15 Oct '17

I am trying to filter messages matching text stored in a txt file (plain txt , exact match , one word each line). but its not working filter f_userlist { in-list("/etc/syslog-ng/userlist.list", value("MESSAGE")); }; ---> NOT WORKING however it works with value(“PROGRAM”) filter f_whitelist { in-list("/etc/syslog-ng/programlist.list", value("PROGRAM")); }; --->WORKING List ( userlist.list ) is not long and has less than 10 words to match. anything missing ? or in-list filter doenot work with message contents . any troubleshooting tips will e helpful.

4 4

unofficial syslog-ng packages: OBS status
by Laszlo Budai 14 Oct '17

14 Oct '17

Hi, if you get 'connection failed' error while you are trying to access the OBS repository, please check status of OBS : https://status.opensuse.org/ openSUSE status page<https://status.opensuse.org/> Stay up to date with the latest service updates from openSUSE status page. status.opensuse.org regards, Laszlo Budai

1 0

Questions regarding support for syslog-ng
by Diana Wiener 12 Oct '17

12 Oct '17

Hi I'm collecting information on various dependencies within infrastructure for internal tracking. I looked on the website and through your documentation and cannot locate the answers needed, so I am reaching out to the mailing list. I was wondering if you can give me any sense of what your cadence for releasing updates for syslog-ng. - Do you have a routine release cadence for updates? If so, what is it (e.g. monthly, every third Tuesday, etc.) If there is no set cadence, can you give me a rough sense of how often you release updates? - How do you deal with potential security vulnerabilities? What does your patching procedure look like? Is it typically visible to users? I am sure you can't give me a ton of detail, but we'd like to be able to document what our dependencies look like and how often we might be falling behind so we can adjust our own roadmaps accordingly. Thanks in advance for any information you can offer! Diana -- Diana Wiener Customer Life Cycle Manager, Support diana.wiener(a)acquia.com

2 1

Trouble installing latest syslog-ng from Laszlo
by Clayton Dukes 06 Oct '17

06 Oct '17

Hi Guys, I'm wondering if I'm doing something wrong or if there's a package issue in the latest deb release. root@estreamer:~# cat /etc/apt/sources.list.d/syslog-ng.list deb http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/xUb… ./<http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/xUb…> root@estreamer:~# apt-get install syslog-ng Reading package lists... Done Building dependency tree Reading state information... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: syslog-ng : Depends: syslog-ng-mod-sql but it is not going to be installed Depends: syslog-ng-mod-mongodb but it is not going to be installed Depends: syslog-ng-mod-json but it is not going to be installed Recommends: syslog-ng-mod-mysql but it is not going to be installed Recommends: syslog-ng-mod-pgsql but it is not going to be installed Recommends: syslog-ng-mod-sqlite3 but it is not going to be installed Recommends: syslog-ng-mod-freetds but it is not going to be installed Recommends: syslog-ng-mod-smtp but it is not going to be installed Recommends: syslog-ng-mod-amqp but it is not going to be installed Recommends: syslog-ng-mod-geoip but it is not going to be installed Recommends: syslog-ng-mod-redis but it is not going to be installed Recommends: syslog-ng-mod-stomp but it is not going to be installed Recommends: syslog-ng-mod-riemann but it is not going to be installed Recommends: syslog-ng-mod-graphite but it is not going to be installed Recommends: syslog-ng-mod-python but it is not going to be installed Recommends: syslog-ng-mod-add-contextual-data but it is not going to be installed Recommends: syslog-ng-mod-getent but it is not going to be installed Recommends: syslog-ng-mod-stardate but it is not going to be installed Recommends: syslog-ng-mod-map-value-pairs but it is not going to be installed Recommends: syslog-ng-mod-snmptrapd-parser but it is not going to be installed Recommends: syslog-ng-mod-geoip2 but it is not going to be installed Recommends: syslog-ng-mod-pacctformat but it is not going to be installed Recommends: syslog-ng-mod-elastic but it is not going to be installed Recommends: syslog-ng-mod-http but it is not going to be installed E: Unable to correct problems, you have held broken packages. [id:image001.png@01D306E3.0FEBC990] Clayton Dukes Founder & CEO LogZilla Corporation 2900 N. Quinlan Park Rd, B240-341 Austin, TX, 78732 Tel: 936-4NetOps (463-8677) Web: www.logzilla.net<http://www.logzilla.net/> [id:image002.png@01D306E3.0FEBC990]<https://twitter.com/logzilla>[id:image003.png@01D306E3.0FEBC990]<https://youtu.be/drg5wv_mgfA>[id:image004.png@01D306E3.0FEBC990]<https://www.linkedin.com/in/lzcdukes/> For NetOps, By NetOps!

3 5

cross compiling syslog-ng
by Scheidler, Balázs 05 Oct '17

05 Oct '17

hi, I've figured I would give syslog-ng cross compilation a spin to check if everything is all right with our sources. Here's a description on how I've cross-compiled syslog-ng to arm64. *1) toolchain* I've used the docker based cross-compile toolchain from here: https://github.com/dockcross/dockcross You'll need docker on your build machine, see docker.com for intallation instructions. To use that, you'll only need to launch a docker command line and it would automatically fetch the image from the docker registry. I've used linux-arm64 as an example, but I guess the others would also work. *2) sources* You will need the release tarball. Please note that you can download both the release tarball and a git snaphot from github, but you'll definitely need the release tarball if you don't have all the build requirements installed (automake, libtool and friends). I've used https://github.com/balabit/syslog-ng/releases/download/syslog-ng-3.12.1/sys… 3) get the dockcross helper ``` $ docker run --rm dockcross/linux-arm64 > dockcross $ chmod +x dockcross ``` *4) launch the cross-compile environment* Extract the sources (e.g. tar xvfz syslog-ng-3.12.1.tar.gz) into a directory, cd into it and launch the environment: ``` $ dockcross bash ``` This will give you a shell within the container, with a /work directory containing the sources. *5) install dependencies* ``` # this one fixes an issue within the container, "cpp" is unfortunately only available # with a versioned binary, which the configure script does not find. $ sudo ln -s /usr/bin/aarch64-linux-gnu-cpp-4.9 /usr/bin/aarch64-linux-gnu-cpp # this is the minimal set of dependencies to compile syslog-ng, you might need others # the bundled sources should however give you a pretty ok feature set $ sudo apt-get install libglib2.0-dev:arm64 pkg-config:arm64 ``` *6) build* ``` $ mkdir build $ cd build $ /work/configure --host aarch64-linux-gnu <long configure output> $ make ``` *7) install* ``` $ make install DESTDIR=/work/build/install ``` The resulting set of binaries you will find in /work/build/install, as if it was installed in the root system (e.g. /usr/local/bin/syslog-ng would become /work/build/install/usr/local/bin/syslog-ng) *8) collect binaries* Exit the docker container, your build products will be in the "build" subdirectory of your source tree, with build/install containing the final binaries. bazsi@bzorp:~/src/syslog-ng-3.12.1/build/install$ file usr/local/sbin/syslog-ng usr/local/sbin/syslog-ng: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=424c7a32112262725b267e6043c25bd40d039568, not stripped I hope this helps. -- Bazsi On Thu, Oct 5, 2017 at 10:37 AM, Scheidler, Balázs < balazs.scheidler(a)balabit.com> wrote: > Hi, > > This might actually be a libtool issue, trying to locate the file in an > invalid location. Do you have libtool in your toolchain? In order to use > the one in your toolchain, you will need the complete set of autotools and > rerun autogen.sh to regenerate all build files. > > BTW: the AX_PREFIX_CONFIG_H macro is also part of autoconf-archive ( > https://www.gnu.org/software/autoconf-archive/ax_prefix_config_h.html) > but probably a later version than what you have. > > But you can always add these macros by downloading them explicitly and > adding them to the m4 subdirectory. > > -- > Bazsi > > On Wed, Oct 4, 2017 at 4:35 PM, Benoit Duffez <notifications(a)github.com> > wrote: > >> I have not solved it. It accepts to compile but the linker seems to fail: >> >> CCLD lib/libsyslog-ng.la >> libtool: link: warning: library `/opt/toolchains/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/lib/libgmodule-2.0.la' was moved. >> libtool: link: warning: library `/opt/toolchains/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/lib/libgthread-2.0.la' was moved. >> libtool: link: warning: library `/opt/toolchains/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/lib/libglib-2.0.la' was moved. >> libtool: link: warning: library `/opt/toolchains/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/lib/libpcre.la' was moved. >> libtool: link: warning: library `/opt/toolchains/sysroots/cortexa9hf-neon-poky-linux-gnueabi/usr/lib/libuuid.la' was moved. >> sed: can't read /usr/lib/libglib-2.0.la: No such file or directory >> libtool: link: `=/usr/lib/libglib-2.0.la' is not a valid libtool archive >> Makefile:7092: recipe for target 'lib/libsyslog-ng.la' failed >> make[2]: *** [lib/libsyslog-ng.la] Error 1 >> Makefile:16376: recipe for target 'all-recursive' failed >> make[1]: *** [all-recursive] Error 1 >> Makefile:6285: recipe for target 'all' failed >> make: *** [all] Error 2 >> >> I think it's a PATH issue because I do have this libglib-2.0.la file, >> just not at /usr/bin (but in a subfolder of /opt/toolchains) >> >> — >> You are receiving this because you are subscribed to this thread. >> Reply to this email directly, view it on GitHub >> <https://github.com/balabit/syslog-ng/issues/1705#issuecomment-334176270>, >> or mute the thread >> <https://github.com/notifications/unsubscribe-auth/AArldnMjDwe5Mfaj6m3FPTfid…> >> . >> > >

2 1

Repository key issue
by Sandbox 05 Oct '17

05 Oct '17

Hai, Are you know your key has some error since ~13:00: GPG error: http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/xUb… ./ BADSIG 3B92A8D27CFDAEDD home:laszlo_budai:syslog-ng OBS Project < home:laszlo_budai:syslog-ng@build.opensuse.org> Regards, Robert

1 0