- syslog-ng - lists.balabit.hu

C_* macros with UTC time?
by Matus UHLAR - fantomas 04 Feb '20

04 Feb '20

Hello, I have syslog-ng OSE 3.8 on debian 9 server (soon to be upgraded). when I receive syslog lines using the syslog protocol, I found out that C_* macros seem to contain date in UTC format (one hour than our current zone). I have tried to use following template: "${C_DATE} ${R_DATE} HOST ${HOST} FULLHOST ${FULLHOST} HOST_FROM ${HOST_FROM} FULLHOST_FROM ${FULLHOST_FROM} ${MESSAGE}\n" with following result: Jan 31 10:42:06 Jan 31 11:42:07 HOST fantomas FULLHOST fantomas HOST_FROM 195.80.174.185 FULLHOST_FROM 195.80.174.185 Configuration reload request received, reloading configuration; Is there any reason why C_DATE should be in different time zone than R_DATE? timezones are not configured on either of servers, both servers are in CET (DST+1) thank you -- Matus UHLAR - fantomas, uhlar(a)fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Fighting for peace is like fucking for virginity...

2 4

unsubscribe
by Zhang, Husen 03 Feb '20

03 Feb '20

-----Original Message----- From: syslog-ng <syslog-ng-bounces(a)lists.balabit.hu> On Behalf Of Peter Kokai (pkokai) Sent: Monday, February 3, 2020 7:33 AM To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Subject: EXTERNAL: Re: [syslog-ng] Not finding header files in syslog-ng-3.25.1 as compared to syslog-ng-3.6.2 Hello, There is no dev doc about it, as you might noticed. Probably no replace that with this answer exists. Luckily git remembers: git log --follow -p -- <path> git log --follow -p -- modules/dbparser/patterndb-int.h commit 1931dabf07b79a9b10d5902b5ad0b7f0509e7594 Author: Balazs Scheidler <bazsi77(a)gmail.com> Date: Wed Jul 29 15:10:06 2015 +0200 dbparser: get rid off patterndb-int.h As we have the broken-down header files, this is not needed anymore. Signed-off-by: Balazs Scheidler <balazs.scheidler(a)balabit.com> This commit might gave you a hint, which headers it was replaced with. Alternative idea: simply check which header provides the function you need. git log --follow -p -- src/tags.h commit ee1d0d7f0720c3ded0d2bfaf22ca6e42d669b7a6 Author: Balazs Scheidler <bazsi77(a)gmail.com> Date: Thu Sep 9 13:48:42 2010 +0200 separate directory hierarchy into lib/, modules/ and syslog-ng/ It was more and more cumbersome to compile syslog-ng during development, since a core change always recompiled (or just relinked) the modules as well even when that was not strictly necessary. (make install in src/) This patch reorganizes the directory hierarchy, lib is where the core lives modules where plugins are and syslog-ng is where the main program resides. The change here more simple, just moving the actual file. (Note tags.h was moved twice since your version, and now it is under lib/logmsg/) -- Kokan On Mon, Feb 03, 2020 at 05:34:21PM +0530, Nitish Saboo wrote: > CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. > > Hi, > > I was using syslog version: syslog-ng-3.6.2. > In that particular version of my c wrapper code I was including the following header files: > > #include <modules/dbparser/patterndb-int.h> > #include <syslog-ng/tags.h> > > When I upgraded the syslog-ng version to 3.25.1,I don't see these > header file in the modules and include folder respectively post > installation > > Can someone please help me clarify, what these files have been replaced with? > > Thanks, > Nitish > ______________________________________________________________________ > ________ Member info: > https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist > s.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7Cpeter > .kokai%40oneidentity.com%7C0616c4f0e4cc4cd1d26f08d7a8a13d03%7C91c369b5 > 1c9e439c989c1867ec606603%7C0%7C0%7C637163282794906164&sdata=mn27bw > mBoTLzPsp%2FeL648GZ81vo%2FfZMrfAeNdfwX5bk%3D&reserved=0 > Documentation: > https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.b > alabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data > =02%7C01%7Cpeter.kokai%40oneidentity.com%7C0616c4f0e4cc4cd1d26f08d7a8a > 13d03%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637163282794916099& > amp;sdata=W5ip5K3TXAoux%2BBRNPoJpw19tQoxaSHF0nkLMc37%2BFQ%3D&reser > ved=0 > FAQ: > https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.b > alabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7Cpeter.kokai%40one > identity.com%7C0616c4f0e4cc4cd1d26f08d7a8a13d03%7C91c369b51c9e439c989c > 1867ec606603%7C0%7C0%7C637163282794916099&sdata=UZs%2FG%2Fg%2BoUES > RozPNJOfyUwDKYpB5gzQ9jKc4XysbSY%3D&reserved=0 > ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

1 0

Not finding header files in syslog-ng-3.25.1 as compared to syslog-ng-3.6.2
by Nitish Saboo 03 Feb '20

03 Feb '20

Hi, I was using syslog version: syslog-ng-3.6.2. In that particular version of my c wrapper code I was including the following header files: #include <modules/dbparser/patterndb-int.h> #include <syslog-ng/tags.h> When I upgraded the syslog-ng version to 3.25.1,I don't see these header file in the modules and include folder respectively post installation Can someone please help me clarify, what these files have been replaced with? Thanks, Nitish

3 2

Memory not freeing up in syslog-ng-3.6.2 during reload
by Nitish Saboo 29 Jan '20

29 Jan '20

Hi, I am using syslog-ng version 3.6.2: syslog-ng --version syslog-ng 3.6.2 Installer-Version: 3.6.2 Revision: Compile-Date: May 2 2019 01:20:32 Available-Modules: afmongodb,afstomp,afuser,csvparser,afprog,afamqp,cryptofuncs,afsocket,graphite,dbparser,affile,syslogformat,afsocket-notls,system-source,confgen,pseudofile,linux-kmsg-format,basicfuncs Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off I initialised syslog-ng with the following configuration: int initialize_engine(const gchar* filename, const gchar* _module_path) { module_path = _module_path; app_startup(); msg_init(TRUE); configuration = cfg_new(0x0302); plugin_load_module("basicfuncs", configuration, NULL); plugin_load_module("syslogformat", configuration, NULL); pattern_db_global_init(); return 0; } int reload_pattern_db(const gchar* filename, key_value_cb cb) { if(patterndb!= NULL){ pattern_db_free(patterndb); } patterndb = pattern_db_new(); pattern_db_reload_ruleset(patterndb, configuration, filename); pattern_db_set_emit_func(patterndb, pdbtool_pdb_emit_accumulate, cb); return 0; } When I reload the syslog-ng, the memory keeps on increasing.Looks like syslog-ng is not freeing up the memory.It continues to occupy the memory unless the process is restarted. Just want to understand if it is a sign of memory leak or default behavior of syslog-ng to not free-up memory during reload ? Thanks, Nitish

2 15

which platform do you use for syslog-ng?
by Peter Czanik (pczanik) 28 Jan '20

28 Jan '20

Hi, I am often asked on which platforms people are running syslog-ng. Based on mailing list questions, github issues and so on I have some guesses, but every once in a while I run a Twitter poll: https://twitter.com/PCzanik/status/1222089870782484480 Please answer it and share it, so more people can answer it! Thanks, Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

facility override
by Alexandre Santos 20 Jan '20

20 Jan '20

Hi, Is there a way of overriding the facility of the messages on the destination, besides using a template like: { template("$(if ('${LEVEL_NUM}' == '2') '<186>1' $(if ('${LEVEL_NUM}' == '3') '<187>1' $(if ('${LEVEL_NUM}' == '4') '<188>1' ${HOST}\\@`HOSTIPV4` ${PROGRAM} ${PID} ${SDATA} ${MESSAGE}\n"); }; Thanks in advance, Alex

2 1

Rewrite set var value with tag?
by Faine, Mark R. (MSFC-IS40)[NICS] 20 Jan '20

20 Jan '20

Is it possible to do something like: rewrite { set-tag("bar"); }; And then use a rewrite to set the value of a variable to the value of the tag rewrite { set("${TAGS}", value("foo")); }; I need to set a value that I've determined in a log path at the top of the file in several log paths that occur after. It looks the scope for variables is at the log path level so that doesn't help me. I'm also trying to avoid nesting all of the other log paths inside of the top log path where I determine the value. I'm trying to avoid: log{ ... filter{ ... tags("bar"); rewrite{ set("bar", "foo"); }; }; }; log{... log{... Thanks, -Mark

3 3

Go away
by johns spam 16 Jan '20

16 Jan '20

Hello, I am wondering if there is any facility to tell a client to go away? i.e. https://http2.github.io/http2-spec/#GOAWAY I know syslog-ng is not http2 but is there any side-channel mechanism a central server can use to make a client go away? The use case I am thinking is doing a rolling restart of a pool of syslog-ng central server. I think this could also be useful to cycle connections periodically, i.e. to rebalance connections to a central server. Thanks

2 1

librdkafka and messages dropped
by John Skopis 15 Jan '20

15 Jan '20

Hello, We have a syslog-ng aggregator that accepts messages in a few different formats and logs the message format to kafka. The mapping looks something like: format1 -> topic1 format2 -> topic2 ... Each agent will emit messages in ~4 different formats and send to the aggregator. The aggregator is configured with a kafka_c destination with 8 threads/destination. We were previously using the java kafka client without issue. We recently upgraded to 3.24.1 + librdkafka 1.22 After upgrading we are seeing an issue where (seemingly randomly) syslog-ng stops producing messages into kafka. Checking syslog-ng-ctl stats I can see d_kafka_format1 has dropped message counter increasing d_kafka_format1 queued messages counter increases to 80000 (10000*8 threads iw-size) d_kafka_format1 processed messages does not increase There are messages printed to the log that the destination queue is full for all of the rdkafka threads, There are spurious failures to a kafka broker but it seems rdkafka reconnects. Seems like once the destination queue becomes full syslog-ng never recovers and the instance must be restarted. It also seems like rdkafka stops trying to reconnect to kafka (possibly?). Is this expected behavior or is there some bug around reconnecting to kafka after a spurious timeout? Just speculating here but does rdkafka stop trying to reconnect after N times? Are there any examples of destroy/create destination client after failure thresholds? Thanks

2 5

3.25 and my RPM stable repos
by Peter Czanik (pczanik) 13 Jan '20

13 Jan '20

Hi, Version 3.25 of syslog-ng was released a week ago, and it seems to be rock solid. Still, as today is my last working day this year, I feel more comfortable to push it into my stable repos only on the first week of January, when I am back. If you still want to experiment with 3.25 on openSUSE/SLES or fedora/RHEL/CentOS you can do that in the versioned repositories. Note that RHEL 8 / CentOS 8 support is now merged with the rest, it's not in a separate repository any more. suse: https://build.opensuse.org/project/show/home:czanik:syslog-ng325 red hat: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng325/ Bye, Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 1