[syslog-ng] Run Syslog NG with standard syslog client ( IDS serveur)

[Simon OBOUNOU]

Dear all 

I need to perform following architecture: 

 	* Serveur: Syslog NG
 	* client: Kind of IDS device/solution ( Stormshield)
 	* Port used: TCP/514

Problem: When I run syslog on client 

 	* none suitable directory is created on serveur
 	* I see flow arrives on serveur by using whireshark

Please, could you help me to fixe this issue 

Regards 

Simon 

Le 2014-12-18 22:58, Tibor Benke a écrit : 

> The patch seems to work, I sent a PR. 
> 
> https://github.com/balabit/syslog-ng/pull/359 [1] 
> 
> 2014-12-18 8:48 GMT+01:00 Tibor Benke <ihrwein at gmail.com>: 
> You can find the patch on this branch: 
> https://github.com/ihrwein/syslog-ng/tree/f/fix-redis-memleak [2] 
> 
> 2014-12-18 3:11 GMT+01:00 Jim Hendrick <jrhendri at roadrunner.com>: 
> 
> I will certainly try. 
> 
> Send me a link, and Thanks! 
> 
> Sent from my Verizon Wireless 4G LTE smartphone 
> 
> -------- Original message --------
> From: Tibor Benke <ihrwein at gmail.com> 
> Date:12/17/2014 5:42 PM (GMT-05:00) 
> To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu> 
> Cc: wernli at in2p3.fr, jrhendri at roadrunner.com 
> Subject: Re: [syslog-ng] syslog-ng memory usage grows 
> 
> Hi All, 
> 
> I checked your valgrind log and the redis module and I think the leak is in redis.c:137: 
> 
> redisCommand(self->c, "ping"); 
> 
> This returns a reply object, but it's ignored and never freed. 
> 
> If we make a patch on GitHub, could you test it? 
> 
> Regards, 
> Tibor 
> 
> 2014-12-17 18:36 GMT+01:00 <jrhendri at roadrunner.com>: Hi Fabien,
> 
> Hope the season is treating you well.
> 
> I ran syslog-ng for 10 minutes (recall, ~7000 EPS incoming) under valgrind and here is the output (attached).
> 
> It looks like there is some sort of leak as the process from yesterday had grown to over 16GB before I stopped it for the valgrind test.
> 
> Thanks for the help (as always).
> 
> And please let me know if I can provide any other data.
> 
> Jim
> 
> ---- Fabien Wernli <wernli at in2p3.fr> wrote:
>> Hi Jim,
>>
>> On Tue, Dec 16, 2014 at 01:24:28PM -0500, Jim Hendrick wrote:
>> > I'm not sure what I am asking, other than general advice on:
>> > - performance using patterndb
>>
>> performance is awesome, I wouldn't worry about it
>> although it would help to have some counters on the parsers in the stats
>> interface. That bieng said, it could help you to increase the stats level.
>>
>> > - performance using redis destination
>>
>> I can't comment on that one, sorry
>>
>> > - advice on debugging where this memory growth is happening
>>
>> Could you run syslog-ng inside `valgrind --tool=memcheck
>> --trace-children=yes --leak-check=yes`?
>>
>> The problem is this will really slow down your instance, but let it run a
>> few minutes nevertheless and pastebin the result somewhere.
>>
>> > As a rough measure - I have a syslog-ng process that has been running for
>> > less than 3 hours and right now is using 1.52GB of resident memory (shown by
>> > "top")
>>
>> I'm suspecting `format-json` to be the source of the leak, or the flow
>> control cache. These have been the leak source of our own observations many
>> times, although many patches have solved the issues.
>>
>> > (I would rather parse things in syslog-ng, but I *could* do this all using
>> > logstash/grok if this proves too much for patterndb at this load)
>>
>> if you're worried about performance, I wouldn't even consider to begin thinking about comparing
>> LS/grok to patterndb
>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng [3]
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng [4]
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq [5]
>> 
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng [3]
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng [4]
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq [5]

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng [3]
Documentation:
http://www.balabit.com/support/documentation/?product=syslog-ng [4]
FAQ: http://www.balabit.com/wiki/syslog-ng-faq [5]

-- 
Bien cordialement, Kind Regards
HIFA, Chairman

32 rue de la République 92190 MEUDON - FRANCE 
Phone: +33 1 46 31 44 25
Mobile: +33 6 11 30 36 57
email: simon.obounou at hifa.biz
 

Links:
------
[1] https://github.com/balabit/syslog-ng/pull/359
[2] https://github.com/ihrwein/syslog-ng/tree/f/fix-redis-memleak
[3] https://lists.balabit.hu/mailman/listinfo/syslog-ng
[4] http://www.balabit.com/support/documentation/?product=syslog-ng
[5] http://www.balabit.com/wiki/syslog-ng-faq
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20141224/a729b33c/attachment.htm