- syslog-ng - lists.balabit.hu

Re : Re : Re : Syslogd + Syslog-ng
by olivier "madmax"rolland 23 Jul '08

23 Jul '08

Could you send us a partial copy of your syslog-ng configuration? ----- Message d'origine ---- De : Leandro Ferreira da Silva <ferreira(a)iqm.unicamp.br> À : syslog-ng(a)lists.balabit.hu Envoyé le : Mardi, 22 Juillet 2008, 21h35mn 02s Objet : Re: [syslog-ng] Re : Re : Syslogd + Syslog-ng The messages is send to server, the problem is the server that can't take the messages. I'm using FreeBSD 7.0 release. > You can try to launch syslogd in the debug mode and look if your > messages from the client are really rend to the syslog-ng server. > What OS do you use? > ----- Message d'origine ---- > De : Leandro Ferreira da Silva <ferreira(a)iqm.unicamp.br> > À : "olivier "madmax"rolland" <madmax2010fr(a)yahoo.fr> > Envoyé le : Mardi, 22 Juillet 2008, 19h54mn 28s > Objet : Re: Re : [syslog-ng] Syslogd + Syslog-ng > > The syslogd is correctly configured with *.* @server, > I can say this because the following command is logging at server. > tcpdump -nn -i re0 host "machine" and udp port 514 > > The problem is that in some machines I can't install syslog-ng, so > these machines have to send their logs through syslogd to the server > that is using the syslog-ng. > Any help? > > > First of all I'm not sure that with *.* your syslogd is able to work, > > you might user *.debug in order to log all messages from debug to the > > maximal level of logging. Secondly if your computer or server is in a > > domain you might test the remote logging as > > *.debug @server.domain_name > > > > ----- Message d'origine ---- > > De : Leandro Ferreira da Silva <ferreira(a)iqm.unicamp.br > <mailto:ferreira@iqm.unicamp.br>> > > À : syslog-ng(a)lists.balabit.hu <mailto:syslog-ng@lists.balabit.hu> > > Envoyé le : Mardi, 22 Juillet 2008, 0h22mn 35s > > Objet : [syslog-ng] Syslogd + Syslog-ng > > > > Dear Friends. > > > > I have some problems.. =P > > I'm building a log server, I installed the syslog-ng at the server. The > > clients has the common syslogd. > > How Can I do for the syslog-ng receive the messages from syslogd? > > > > The configuration of syslogd. > > # uncomment this to enable logging to a remote loghost named loghost > > *.* @loghost > > > > Is this possible? > > I need to configure of this form, because I can't install and configure > > the syslog-ng in all my machines. > > > > Thanks for all.. > > > > Leandro Ferreira > > > > > ______________________________________________________________________________ > > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Documentation: > > http://www.balabit.com/support/documentation/?product=syslog-ng > > FAQ: http://www.campin.net/syslog-ng/faq.html > > > > > > ------------------------------------------------------------------------ > > Envoyé avec Yahoo! Mail > > > <http://us.rd.yahoo.com/mailuk/taglines/isp/control/*http://us.rd.yahoo.com/…>. > > Une boite mail plus intelligente. > > > ------------------------------------------------------------------------ > Envoyé avec Yahoo! Mail > <http://us.rd.yahoo.com/mailuk/taglines/isp/control/*http://us.rd.yahoo.com/…>. > Une boite mail plus intelligente. ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html _____________________________________________________________________________ Envoyez avec Yahoo! Mail. Une boite mail plus intelligente http://mail.yahoo.fr

3 3

Re : Re : Re : Re : Syslogd + Syslog-ng
by olivier "madmax"rolland 23 Jul '08

23 Jul '08

Is syslogd running on your central server? In this case you have to disable syslogd logging from remote (using -s option) . personally in order to recieve logs from syslogd clients i use this config: source s_sys { udp(ip(0.0.0.0) port(514)); }; filter f_sys { level(info..emerg) and not facility(mail); }; destination d_sys {file("/foo/full$DAY.log");}; log { source(s_sys); filter(f_sys) ; destination(d_sys) ;}; ----- Message d'origine ---- De : Leandro Ferreira da Silva <ferreira(a)iqm.unicamp.br> À : Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Envoyé le : Mercredi, 23 Juillet 2008, 14h14mn 29s Objet : Re: [syslog-ng] Re : Re : Re : Syslogd + Syslog-ng The configuration is the standart, I only do the follow changes. # sources source src { unix-dgram("/var/run/log"); unix-dgram("/var/run/logpriv" perm(0600)); udp(); internal(); file("/dev/klog"); }; I add this, source r_src { udp(ip("client.domain") port(514)); }; I add this, destination gafanhoto_messages { file("/var/log/gafanhoto/messages" owner("root") group("wheel") perm(0640)); }; I add this, log { source(r_src); destination(gafanhoto_messages); }; When I start the syslog-ng.. scorpion# /usr/local/etc/rc.d/syslog-ng start Starting syslog_ng. Error binding socket; addr='AF_INET(client.domain:514)', error='Can\'t assign requested address (49)' Error initializing source driver; source='r_src' Could you send us a partial copy of your syslog-ng configuration? ----- Message d'origine ---- De : Leandro Ferreira da Silva <ferreira(a)iqm.unicamp.br> À : syslog-ng(a)lists.balabit.hu Envoyé le : Mardi, 22 Juillet 2008, 21h35mn 02s Objet : Re: [syslog-ng] Re : Re : Syslogd + Syslog-ng The messages is send to server, the problem is the server that can't take the messages. I'm using FreeBSD 7.0 release. > You can try to launch syslogd in the debug mode and look if your > messages from the client are really rend to the syslog-ng server. > What OS do you use? > ----- Message d'origine ---- > De : Leandro Ferreira da Silva <ferreira(a)iqm.unicamp.br> > À : "olivier "madmax"rolland" <madmax2010fr(a)yahoo.fr> > Envoyé le : Mardi, 22 Juillet 2008, 19h54mn 28s > Objet : Re: Re : [syslog-ng] Syslogd + Syslog-ng > > The syslogd is correctly configured with *.* @server, > I can say this because the following command is logging at server. > tcpdump -nn -i re0 host "machine" and udp port 514 > > The problem is that in some machines I can't install syslog-ng, so > these machines have to send their logs through syslogd to the server > that is using the syslog-ng. > Any help? > > > First of all I'm not sure that with *.* your syslogd is able to work, > > you might user *.debug in order to log all messages from debug to the > > maximal level of logging. Secondly if your computer or server is in a > > domain you might test the remote logging as > > *.debug @server.domain_name > > > > ----- Message d'origine ---- > > De : Leandro Ferreira da Silva <ferreira(a)iqm.unicamp.br > <mailto:ferreira@iqm.unicamp.br>> > > À : syslog-ng(a)lists.balabit.hu <mailto:syslog-ng@lists.balabit.hu> > > Envoyé le : Mardi, 22 Juillet 2008, 0h22mn 35s > > Objet : [syslog-ng] Syslogd + Syslog-ng > > > > Dear Friends. > > > > I have some problems.. =P > > I'm building a log server, I installed the syslog-ng at the server. The > > clients has the common syslogd. > > How Can I do for the syslog-ng receive the messages from syslogd? > > > > The configuration of syslogd. > > # uncomment this to enable logging to a remote loghost named loghost > > *.* @loghost > > > > Is this possible? > > I need to configure of this form, because I can't install and configure > > the syslog-ng in all my machines. > > > > Thanks for all.. > > > > Leandro Ferreira > > > > > ______________________________________________________________________________ > > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Documentation: > > http://www.balabit.com/support/documentation/?product=syslog-ng > > FAQ: http://www.campin.net/syslog-ng/faq.html > > > > > > ------------------------------------------------------------------------ > > Envoyé avec Yahoo! Mail > > > <http://us.rd.yahoo.com/mailuk/taglines/isp/control/*http://us.rd.yahoo.com/…>. > > Une boite mail plus intelligente. > > > ------------------------------------------------------------------------ > Envoyé avec Yahoo! Mail > <http://us.rd.yahoo.com/mailuk/taglines/isp/control/*http://us.rd.yahoo.com/…>. > Une boite mail plus intelligente. ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html ________________________________ Envoyé avec Yahoo! Mail. Une boite mail plus intelligente. ________________________________ ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html _____________________________________________________________________________ Envoyez avec Yahoo! Mail. Une boite mail plus intelligente http://mail.yahoo.fr

1 0

Re: [syslog-ng] Re : Re : Syslogd + Syslog-ng
by Leandro Ferreira da Silva 22 Jul '08

22 Jul '08

The messages is send to server, the problem is the server that can't take the messages. I'm using FreeBSD 7.0 release. > You can try to launch syslogd in the debug mode and look if your > messages from the client are really rend to the syslog-ng server. > What OS do you use? > ----- Message d'origine ---- > De : Leandro Ferreira da Silva <ferreira(a)iqm.unicamp.br> > À : "olivier "madmax"rolland" <madmax2010fr(a)yahoo.fr> > Envoyé le : Mardi, 22 Juillet 2008, 19h54mn 28s > Objet : Re: Re : [syslog-ng] Syslogd + Syslog-ng > > The syslogd is correctly configured with *.* @server, > I can say this because the following command is logging at server. > tcpdump -nn -i re0 host "machine" and udp port 514 > > The problem is that in some machines I can't install syslog-ng, so > these machines have to send their logs through syslogd to the server > that is using the syslog-ng. > Any help? > > > First of all I'm not sure that with *.* your syslogd is able to work, > > you might user *.debug in order to log all messages from debug to the > > maximal level of logging. Secondly if your computer or server is in a > > domain you might test the remote logging as > > *.debug @server.domain_name > > > > ----- Message d'origine ---- > > De : Leandro Ferreira da Silva <ferreira(a)iqm.unicamp.br > <mailto:ferreira@iqm.unicamp.br>> > > À : syslog-ng(a)lists.balabit.hu <mailto:syslog-ng@lists.balabit.hu> > > Envoyé le : Mardi, 22 Juillet 2008, 0h22mn 35s > > Objet : [syslog-ng] Syslogd + Syslog-ng > > > > Dear Friends. > > > > I have some problems.. =P > > I'm building a log server, I installed the syslog-ng at the server. The > > clients has the common syslogd. > > How Can I do for the syslog-ng receive the messages from syslogd? > > > > The configuration of syslogd. > > # uncomment this to enable logging to a remote loghost named loghost > > *.* @loghost > > > > Is this possible? > > I need to configure of this form, because I can't install and configure > > the syslog-ng in all my machines. > > > > Thanks for all.. > > > > Leandro Ferreira > > > > > ______________________________________________________________________________ > > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Documentation: > > http://www.balabit.com/support/documentation/?product=syslog-ng > > FAQ: http://www.campin.net/syslog-ng/faq.html > > > > > > ------------------------------------------------------------------------ > > Envoyé avec Yahoo! Mail > > > <http://us.rd.yahoo.com/mailuk/taglines/isp/control/*http://us.rd.yahoo.com/…>. > > Une boite mail plus intelligente. > > > ------------------------------------------------------------------------ > Envoyé avec Yahoo! Mail > <http://us.rd.yahoo.com/mailuk/taglines/isp/control/*http://us.rd.yahoo.com/…>. > Une boite mail plus intelligente.

1 0

[Fwd: Re: Re : Syslogd + Syslog-ng]
by Leandro Ferreira da Silva 22 Jul '08

22 Jul '08

1 0

Syslogd + Syslog-ng
by Leandro Ferreira da Silva 22 Jul '08

22 Jul '08

Dear Friends. I have some problems.. =P I'm building a log server, I installed the syslog-ng at the server. The clients has the common syslogd. How Can I do for the syslog-ng receive the messages from syslogd? The configuration of syslogd. # uncomment this to enable logging to a remote loghost named loghost *.* @loghost Is this possible? I need to configure of this form, because I can't install and configure the syslog-ng in all my machines. Thanks for all.. Leandro Ferreira

1 0

syslog-ng-2.1beta1 on NetBSD 3.0, duplicate suppression docs?
by Ed Ravin 20 Jul '08

20 Jul '08

I encountered two problems building syslog-ng-2.1beta1 on NetBSD 3.0: 1) I needed to patch loggen.c (stole this patch from a FreeBSD site, apparently it happens on that OS too) --- tests/loggen/loggen.c.orig 2008-01-13 00:04:22.000000000 -0800 +++ tests/loggen/loggen.c 2008-03-26 08:27:43.000000000 -0700 @@ -226,7 +226,9 @@ memset(&hints, 0, sizeof(hints)); hints.ai_family = AF_UNSPEC; hints.ai_socktype = sock_type; +#ifdef AI_ADDRCONFIG hints.ai_flags = AI_ADDRCONFIG; +#endif hints.ai_protocol = 0; if (getaddrinfo(argv[optind], argv[optind + 1], &hints, &res) != 0) { ------------------------------------- 2) I built syslog-ng with --enable-tcp-wrappers, which needlessly put "-lwrap" on the LIBS line in the Makefiles for programs that did not actually use TCP wrappers. This causes link-time errors, since libwrap requires you to declare two symbols in your program. The fix was to run this in the source directory: find tests/ -name Makefile -exec perl -p -i -e 's/-lwrap//' {} ";" ------------------------------------- I also have a couple of observations/questions: 3) It turns out I really wanted syslog-ng2.1beta2, but although that was announced as released, only the beta1 tarball is actually on the source download site. I had to go to the snapshots. The above fixes were also needed for the beta2 sources. ------------------------------------- 4) I'm looking for info on how to use the duplicate suppression option - it doesn't seem to be in the docs included in the source. Can someone give me a quick summary? This was (IMHO) the only extant flaw in syslog-ng and I'm looking forward to trying it out! Thanks, -- Ed

3 5

Odd behavior with very small log_fifo_size()
by Joe Shaw 20 Jul '08

20 Jul '08

Hi, I was doing a little bit of testing with syslog-ng today, and I think that with a very small log_fifo_size() it was dropping messages but not telling me. If I set my log_fifo_size to 1, here are the results: [jshaw@ars1dev7 ~/syslog-ng-2.0.9/tests/loggen]$ ./loggen -r 1000000 -s 1024000 -I 60 localhost 2000 average rate = 96.48 msg/sec, count=5789 [jshaw@ars1dev7 ~/syslog-ng-2.0.9/tests/loggen]$ ls -lh /ita/jshaw/messages -rw------- 1 jshaw ita 101M Jul 1 13:30 /ita/jshaw/messages However, 1024000 * 5789 = 5.9 GB Grepping for drop messages in the output file: [jshaw@ars1dev7 ~/syslog-ng-2.0.9/tests/loggen]$ grep -i drop /ita/jshaw/messages returns nothing. I have stats_freq(5) in my config file, and that messages file is the only output. I do have other statistical lines, which imply to me that no messages are dropped: Jul 1 13:49:52 s_local@ars1dev7 syslog-ng[6344]: Log statistics; processed='center(queued)=6045', processed='center(received)=6045', processed='destination(d_file)=6045', processed='source(s_local)=6045' But it seems clear to me that they are -- or loggen is lying. With more normal log_fifo_sizes (100, 2000) the throughput rates from loggen are lower and the file sizes match. Any ideas? Thanks, Joe

3 5

Re: [syslog-ng] Not able to match on a field in syslog messages from a DataPower Appliance
by Salowitz, Adam (AS.) 17 Jul '08

17 Jul '08

>USER.DEBUG: Jul 15 18:23:44 DPRASSyslogAudit3 [system][debug] >trans(383): cpu usage: 57%(10 sec) 57%(1 min) 57%(10 >min) 56%(1 hour) 58%(1 day) > >In syslog wire protocol the field immediately after the >timestamp is the hostname, so this message would be >interpreted as being from a host called DPRASSyslogAudit3. >Your filters are not matching these messages because match() >only matches elements of the message body and the text you're >looking for does not appear in the message body. > >Joe. That's actually what thought (but forgot to mention in my earlier email), but when I try to match on host it doesn't catch anything either. Neither of these catch anything: filter group_syncDPextest { host("DP[:digit:]+Syslog") or host("DPRASSyslogAudit[:digit:]+") ;}; filter group_syncDPextest { host("DP") ;}; Is this because the host filter actually comes from the source host of the packet and not the message itself? Are DataPower's syslog messages just broken or non-rfc complient? Should I just give up and try to match on other strings in the message? Thanks again for all your help on this weird problem. Adam

2 2

Not able to match on a field in syslog messages from a DataPower Appliance
by Salowitz, Adam (AS.) 17 Jul '08

17 Jul '08

I am using syslog-ng 1.6.12 to collect logs from an IBM DataPower appliance. The DataPower appliance allows a certain custom string to be added to the outgoing syslogs called a Local Identifier. I am trying to use a regex to match on this Local Identifier. I have used tcpdump to capture actual packets and can see that the Local Identifier is in the Application layer Syslog message field. My question is why is match not working when I try to use a filter to grab just a particular Local ID? No logs are caught by my filter. I have tried: match ("DP\d+Syslog") match ("DPRASSyslogAudit3") match ("DPRASSyslogAudit") match ("DP") I have been able to catch other strings in the messages, such as system, debug, dpHandler, etc..." These example packets below are copy and pasted from wireshark by clicking on the Syslog message field in the packet details from and right clicking -> Copy -> Description. * Three Syslog messages from the DataPower appliance that show the Local identifier that I want to match against in the fourth field (DPRASSyslogAudit3, DP472Syslog). Syslog message: USER.DEBUG: Jul 15 18:23:44 DPRASSyslogAudit3 [system][debug] trans(383): cpu usage: 57%(10 sec) 57%(1 min) 57%(10 min) 56%(1 hour) 58%(1 day)\n Syslog message: USER.ERR: Jul 15 18:23:43 DP472Syslog [mpgw][error] trans(8152497)[19.x.x.x]: Request processing failed: Connection terminated before request headers read\n Syslog message: USER.ERR: Jul 15 18:25:11 DP474Syslog [dpHandler_prod_host474][ssl][error] valcred(pubcert): trans(8043025)[19.x.x.x]: SSL Proxy Profile 'xgtws': connection error: peer did not send a certificate\n * One syslog messsage from a Cisco device which shows the hostname in the fourth field Syslog message: LOCAL7.NOTICE: Jul 15 22:23:48 hostname 2008 Jul 15 22:23:46 %SNMP-5-SNMPAUTHFAIL:Authentication failed for message from 19.x.x.x\n Does anyone have any idea why this field might be getting missed or can anyone give me some guidance as to where in the source the incoming messages are parsed? I have tried all the macros listed in the code and never get the local ID. #DataPower Appliances Extreme Test destination syncDPextest { file("/logs/syncDPextest/$YEAR$MONTH$DAY.txt" template("$FACILITY $FACILITY_NUM $PRIORITY $LEVEL $LEVEL_NUM $TAG $PRI $HOST $DATE $FULLHOST_FROM $MESSAGE\n") ); }; user 1 err err 3 0b 11 hostname Jul 16 16:10:59 hostname [dpHandler_prod_host472][ssl][error] valcred(pubcert): trans(8320177)[19.x.x.x]: SSL Proxy Profile \'xgtws\': connection error: peer did not send a certificate Any help or suggestions whould be appreciated. Thanks, Adam

3 2

Using Regex information for destination
by James Whitt 11 Jul '08

11 Jul '08

I had asked this question about a year ago, but was told to wait for the new 2.0 version. I'm currently running the latest syslog-ng 2.0.7. What I would like to do is put the line in a specific file based on part of the hostname. For example: Hostname: subdomain.domain.tld destination customer_firewall_logs { file("/logs/$TLD/$DOMAIN/$SUBDOMAIN/$R_YEAR-$R_MONTH-$R_DAY.log"); }; WHERE $TLD is the tld of the hostname, $DOMAIN is the domain from the hostname, and $SUBDOMAIN is teh subdomain from the hostname. I would like to be able to regex this information out of the hostname to use in the destination. Is this possible and any tips on getting it to work correctly would be greatly appreciated.

3 6