August 2008 - syslog-ng - lists.balabit.hu

Re: [syslog-ng] syslog-ng Digest, Vol 40, Issue 16
by vadi 27 Aug '08

27 Aug '08

Hi Joe, Thanks for your replay. I am not able to understand the regular expression statement with are there in the internet. For example: \s(\S+)\s(\S+)\[\d+\]\:\s\[ID \d+ (\S+)\.(\S+)\]\s \S+\s+\d+\s+(\d+)\:\d+\:\d+\s I want to know what exactly text of this pattern? Is there any tool will convert RegExp to text?. So that it will helps for me to understand better. Regards, Vadiraj On Mon, Aug 25, 2008 at 3:30 PM, <syslog-ng-request(a)lists.balabit.hu> wrote: > Send syslog-ng mailing list submissions to > syslog-ng(a)lists.balabit.hu > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.balabit.hu/mailman/listinfo/syslog-ng > or, via email, send a message with subject or body 'help' to > syslog-ng-request(a)lists.balabit.hu > > You can reach the person managing the list at > syslog-ng-owner(a)lists.balabit.hu > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of syslog-ng digest..." > > > Today's Topics: > > 1. Re: tool to convert regular expression to text (Fegan, Joe) > 2. Re: syslog-ng 2 nics (Balazs Scheidler) > 3. Re: [patch] Add follow_freq_ms option (Balazs Scheidler) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sun, 24 Aug 2008 23:34:59 +0000 > From: "Fegan, Joe" <Joe.Fegan(a)hp.com> > Subject: Re: [syslog-ng] tool to convert regular expression to text > To: Syslog-ng users' and developers' mailing list > <syslog-ng(a)lists.balabit.hu> > Message-ID: > < > 0E6222894DE49B40A3A9FD929C630BCB379EB5A2EF(a)GVW1121EXC.americas.hpqcorp.net > > > > Content-Type: text/plain; charset="us-ascii" > > Hi Vadiraj, > > Your question doesn't really make sense. Can you please elaborate. That > might help. > > Joe. > > ________________________________ > From: syslog-ng-bounces(a)lists.balabit.hu [mailto: > syslog-ng-bounces(a)lists.balabit.hu] On Behalf Of vadi > Sent: 22 August 2008 22:15 > To: syslog-ng(a)lists.balabit.hu > Subject: [syslog-ng] tool to convert regular expression to text > > Dear All, > > Do we have any tool to convert regular expression to text?. Please let me > know about it. > > Thanking u, > > Regards, > Vadiraj > >

2 1

Re: [syslog-ng] FreeBSD dist syslog-ng.conf-sample dumps mail.info to maillog & messages
by Len Conrad 26 Aug '08

26 Aug '08

> > > > postfix is sending only warnings to messages > >Well, this rings a bell. When you run a chrooted postfix then it >could fall back using /dev/log (or its FreeBSD equivalent) in the >chroot when syslog-ng is restarted. A commonly used workaround on >linux is to create the /var/spool/postfix/dev directory by hand >and add /var/spool/postfix/dev/log to the sources of syslog-ng. Interesting point, thanks. We aren't running chroot, but noted for the future. To the "log" directives feeding stuff into destination file "messages", I added a NOT filter for "postfix.*warning" and for "postgrey". >I don't see any problems with your config, although I did only >a quick check. fixed my error: filter f_conn_from_unk { match(": connection from unknown"); }; to filter f_conn_from_unk { match(": connect from unknown"); }; thanks, Len

1 0

FreeBSD dist syslog-ng.conf-sample dumps mail.info to maillog & messages
by Len Conrad 26 Aug '08

26 Aug '08

I can't see how this is happening. I can stop the mail.info to maillog, but I can't stop it to messages since I can't find any destination (messages) being fed by mail.info I found this and commented it out, but no change: #log { source(src); filter(f_mail); filter(f_crit); destination(messages); }; I'm looking for : log { source(src); filter(f_mail); filter(f_info); destination(messages); }; or anything similar but can't find it. It's weird that this is the vanilla syslog-ng.conf-sample with the FreeBSD 6 package. thanks, Len

2 2

Re: [syslog-ng] FreeBSD dist syslog-ng.conf-sample dumps mail.info tomaillog & messages
by infosec＠gmail.com 26 Aug '08

26 Aug '08

Len, Go ahead and send the whole file to the list, or if it's really large post it somewhere and send us the link. Is it possible that you're still running another syslog daemon and it isn't syslog-ng sending the messages there? -----Original Message----- From: Len Conrad <LConrad(a)Go2France.com> Subj: [syslog-ng] FreeBSD dist syslog-ng.conf-sample dumps mail.info tomaillog & messages Date: Mon Aug 25, 2008 9:03 pm Size: 852 bytes To: syslog-ng(a)lists.balabit.hu I can't see how this is happening. I can stop the mail.info to maillog, but I can't stop it to messages since I can't find any destination (messages) being fed by mail.info I found this and commented it out, but no change: #log { source(src); filter(f_mail); filter(f_crit); destination(messages); }; I'm looking for : log { source(src); filter(f_mail); filter(f_info); destination(messages); }; or anything similar but can't find it. It's weird that this is the vanilla syslog-ng.conf-sample with the FreeBSD 6 package. thanks, Len ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

1 0

tool to convert regular expression to text
by vadi 25 Aug '08

25 Aug '08

Dear All, Do we have any tool to convert regular expression to text?. Please let me know about it. Thanking u, Regards, Vadiraj

2 1

lost messages with follow_freq()?
by Joe Shaw 21 Aug '08

21 Aug '08

Hi, I'm trying to use the follow_freq() option to tail a growing log file, but not all of the messages are making it from the source end to the destination end. However, according to the statistics, no messages are dropped. I am using syslog-ng 2.0.9. The file on the source side looks like this: -rw-r--r-- 1 jshaw ita 251M Aug 19 15:07 dump-file and is growing rather rapidly. (It is basically being created by replaying another stored log file.) At this same point on the destination side, this is the corresponding file: -rw-r--r-- 1 root root 30M Aug 19 15:07 syslog-messages And doing a diff on those files does show large missing chunks from this file. The source's statistics say that nothing has been dropped: Aug 19 15:07:45 source-host syslog-ng[18574]: Log statistics; dropped='tcp(AF_INET(10.1.73.18:2000))=0', processed='center(queued)=303967', processed='center(received)=304017', processed='destination(d_file)=6', processed='destination(d_remote)=303961', processed='source(s_sys)=56', processed='source(s_internal)=6', processed='source(s_file)=303955' And on the destination side: Aug 19 15:08:04 dest-host syslog-ng[21023]: Log statistics; processed='center(queued)=318524', processed='center(received)=318521', processed='destination(d_file)=318521', processed='destination(d_stats)=3', processed='source(s_tcp)=318518', processed='source(s_internal)=3' The source syslog-ng.conf file looks like this: options { sync(0); time_reopen(10); log_fifo_size(1000); long_hostnames(off); use_dns(yes); dns_cache(yes); use_fqdn(no); keep_hostname(yes); use_time_recvd(no); log_msg_size(65536); stats_freq(180); }; source s_internal { internal(); }; source s_sys { file ("/proc/kmsg" log_prefix("kernel: ")); unix-stream ("/dev/log"); }; source s_file { file("/ita/dump-file" follow_freq(1) flags(no-parse)); }; # Local destination for statistics destination d_file { file("/ita/syslog-messages" perm(0644)); }; # Remote destination destination d_remote { tcp("dest-host" port(2000)); }; # Send stats locally log { source(s_internal); destination(d_file); }; # Send everything remotely log { source(s_internal); source(s_file); destination(d_remote); }; (Yes, the s_sys source is unused.) On the destination side: options { sync(0); time_reopen(10); log_fifo_size(1000); long_hostnames(off); use_dns(yes); dns_cache(yes); use_fqdn(no); keep_hostname(yes); use_time_recvd(no); log_msg_size(65536); stats_freq(180); }; # Remote source source s_tcp { tcp(port(2000) log-fetch-limit(128) max-connections(1000)); }; # syslog-ng statistics source s_internal { internal(); }; destination d_file { file("/ita/syslog-messages" perm(0644) log_fifo_size(100)); }; destination d_stats { file("/ita/syslog-stats" perm(0644)); }; # Save stats separately log { source(s_internal); destination(d_stats); }; # Take all remote data and save it locally log { source(s_tcp); source(s_internal); destination(d_file); }; Any ideas what might be going on, or how to analyze this further? Thanks, Joe

2 6

how to fetch logs from linux, windows, and devices (firewall, router)
by senthil kumar 21 Aug '08

21 Aug '08

hi pals i am new to syslog-ng. it is a very nice software. i want to used opensource edition of syslog-ng i want to log all linux, windows, sonicwall firewall, watchguard firewall, cisco router, dlink router, DSL router. is it possible to log all above things just using opensource edition of syslog-ng. and i want the detailed notes for "HOW TO INSTALL SYSLOG-NG SERVER IN LINUX" and i want configuration details for "GETTING LOGS FROM LINUX, WINDOWS AND DEVICES" pls help me. Regards N.Senthil Kumar Mobile +91 9787150175 Unlimited freedom, unlimited storage. Get it now, on http://help.yahoo.com/l/in/yahoo/mail/yahoomail/tools/tools-08.html/

1 0

destination program is killed due to broken pipe
by midnight codder 19 Aug '08

19 Aug '08

I am trying to redirect all log messages to a destination program (written in java). This did not do much so I redirected all messages to a single catch-all log file AND also to a bash shell script that runs an endless loop, reads a line and spit it out to a text file. The shell script is killed by syslog-ng after the 5th message. The catch-all file, however, keeps getting all of the messages (that's how I know who is killing who). Here is the config file for syslog-ng: options { ts_format(iso); frac_digits(3); }; template bds_template{ template("<pri=$PRI> <date=$ISODATE> <host=$HOST> <msg=$MSG>\n"); template_escape(no); }; source src{ internal(); unix_stream("/dev/log"); unix-dgram("/dev/log"); file("/proc/kmsg" log_prefix("kernel:")); udp(); tcp(); }; destination test_adapter{ program("bash /usr/local/bds/logger/syslogAdapterTest.sh" template(bds_template)); }; log { source(src); destination(test_adapter); }; destination catchAll{ file("/var/log/syslog-ng-catch-all.log", template(bds_template)); }; log { source(src); destination(catchAll); }; Here is the bash script: #!/bin/bash file="/root/syslogAdapterTest" counter=1 limit=3 while true do read line echo "$line" > $file.$counter ((counter++)) done and finally the content of the catch-all file, in blue are the log messages that the shell script still captures and spits to text files, in red are the messages are the messages the shell does not get: <pri=45> <date=2008-08-19T15:13:56.191-05:00> <host=oracle-test> <msg=syslog-ng[27971]: Server license found, starting in server mode; customer='University of Illinois', serial='49544f30-573e-11dd-9892-0007e9195011', limit='25'> <pri=46> <date=2008-08-19T15:13:56.191-05:00> <host=oracle-test> <msg=syslog-ng[27971]: Starting destination program; cmdline='bash /usr/local/bds/logger/syslogAdapterTest.sh'> <pri=45> <date=2008-08-19T15:13:56.193-05:00> <host=oracle-test> <msg=syslog-ng[27971]: syslog-ng starting up; version='2.1.11a'> <pri=46> <date=2008-08-19T15:13:56.193-05:00> <host=oracle-test> <msg=syslog-ng[27971]: Initializing destination file writer; template='/var/log/syslog-ng-catch-all.log', filename='/var/log/syslog-ng-catch-all.log'> <pri=45> <date=2008-08-19T15:13:56.193-05:00> <host=oracle-test> <msg=syslog-ng[27971]: License validity expires in one week, logging service will terminate at that point; error='License outside the validity period (option=syslog-ng, not-before=2008/07/21, not-after=2008/08/21)'> *<pri=43> <date=2008-08-19T15:13:56.198-05:00> <host=oracle-test> <msg=syslog-ng[27971]: I/O error occurred while writing; fd='11', error='Broken pipe (32)'> <pri=46> <date=2008-08-19T15:13:56.198-05:00> <host=oracle-test> <msg=syslog-ng[27971]: Sending child a TERM signal; child_pid='27972'>* Any idea?

1 0

Store and forward for ~800 hosts
by Josh Rivel 19 Aug '08

19 Aug '08

Hello. I have roughly 800 OpenSolaris hosts running syslog-ng 1.6.11, logging back to another OpenSolaris host running 2.1beta2. Most of the remote locations are over slow DSL lines, so the client has requested that we only do log collection between 04:00 and 06:00 (for example) What's the best method for this? I thought about just logging everything locally at the remote locations and rsync'ing the log files back to the syslog-ng server during those hours, but I' not sure if that two hour window will be enough to gather all the data. If the commercial version of syslog-ng supports such a configuration, then I'm sure that the client would be consider purchasing it, so if that's an option let me know. Ideally there should be a way to only send logs remotely between a certain time window from the remote machines, all other times log locally, and then dump the remaining logs during that time window. I've searched the mailing list archives and couldn't find anything applicable. Thanks in advance, Josh

1 0

Syslog-ng wont start after centos updates
by John Duraisw 19 Aug '08

19 Aug '08

Hello, I did a centos update yesterday and now my syslog-ng wont start. Please help. [root@central-logger init.d]# /etc/init.d/syslog-ng.init start Starting syslog-ng: syntax error at 189 Starting Kernel Logger: [FAILED] Thanks, -john

2 2