Syslog-ng3 : syslog-ng Digest, Vol 53, Issue 25
Hi, The syslog-ng conf file is @version:3.0 options { use_dns(persist_only); dns_cache_hosts(/etc/hosts); #chain_hostnames(yes); keep_hostname(yes); keep_timestamp(yes); }; source all_sources { # Solaris used streams to send msgs to Syslogd process, replicate for syslog-ng sun-streams("/dev/log" door("/etc/.syslog_door")); # sun-streams( door("/etc/.syslog_door")); # Messages generated internally by Syslog-ng internal(); # Listens on the specified udp port for incoming messages udp(ip(0.0.0.0) port(514)); }; destination local7apm { file("/var/log/syslog" template("$DATE $HOST $MSGHDR $MSGONLY\n")); }; #destination local7apm { udp("smarts5" port(514) spoof-source(yes)); }; #destination d_tcp { tcp("10.1.2.3" port(1999); localport(999)); }; #destination messages { file("/var/adm/messages" template("$DATE $HOST $MSG\n")); }; # Below 5 lines taken from existing configuration #destination sysmsg { file("/dev/sysmsg" template("$DATE $HOST $MSG\n")); }; #destination operator { usertty("operator" template("$DATE $HOST $MSG\n")); }; destination operator { usertty("operator"); }; #destination all_alerts { usertty("operator" template("$DATE $HOST $MSG\n")); }; destination all_alerts { usertty("operator"); }; #destination all_alerts1 { usertty("root" template("$DATE $HOST $MSG\n")); }; destination all_alerts1 { usertty("root"); }; #destination emergency { usertty("*" template("$DATE $HOST $MSG\n")); }; destination emergency { usertty("*"); }; # filter local7msgs { facility(local7) and level(info..emerg); }; filter messages { facility(kern, daemon, mail) and level(debug, info, notice, warning, err, crit alert, emerg); }; filter f_sysmsg { facility(kern, auth) and level(notice); }; filter f_all_errors { level(err); }; filter f_operator { facility(kern, daemon) and level(err); }; filter f_all_alerts { level(alert); }; filter f_all_alerts1 { level(alert); }; filter f_emergency { level(emerg); }; Thanks & Regards, Ashish Shrimali -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of syslog-ng-request@lists.balabit.hu Sent: Wednesday, September 30, 2009 9:52 AM To: syslog-ng@lists.balabit.hu Subject: syslog-ng Digest, Vol 53, Issue 25 Send syslog-ng mailing list submissions to syslog-ng@lists.balabit.hu To subscribe or unsubscribe via the World Wide Web, visit https://lists.balabit.hu/mailman/listinfo/syslog-ng or, via email, send a message with subject or body 'help' to syslog-ng-request@lists.balabit.hu You can reach the person managing the list at syslog-ng-owner@lists.balabit.hu When replying, please edit your Subject line so it is more specific than "Re: Contents of syslog-ng digest..." Today's Topics: 1. Re: Colon embedded in messages (:) (chris packham) 2. Re: Colon embedded in messages (:) (Evan Rempel) 3. Solaris Syslog-ng install lib error : syslog-ng Digest, Vol 53, Issue 22 (Ashish Shrimali) ---------------------------------------------------------------------- Message: 1 Date: Wed, 30 Sep 2009 09:49:59 +1300 From: "chris packham" <chris.packham@alliedtelesis.co.nz> Subject: Re: [syslog-ng] Colon embedded in messages (:) To: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu> Message-ID: <4AC329C7020000680001703F@gwia.alliedtelesyn.co.nz> Content-Type: text/plain; charset=US-ASCII
On 9/30/2009 at 9:26 AM, James Kelly <james.kelly@hmsinc.com> wrote: Thanks a lot. That seems to have worked a lot better but has introduced another problem. The messages are split up into too many lines and, when there is a match, the first of N lines are removed, still creating unwanted messages.
For example, a pretty small insert spans 3 lines... There was a match on the 2nd line, so it is not printed. However, the 1st and 3rd were.
2009-09-29T16:21:16-04:00 blahh-rep2 postgres [10440-1] postgres-blahh-10.100.10.34(15779)-4690-2009-09-29 16:21:16 EDT-LOG: statement: insert into 2009-09-29T16:21:16-04:00 blahh-rep2 postgres [10440-3] '1000000000000000');
I think you'll find is a postgres thing. syslog-ng does not split messages up based on size (at least as far as I've seen). I think there is some behaviour to do with carriage returns "\n" embedded in your log data but there are some config options to help with that, search the mailing list/admin guide for that if it turns out to be the case. ------------------------------ Message: 2 Date: Tue, 29 Sep 2009 14:19:46 -0700 From: Evan Rempel <erempel@uvic.ca> Subject: Re: [syslog-ng] Colon embedded in messages (:) To: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu> Message-ID: <4AC279F2.50205@uvic.ca> Content-Type: text/plain; charset=windows-1252; format=flowed As far as I know, the messages will always be split on multiple lines by postgresql, and you will need to match on the process number. In your example, the 10440 is the process number, and the -1, -2, -3 are the line numers of the messages that the single process creates. A single connection that runs multiple queries can create many messages, with ever increasing line number counts. Evan. James Kelly wrote:
Thanks a lot. That seems to have worked a lot better but has introduced another problem. The messages are split up into too many lines and, when there is a match, the first of N lines are removed, still creating unwanted messages.
For example, a pretty small insert spans 3 lines... There was a match on the 2nd line, so it is not printed. However, the 1st and 3rd were.
2009-09-29T16:21:16-04:00 blahh-rep2 postgres [10440-1] postgres-blahh-10.100.10.34(15779)-4690-2009-09-29 16:21:16 EDT-LOG: statement: insert into 2009-09-29T16:21:16-04:00 blahh-rep2 postgres [10440-3] '1000000000000000');
I played around with log_msg_size thinking I could control how much is on each line, but it didn't help. Am I missing something obvious? If a "fragment" of the message is matched, I would want the entire message to not be logged.
Honestly, I have not spent nearly as much time on this most recent problem, so if it is an RTFM situation, just let me know. I did do some reading and searching on this though and came up empty handed.
Thanks again for this great help, James
On Tue, Sep 29, 2009 at 3:43 PM, Evan Rempel <erempel@uvic.ca <mailto:erempel@uvic.ca>> wrote:
PostgeSQL can log to syslog using appropriately formated messages. Configure your postgresql.conf file to include the line
log_destination = 'syslog'
and then you will get lines in syslog like;
2009-09-29T12:42:10-07:00 hostname facility.level postgres[20288]: [32-1] postgresql statement
then all of the syslog-ng parsing, macros and templates will work correctly.
Evan.
James Kelly wrote: > Thanks. This is exactly what is happening. If, using a template, I > just use $PROGRAM, I get a bunch of lines with just "insert". > > If I use something like the following: > > template t_postgres_msgs { > template("$ISODATE $HOST $PROGRAM $MSG \n"); }; > > I no longer see the colons. However, if something is matched by the > filter, it still prints a blank message (hence, wasting a ton of space > and clogging up the logs). For example, this is what I get instead of > the matched sql statement: > > 2009-09-29T10:30:30-04:00 hcdb1-rep2 > > I thought the filter would just "trash" or not record the message at > all, not print the host and timestamp for each filtered message. > > Thanks a lot... I appreciate this help a lot. > James > > On Tue, Sep 29, 2009 at 10:05 AM, Srinivasan Sreenivasan > <srinivasan.srinivasan@sabre.com <mailto:srinivasan.srinivasan@sabre.com> > <mailto:srinivasan.srinivasan@sabre.com <mailto:srinivasan.srinivasan@sabre.com>>> wrote: > > I think Syslog-ng thinks insert is a program name. Use a template > with $PROGRAM in it to see if it prints ?insert? to confirm this. > > Solution: > Send a program name before your sql statement using a template. > > > > On 9/29/09 8:57 AM, "James Kelly" <james.kelly@hmsinc.com <mailto:james.kelly@hmsinc.com> > <mailto:james.kelly@hmsinc.com <mailto:james.kelly@hmsinc.com>>> wrote: > > Hello, > > I am attempting to implement syslog-ng on our servers hosting > postgresql databases. The general idea is to log "too much" at > the database level and then, using syslog filters, reduce it to > the information we want to actually hold in the syslog and send > to the log server. So far I am extremely pleased with how easy > it is to implement and well documented. However, I do have one > problem that is making it difficult to use. > > The problem is for each message that the filter matches, it does > not completely drop the message.. rather, it logs the date / > server / and a colon: > > For example, this is what I get for a message that is matched by > a filter: > > "Sep 29 09:43:29 hcdb1-rep2 :" > > I notice that even with the unmatched statements, there is a > colon. For example, in the postgresql log, I see: > > insert into "public"."table"(blah,blah,blah) values > (blah,blah,blah); > > but the same message once captured from syslog shows the > following in the log and logserver: > > Sep 29 09:43:29 hcdb1-rep2 insert: into > "public"."table"(blah,blah,blah) values (blah,blah,blah); > > *Note the colon after "insert".* I can't seem to figure out > where this is coming from or how to avoid it. It also causes > some filtering problems that I won't go into here so as to not > confuse the issues, but safe to say it is also related to the colon. > > I have spent a lot of time trying to figure this out and am at a > dead-end. Due to the amount of messages I need to filter out > (below is just one of the many filters I need to put in and have > tested with the same result), not being able to completely > filter these out is a killer. > > I am using the 3.0.4 open-source edition on Ubuntu 8. Here is > my config: > > ****************************************************************************************** > @version: 3.0 > > options { > }; > > ###### > # sources > source s_local { > # message generated by Syslog-NG > internal(); > # standard Linux log source (this is the default place for the > syslog() > # function to send logs to) > unix-stream("/dev/log"); > # messages from the kernel > file("/proc/kmsg" program_override("kernel: ")); > file("/var/log/postgresql/postgresql-8.3-main.log"); > }; > > > ###### > # destinations > destination d_messages { file("/var/log/messages"); }; > > ##### > # filters > > filter f_inserts2 { > not match("_health_central" value("MESSAGE")); > }; > > > destination d_logserver { tcp("internal.host.com <http://internal.host.com> > <http://internal.host.com> <http://internal.host.com> "); }; > > > > log { > source(s_local); > filter(f_inserts2); > destination(d_messages); > #destination(d_logserver); > }; > ****************************************************************************************** > > Thanks! > James Kelly > > ------------------------------------------------------------------------ > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > > >
-- Evan Rempel erempel@uvic.ca <mailto:erempel@uvic.ca> Senior Programmer Analyst 250.721.7691 Unix Services, University Systems, University of Victoria ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
-- Evan Rempel erempel@uvic.ca Senior Programmer Analyst 250.721.7691 Unix Services, University Systems, University of Victoria ------------------------------ Message: 3 Date: Wed, 30 Sep 2009 09:51:37 +0530 From: Ashish Shrimali <Ashish_Shrimali@infosys.com> Subject: [syslog-ng] Solaris Syslog-ng install lib error : syslog-ng Digest, Vol 53, Issue 22 To: "syslog-ng@lists.balabit.hu" <syslog-ng@lists.balabit.hu> Message-ID: <AE1A7E8803F348438B970CCDCA075C77115F91731A@BLRKECMBX07.ad.infosys.com> Content-Type: text/plain; charset="us-ascii" Hi, You need to install libnet. Steps are as follows: 1. gunzip and untar the libnet package. 2. cd <libnetdir> 3. ./configure 4. make 5. make install 6. While installing syslog-ng use this command.(with libnet) ./configure -enable-spoof-source -with-libnet=<libnet dir path>/libnet -enable-dynamic-linking It worked successfully for me. Thanks & Regards, Ashish Shrimali -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of syslog-ng-request@lists.balabit.hu Sent: Tuesday, September 29, 2009 7:36 PM To: syslog-ng@lists.balabit.hu Subject: syslog-ng Digest, Vol 53, Issue 22 Send syslog-ng mailing list submissions to syslog-ng@lists.balabit.hu To subscribe or unsubscribe via the World Wide Web, visit https://lists.balabit.hu/mailman/listinfo/syslog-ng or, via email, send a message with subject or body 'help' to syslog-ng-request@lists.balabit.hu You can reach the person managing the list at syslog-ng-owner@lists.balabit.hu When replying, please edit your Subject line so it is more specific than "Re: Contents of syslog-ng digest..." Today's Topics: 1. Syslog-ng 3 (Ashish Shrimali) 2. Re: Syslog-ng 3 (Fegan, Joe) 3. Re: sun solaris syslogng error (Balazs Scheidler) 4. Re: syslog-ng client machine IP address in message header (Balazs Scheidler) 5. Colon embedded in messages (:) (James Kelly) 6. Re: Colon embedded in messages (:) (Srinivasan Sreenivasan) ---------------------------------------------------------------------- Message: 1 Date: Tue, 29 Sep 2009 16:41:50 +0530 From: Ashish Shrimali <Ashish_Shrimali@infosys.com> Subject: [syslog-ng] Syslog-ng 3 To: "syslog-ng@lists.balabit.hu" <syslog-ng@lists.balabit.hu> Message-ID: <AE1A7E8803F348438B970CCDCA075C77115F917101@BLRKECMBX07.ad.infosys.com> Content-Type: text/plain; charset="us-ascii" Hi All, My Syslog-ng is running , which I can see by ps -ef command, but still it is not writing to the log file. I have specified the log file in the conf dir. Any inputs would be highly appreciated. Thanks & Regards, Ashish Shrimali -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of syslog-ng-request@lists.balabit.hu Sent: Tuesday, September 29, 2009 3:11 PM To: syslog-ng@lists.balabit.hu Subject: syslog-ng Digest, Vol 53, Issue 21 Send syslog-ng mailing list submissions to syslog-ng@lists.balabit.hu To subscribe or unsubscribe via the World Wide Web, visit https://lists.balabit.hu/mailman/listinfo/syslog-ng or, via email, send a message with subject or body 'help' to syslog-ng-request@lists.balabit.hu You can reach the person managing the list at syslog-ng-owner@lists.balabit.hu When replying, please edit your Subject line so it is more specific than "Re: Contents of syslog-ng digest..." Today's Topics: 1. Solaris Syslog-ng install lib error (list@net-secured.com) 2. Re: Solaris Syslog-ng install lib error (Srinivasan Sreenivasan) 3. sun solaris syslogng error (list@net-secured.com) 4. syslog-ng client machine IP address in message header (Jain, Vaibhav (GE Healthcare)) ---------------------------------------------------------------------- Message: 1 Date: Mon, 28 Sep 2009 03:52:33 -0700 From: list@net-secured.com Subject: [syslog-ng] Solaris Syslog-ng install lib error To: "syslogng " <syslog-ng@lists.balabit.hu> Message-ID: <20090928035233.9419518bf8837c8c492fb3b5d9b71733.3b4f820868.wbe@email03.secureserver.net> Content-Type: text/plain; charset="us-ascii" An HTML attachment was scrubbed... URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20090928/9ae9e2ac/at... ------------------------------ Message: 2 Date: Mon, 28 Sep 2009 08:00:09 -0500 From: Srinivasan Sreenivasan <srinivasan.srinivasan@sabre.com> Subject: Re: [syslog-ng] Solaris Syslog-ng install lib error To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Message-ID: <C6E61D89.5EF6%srinivasan.srinivasan@sabre.com> Content-Type: text/plain; charset="us-ascii" The PKG_CONFIG_PATH should include directories that have .pc files in them. On 9/28/09 5:52 AM, "list@net-secured.com" <list@net-secured.com> wrote:
Hi I tryed to install on Sparc v. 10 syslog-ng v3.0.4, runnning: ./configure --enable-sun-streams, --enable-sun-door gives this warning: configure: WARNING: unrecognized options: --enable-sun-streams, --enable-sun-door
any ideas?
furthermore, I get this error: checking for GLIB - version >= 2.10.1... no *** Could not run GLIB test program, checking why... *** The test program failed to compile or link. See the file config.log for the *** exact error that occured. This usually means GLIB is incorrectly installed. configure: error: Cannot find GLIB version >= 2.10.1: is pkg-config in path?
=== Adding the path below : produces another error
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/:/usr/lib/pkgconfig/
checking for LIBDBI... Package dbi was not found in the pkg-config search path. Perhaps you should add the directory containing `dbi.pc' to the PKG_CONFIG_PATH environment variable No package 'dbi' found no checking for dbi_initialize in -ldbi... no checking pthread.h usability... yes checking pthread.h presence... yes checking for pthread.h... yes checking for pthread_create in -lpthread... yes checking for LIBNET... no checking whether to enable Sun STREAMS support... yes checking whether to enable Sun door support... yes checking whether to enable IPv6 support... yes checking whether to enable SQL support... no checking whether to enable Linux capability support... no checking whether to enable PCRE support... yes checking for pkg-config... (cached) /usr/bin/pkg-config checking pkg-config is at least version 0.14... yes checking for GLIB - version >= 2.10.1... yes (version 2.14.1) checking for EVTLOG... yes checking for LIBNET... configure: error: libnet-config not found
thanks
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
Hi, I am using syslog-ng open source (3.0.3-1 rhel5 amd64). Syslog-ng install files in /opt/syslog-ng directory but in the uninstalling syslog-ng rpm does not remove this directory. Let me know how to solve this problem? Is there any release with this fix? I don't want to remove /opt/syslog-ng directory manually. Please help! Regards, Vaibhav
On Wed, 2009-09-30 at 11:01 +0530, Jain, Vaibhav (GE Healthcare) wrote:
Hi,
I am using syslog-ng open source (3.0.3-1 rhel5 amd64). Syslog-ng install files in /opt/syslog-ng directory but in the uninstalling syslog-ng rpm does not remove this directory. Let me know how to solve this problem? Is there any release with this fix? I don't want to remove /opt/syslog-ng directory manually.
Please help!
if that directory is not removed after removing the RPM, it probably indicates that there are some leftover files in the hierarchy, e.g. files that were not installed by the RPM package. After checking that you indeed don't need those files, you can remove the directory safely. -- Bazsi
Hi, when the specfile contains a separate entry for this directory in the %files section, it will be removed upon removal of the rpm (when the directory is empty). When the specfile does not contain this separate entry, the directory will be created implicitly. In that case rpm does not "know" this directory and will never remove it. regards, Siem Korteweg -----Oorspronkelijk bericht----- Van: syslog-ng-bounces@lists.balabit.hu namens Balazs Scheidler Verzonden: wo 30-9-2009 10:40 Aan: Syslog-ng users' and developers' mailing list Onderwerp: Re: [syslog-ng] Syslog-ng uninstallation issue On Wed, 2009-09-30 at 11:01 +0530, Jain, Vaibhav (GE Healthcare) wrote:
Hi,
I am using syslog-ng open source (3.0.3-1 rhel5 amd64). Syslog-ng install files in /opt/syslog-ng directory but in the uninstalling syslog-ng rpm does not remove this directory. Let me know how to solve this problem? Is there any release with this fix? I don't want to remove /opt/syslog-ng directory manually.
Please help!
if that directory is not removed after removing the RPM, it probably indicates that there are some leftover files in the hierarchy, e.g. files that were not installed by the RPM package. After checking that you indeed don't need those files, you can remove the directory safely. -- Bazsi _____________________________________________________________________________ _ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
Hi, Thanks for your quick response. I am using syslog-ng open source RPM (3.0.3-1 rhel5 amd64) for installation. http://www.balabit.com/downloads/files/syslog-ng/sources/3.0.3/setups/rh el-5-amd64/ I am not using SPEC file or source code for installation. Regards, V -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Siem Korteweg Sent: Wednesday, September 30, 2009 2:15 PM To: Syslog-ng users' and developers' mailing list Subject: RE: [syslog-ng] Syslog-ng uninstallation issue Hi, when the specfile contains a separate entry for this directory in the %files section, it will be removed upon removal of the rpm (when the directory is empty). When the specfile does not contain this separate entry, the directory will be created implicitly. In that case rpm does not "know" this directory and will never remove it. regards, Siem Korteweg -----Oorspronkelijk bericht----- Van: syslog-ng-bounces@lists.balabit.hu namens Balazs Scheidler Verzonden: wo 30-9-2009 10:40 Aan: Syslog-ng users' and developers' mailing list Onderwerp: Re: [syslog-ng] Syslog-ng uninstallation issue On Wed, 2009-09-30 at 11:01 +0530, Jain, Vaibhav (GE Healthcare) wrote:
Hi,
I am using syslog-ng open source (3.0.3-1 rhel5 amd64). Syslog-ng install files in /opt/syslog-ng directory but in the uninstalling syslog-ng rpm does not remove this directory. Let me know how to solve
this problem? Is there any release with this fix? I don't want to remove /opt/syslog-ng directory manually.
Please help!
if that directory is not removed after removing the RPM, it probably indicates that there are some leftover files in the hierarchy, e.g. files that were not installed by the RPM package. After checking that you indeed don't need those files, you can remove the directory safely. -- Bazsi ________________________________________________________________________ ______ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
Hi, a spec-file is used by rpmbuild to produce the rpm. When you are not happy with the way this rpm behaves, dive into RPM and improve the specfile. regards, Siem Korteweg -----Oorspronkelijk bericht----- Van: syslog-ng-bounces@lists.balabit.hu namens Jain, Vaibhav (GE Healthcare) Verzonden: wo 30-9-2009 10:56 Aan: Syslog-ng users' and developers' mailing list Onderwerp: Re: [syslog-ng] Syslog-ng uninstallation issue Hi, Thanks for your quick response. I am using syslog-ng open source RPM (3.0.3-1 rhel5 amd64) for installation. http://www.balabit.com/downloads/files/syslog-ng/sources/3.0.3/setups/rh el-5-amd64/ I am not using SPEC file or source code for installation. Regards, V -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Siem Korteweg Sent: Wednesday, September 30, 2009 2:15 PM To: Syslog-ng users' and developers' mailing list Subject: RE: [syslog-ng] Syslog-ng uninstallation issue Hi, when the specfile contains a separate entry for this directory in the %files section, it will be removed upon removal of the rpm (when the directory is empty). When the specfile does not contain this separate entry, the directory will be created implicitly. In that case rpm does not "know" this directory and will never remove it. regards, Siem Korteweg -----Oorspronkelijk bericht----- Van: syslog-ng-bounces@lists.balabit.hu namens Balazs Scheidler Verzonden: wo 30-9-2009 10:40 Aan: Syslog-ng users' and developers' mailing list Onderwerp: Re: [syslog-ng] Syslog-ng uninstallation issue On Wed, 2009-09-30 at 11:01 +0530, Jain, Vaibhav (GE Healthcare) wrote:
Hi,
I am using syslog-ng open source (3.0.3-1 rhel5 amd64). Syslog-ng install files in /opt/syslog-ng directory but in the uninstalling syslog-ng rpm does not remove this directory. Let me know how to solve
this problem? Is there any release with this fix? I don't want to remove /opt/syslog-ng directory manually.
Please help!
if that directory is not removed after removing the RPM, it probably indicates that there are some leftover files in the hierarchy, e.g. files that were not installed by the RPM package. After checking that you indeed don't need those files, you can remove the directory safely. -- Bazsi ________________________________________________________________________ ______ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html _____________________________________________________________________________ _ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
Hi, Syslog-ng client rpm uninstallation even not deleting the syslog-ng.conf file. All the files and directories are there. Its means syslog-ng is not removing any file and directory from /opt/syslog-ng Regards, V -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Balazs Scheidler Sent: Wednesday, September 30, 2009 2:10 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Syslog-ng uninstallation issue On Wed, 2009-09-30 at 11:01 +0530, Jain, Vaibhav (GE Healthcare) wrote:
Hi,
I am using syslog-ng open source (3.0.3-1 rhel5 amd64). Syslog-ng install files in /opt/syslog-ng directory but in the uninstalling syslog-ng rpm does not remove this directory. Let me know how to solve
this problem? Is there any release with this fix? I don't want to remove /opt/syslog-ng directory manually.
Please help!
if that directory is not removed after removing the RPM, it probably indicates that there are some leftover files in the hierarchy, e.g. files that were not installed by the RPM package. After checking that you indeed don't need those files, you can remove the directory safely. -- Bazsi ________________________________________________________________________ ______ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
There are no "log" statements in that conf file. That's the problem. You declare a bunch of message sources, a bunch of filters and a bunch of destinations. You need "log" statements to declare that messages from source A matching filter B should go to destination C. -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Ashish Shrimali Sent: 30 September 2009 05:25 To: syslog-ng@lists.balabit.hu Subject: [syslog-ng] Syslog-ng3 : syslog-ng Digest, Vol 53, Issue 25 Hi, The syslog-ng conf file is @version:3.0 options { use_dns(persist_only); dns_cache_hosts(/etc/hosts); #chain_hostnames(yes); keep_hostname(yes); keep_timestamp(yes); }; source all_sources { # Solaris used streams to send msgs to Syslogd process, replicate for syslog-ng sun-streams("/dev/log" door("/etc/.syslog_door")); # sun-streams( door("/etc/.syslog_door")); # Messages generated internally by Syslog-ng internal(); # Listens on the specified udp port for incoming messages udp(ip(0.0.0.0) port(514)); }; destination local7apm { file("/var/log/syslog" template("$DATE $HOST $MSGHDR $MSGONLY\n")); }; #destination local7apm { udp("smarts5" port(514) spoof-source(yes)); }; #destination d_tcp { tcp("10.1.2.3" port(1999); localport(999)); }; #destination messages { file("/var/adm/messages" template("$DATE $HOST $MSG\n")); }; # Below 5 lines taken from existing configuration #destination sysmsg { file("/dev/sysmsg" template("$DATE $HOST $MSG\n")); }; #destination operator { usertty("operator" template("$DATE $HOST $MSG\n")); }; destination operator { usertty("operator"); }; #destination all_alerts { usertty("operator" template("$DATE $HOST $MSG\n")); }; destination all_alerts { usertty("operator"); }; #destination all_alerts1 { usertty("root" template("$DATE $HOST $MSG\n")); }; destination all_alerts1 { usertty("root"); }; #destination emergency { usertty("*" template("$DATE $HOST $MSG\n")); }; destination emergency { usertty("*"); }; # filter local7msgs { facility(local7) and level(info..emerg); }; filter messages { facility(kern, daemon, mail) and level(debug, info, notice, warning, err, crit alert, emerg); }; filter f_sysmsg { facility(kern, auth) and level(notice); }; filter f_all_errors { level(err); }; filter f_operator { facility(kern, daemon) and level(err); }; filter f_all_alerts { level(alert); }; filter f_all_alerts1 { level(alert); }; filter f_emergency { level(emerg); }; Thanks & Regards, Ashish Shrimali -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of syslog-ng-request@lists.balabit.hu Sent: Wednesday, September 30, 2009 9:52 AM To: syslog-ng@lists.balabit.hu Subject: syslog-ng Digest, Vol 53, Issue 25 Send syslog-ng mailing list submissions to syslog-ng@lists.balabit.hu To subscribe or unsubscribe via the World Wide Web, visit https://lists.balabit.hu/mailman/listinfo/syslog-ng or, via email, send a message with subject or body 'help' to syslog-ng-request@lists.balabit.hu You can reach the person managing the list at syslog-ng-owner@lists.balabit.hu When replying, please edit your Subject line so it is more specific than "Re: Contents of syslog-ng digest..." Today's Topics: 1. Re: Colon embedded in messages (:) (chris packham) 2. Re: Colon embedded in messages (:) (Evan Rempel) 3. Solaris Syslog-ng install lib error : syslog-ng Digest, Vol 53, Issue 22 (Ashish Shrimali) ---------------------------------------------------------------------- Message: 1 Date: Wed, 30 Sep 2009 09:49:59 +1300 From: "chris packham" <chris.packham@alliedtelesis.co.nz> Subject: Re: [syslog-ng] Colon embedded in messages (:) To: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu> Message-ID: <4AC329C7020000680001703F@gwia.alliedtelesyn.co.nz> Content-Type: text/plain; charset=US-ASCII
On 9/30/2009 at 9:26 AM, James Kelly <james.kelly@hmsinc.com> wrote: Thanks a lot. That seems to have worked a lot better but has introduced another problem. The messages are split up into too many lines and, when there is a match, the first of N lines are removed, still creating unwanted messages.
For example, a pretty small insert spans 3 lines... There was a match on the 2nd line, so it is not printed. However, the 1st and 3rd were.
2009-09-29T16:21:16-04:00 blahh-rep2 postgres [10440-1] postgres-blahh-10.100.10.34(15779)-4690-2009-09-29 16:21:16 EDT-LOG: statement: insert into 2009-09-29T16:21:16-04:00 blahh-rep2 postgres [10440-3] '1000000000000000');
I think you'll find is a postgres thing. syslog-ng does not split messages up based on size (at least as far as I've seen). I think there is some behaviour to do with carriage returns "\n" embedded in your log data but there are some config options to help with that, search the mailing list/admin guide for that if it turns out to be the case. ------------------------------ Message: 2 Date: Tue, 29 Sep 2009 14:19:46 -0700 From: Evan Rempel <erempel@uvic.ca> Subject: Re: [syslog-ng] Colon embedded in messages (:) To: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu> Message-ID: <4AC279F2.50205@uvic.ca> Content-Type: text/plain; charset=windows-1252; format=flowed As far as I know, the messages will always be split on multiple lines by postgresql, and you will need to match on the process number. In your example, the 10440 is the process number, and the -1, -2, -3 are the line numers of the messages that the single process creates. A single connection that runs multiple queries can create many messages, with ever increasing line number counts. Evan. James Kelly wrote:
Thanks a lot. That seems to have worked a lot better but has introduced another problem. The messages are split up into too many lines and, when there is a match, the first of N lines are removed, still creating unwanted messages.
For example, a pretty small insert spans 3 lines... There was a match on the 2nd line, so it is not printed. However, the 1st and 3rd were.
2009-09-29T16:21:16-04:00 blahh-rep2 postgres [10440-1] postgres-blahh-10.100.10.34(15779)-4690-2009-09-29 16:21:16 EDT-LOG: statement: insert into 2009-09-29T16:21:16-04:00 blahh-rep2 postgres [10440-3] '1000000000000000');
I played around with log_msg_size thinking I could control how much is on each line, but it didn't help. Am I missing something obvious? If a "fragment" of the message is matched, I would want the entire message to not be logged.
Honestly, I have not spent nearly as much time on this most recent problem, so if it is an RTFM situation, just let me know. I did do some reading and searching on this though and came up empty handed.
Thanks again for this great help, James
On Tue, Sep 29, 2009 at 3:43 PM, Evan Rempel <erempel@uvic.ca <mailto:erempel@uvic.ca>> wrote:
PostgeSQL can log to syslog using appropriately formated messages. Configure your postgresql.conf file to include the line
log_destination = 'syslog'
and then you will get lines in syslog like;
2009-09-29T12:42:10-07:00 hostname facility.level postgres[20288]: [32-1] postgresql statement
then all of the syslog-ng parsing, macros and templates will work correctly.
Evan.
James Kelly wrote: > Thanks. This is exactly what is happening. If, using a template, I > just use $PROGRAM, I get a bunch of lines with just "insert". > > If I use something like the following: > > template t_postgres_msgs { > template("$ISODATE $HOST $PROGRAM $MSG \n"); }; > > I no longer see the colons. However, if something is matched by the > filter, it still prints a blank message (hence, wasting a ton of space > and clogging up the logs). For example, this is what I get instead of > the matched sql statement: > > 2009-09-29T10:30:30-04:00 hcdb1-rep2 > > I thought the filter would just "trash" or not record the message at > all, not print the host and timestamp for each filtered message. > > Thanks a lot... I appreciate this help a lot. > James > > On Tue, Sep 29, 2009 at 10:05 AM, Srinivasan Sreenivasan > <srinivasan.srinivasan@sabre.com <mailto:srinivasan.srinivasan@sabre.com> > <mailto:srinivasan.srinivasan@sabre.com <mailto:srinivasan.srinivasan@sabre.com>>> wrote: > > I think Syslog-ng thinks insert is a program name. Use a template > with $PROGRAM in it to see if it prints ?insert? to confirm this. > > Solution: > Send a program name before your sql statement using a template. > > > > On 9/29/09 8:57 AM, "James Kelly" <james.kelly@hmsinc.com <mailto:james.kelly@hmsinc.com> > <mailto:james.kelly@hmsinc.com <mailto:james.kelly@hmsinc.com>>> wrote: > > Hello, > > I am attempting to implement syslog-ng on our servers hosting > postgresql databases. The general idea is to log "too much" at > the database level and then, using syslog filters, reduce it to > the information we want to actually hold in the syslog and send > to the log server. So far I am extremely pleased with how easy > it is to implement and well documented. However, I do have one > problem that is making it difficult to use. > > The problem is for each message that the filter matches, it does > not completely drop the message.. rather, it logs the date / > server / and a colon: > > For example, this is what I get for a message that is matched by > a filter: > > "Sep 29 09:43:29 hcdb1-rep2 :" > > I notice that even with the unmatched statements, there is a > colon. For example, in the postgresql log, I see: > > insert into "public"."table"(blah,blah,blah) values > (blah,blah,blah); > > but the same message once captured from syslog shows the > following in the log and logserver: > > Sep 29 09:43:29 hcdb1-rep2 insert: into > "public"."table"(blah,blah,blah) values (blah,blah,blah); > > *Note the colon after "insert".* I can't seem to figure out > where this is coming from or how to avoid it. It also causes > some filtering problems that I won't go into here so as to not > confuse the issues, but safe to say it is also related to the colon. > > I have spent a lot of time trying to figure this out and am at a > dead-end. Due to the amount of messages I need to filter out > (below is just one of the many filters I need to put in and have > tested with the same result), not being able to completely > filter these out is a killer. > > I am using the 3.0.4 open-source edition on Ubuntu 8. Here is > my config: > > ****************************************************************************************** > @version: 3.0 > > options { > }; > > ###### > # sources > source s_local { > # message generated by Syslog-NG > internal(); > # standard Linux log source (this is the default place for the > syslog() > # function to send logs to) > unix-stream("/dev/log"); > # messages from the kernel > file("/proc/kmsg" program_override("kernel: ")); > file("/var/log/postgresql/postgresql-8.3-main.log"); > }; > > > ###### > # destinations > destination d_messages { file("/var/log/messages"); }; > > ##### > # filters > > filter f_inserts2 { > not match("_health_central" value("MESSAGE")); > }; > > > destination d_logserver { tcp("internal.host.com <http://internal.host.com> > <http://internal.host.com> <http://internal.host.com> "); }; > > > > log { > source(s_local); > filter(f_inserts2); > destination(d_messages); > #destination(d_logserver); > }; > ****************************************************************************************** > > Thanks! > James Kelly > > ------------------------------------------------------------------------ > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > > >
-- Evan Rempel erempel@uvic.ca <mailto:erempel@uvic.ca> Senior Programmer Analyst 250.721.7691 Unix Services, University Systems, University of Victoria ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
-- Evan Rempel erempel@uvic.ca Senior Programmer Analyst 250.721.7691 Unix Services, University Systems, University of Victoria ------------------------------ Message: 3 Date: Wed, 30 Sep 2009 09:51:37 +0530 From: Ashish Shrimali <Ashish_Shrimali@infosys.com> Subject: [syslog-ng] Solaris Syslog-ng install lib error : syslog-ng Digest, Vol 53, Issue 22 To: "syslog-ng@lists.balabit.hu" <syslog-ng@lists.balabit.hu> Message-ID: <AE1A7E8803F348438B970CCDCA075C77115F91731A@BLRKECMBX07.ad.infosys.com> Content-Type: text/plain; charset="us-ascii" Hi, You need to install libnet. Steps are as follows: 1. gunzip and untar the libnet package. 2. cd <libnetdir> 3. ./configure 4. make 5. make install 6. While installing syslog-ng use this command.(with libnet) ./configure -enable-spoof-source -with-libnet=<libnet dir path>/libnet -enable-dynamic-linking It worked successfully for me. Thanks & Regards, Ashish Shrimali -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of syslog-ng-request@lists.balabit.hu Sent: Tuesday, September 29, 2009 7:36 PM To: syslog-ng@lists.balabit.hu Subject: syslog-ng Digest, Vol 53, Issue 22 Send syslog-ng mailing list submissions to syslog-ng@lists.balabit.hu To subscribe or unsubscribe via the World Wide Web, visit https://lists.balabit.hu/mailman/listinfo/syslog-ng or, via email, send a message with subject or body 'help' to syslog-ng-request@lists.balabit.hu You can reach the person managing the list at syslog-ng-owner@lists.balabit.hu When replying, please edit your Subject line so it is more specific than "Re: Contents of syslog-ng digest..." Today's Topics: 1. Syslog-ng 3 (Ashish Shrimali) 2. Re: Syslog-ng 3 (Fegan, Joe) 3. Re: sun solaris syslogng error (Balazs Scheidler) 4. Re: syslog-ng client machine IP address in message header (Balazs Scheidler) 5. Colon embedded in messages (:) (James Kelly) 6. Re: Colon embedded in messages (:) (Srinivasan Sreenivasan) ---------------------------------------------------------------------- Message: 1 Date: Tue, 29 Sep 2009 16:41:50 +0530 From: Ashish Shrimali <Ashish_Shrimali@infosys.com> Subject: [syslog-ng] Syslog-ng 3 To: "syslog-ng@lists.balabit.hu" <syslog-ng@lists.balabit.hu> Message-ID: <AE1A7E8803F348438B970CCDCA075C77115F917101@BLRKECMBX07.ad.infosys.com> Content-Type: text/plain; charset="us-ascii" Hi All, My Syslog-ng is running , which I can see by ps -ef command, but still it is not writing to the log file. I have specified the log file in the conf dir. Any inputs would be highly appreciated. Thanks & Regards, Ashish Shrimali -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of syslog-ng-request@lists.balabit.hu Sent: Tuesday, September 29, 2009 3:11 PM To: syslog-ng@lists.balabit.hu Subject: syslog-ng Digest, Vol 53, Issue 21 Send syslog-ng mailing list submissions to syslog-ng@lists.balabit.hu To subscribe or unsubscribe via the World Wide Web, visit https://lists.balabit.hu/mailman/listinfo/syslog-ng or, via email, send a message with subject or body 'help' to syslog-ng-request@lists.balabit.hu You can reach the person managing the list at syslog-ng-owner@lists.balabit.hu When replying, please edit your Subject line so it is more specific than "Re: Contents of syslog-ng digest..." Today's Topics: 1. Solaris Syslog-ng install lib error (list@net-secured.com) 2. Re: Solaris Syslog-ng install lib error (Srinivasan Sreenivasan) 3. sun solaris syslogng error (list@net-secured.com) 4. syslog-ng client machine IP address in message header (Jain, Vaibhav (GE Healthcare)) ---------------------------------------------------------------------- Message: 1 Date: Mon, 28 Sep 2009 03:52:33 -0700 From: list@net-secured.com Subject: [syslog-ng] Solaris Syslog-ng install lib error To: "syslogng " <syslog-ng@lists.balabit.hu> Message-ID: <20090928035233.9419518bf8837c8c492fb3b5d9b71733.3b4f820868.wbe@email03.secureserver.net> Content-Type: text/plain; charset="us-ascii" An HTML attachment was scrubbed... URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20090928/9ae9e2ac/at... ------------------------------ Message: 2 Date: Mon, 28 Sep 2009 08:00:09 -0500 From: Srinivasan Sreenivasan <srinivasan.srinivasan@sabre.com> Subject: Re: [syslog-ng] Solaris Syslog-ng install lib error To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Message-ID: <C6E61D89.5EF6%srinivasan.srinivasan@sabre.com> Content-Type: text/plain; charset="us-ascii" The PKG_CONFIG_PATH should include directories that have .pc files in them. On 9/28/09 5:52 AM, "list@net-secured.com" <list@net-secured.com> wrote:
Hi I tryed to install on Sparc v. 10 syslog-ng v3.0.4, runnning: ./configure --enable-sun-streams, --enable-sun-door gives this warning: configure: WARNING: unrecognized options: --enable-sun-streams, --enable-sun-door
any ideas?
furthermore, I get this error: checking for GLIB - version >= 2.10.1... no *** Could not run GLIB test program, checking why... *** The test program failed to compile or link. See the file config.log for the *** exact error that occured. This usually means GLIB is incorrectly installed. configure: error: Cannot find GLIB version >= 2.10.1: is pkg-config in path?
=== Adding the path below : produces another error
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/:/usr/lib/pkgconfig/
checking for LIBDBI... Package dbi was not found in the pkg-config search path. Perhaps you should add the directory containing `dbi.pc' to the PKG_CONFIG_PATH environment variable No package 'dbi' found no checking for dbi_initialize in -ldbi... no checking pthread.h usability... yes checking pthread.h presence... yes checking for pthread.h... yes checking for pthread_create in -lpthread... yes checking for LIBNET... no checking whether to enable Sun STREAMS support... yes checking whether to enable Sun door support... yes checking whether to enable IPv6 support... yes checking whether to enable SQL support... no checking whether to enable Linux capability support... no checking whether to enable PCRE support... yes checking for pkg-config... (cached) /usr/bin/pkg-config checking pkg-config is at least version 0.14... yes checking for GLIB - version >= 2.10.1... yes (version 2.14.1) checking for EVTLOG... yes checking for LIBNET... configure: error: libnet-config not found
thanks
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
participants (5)
-
Ashish Shrimali
-
Balazs Scheidler
-
Fegan, Joe
-
Jain, Vaibhav (GE Healthcare)
-
Siem Korteweg