[syslog-ng] Tracking down a failure between NG and FIFO

[Rob Munsch]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Addendum:  *GAH!*

This is a basically unattended system.  No new hosts were added.  I see
no glut of events that could have caused this.  And in any event, with
syslog-ng / mysql / stunnel / the pipe all shut down and started over, i
should be seeing SOMETHING, right?

Now for your amusement:

- -----
Aug 31 18:35:51 s_all at isis syslog-ng[6215]: STATS: dropped 0
Sep  1 06:35:51 s_all at isis syslog-ng[6215]: STATS: dropped 0
Sep  1 18:35:51 s_all at isis syslog-ng[6215]: STATS: dropped 0
Sep  2 06:35:51 s_all at isis syslog-ng[6215]: STATS: dropped 0
Sep  2 18:35:51 s_all at isis syslog-ng[6215]: STATS: dropped 0
Sep  3 06:35:51 s_all at isis syslog-ng[6215]: STATS: dropped 0
Sep  3 18:35:51 s_all at isis syslog-ng[6215]: STATS: dropped 0
Sep  4 06:35:51 s_all at isis syslog-ng[6215]: STATS: dropped 0
Sep  4 18:35:51 s_all at isis syslog-ng[6215]: STATS: dropped 35136
Sep  5 06:35:51 s_all at isis syslog-ng[6215]: STATS: dropped 2237082
Sep  5 18:35:51 s_all at isis syslog-ng[6215]: STATS: dropped 2485684
Sep  6 06:35:51 s_all at isis syslog-ng[6215]: STATS: dropped 2225476
Sep  6 18:35:52 s_all at isis syslog-ng[6215]: STATS: dropped 947716
Sep  7 06:35:52 s_all at isis syslog-ng[6215]: STATS: dropped 37679
Sep  7 18:35:52 s_all at isis syslog-ng[6215]: STATS: dropped 63702
- ----

I do not yet see anything on the network or the various systems here
from the 4th that could account for this insanity o_O.


Rob Munsch wrote:
> Hello,
> 
> i've had a running central loghost server for months on end without any
> problems.  I've noticed however that something seems to have died and i
> can't pinpoint any changes made to any of the involved systems.
> 
> After some testing, the central logserver is getting remote messages via
> stunnel; syslog-ng is logging locally; and if i tell it to put
> everything into a file, the remote as well as local log entries appear
> in the file.
> 
> MySQL seems to be running and that end seems OK.  The point of failure
> seems to be that syslog-ng is no longer putting anything into the named
> pipe.  MySQL is patiently waiting for something to appear there to read;
> syslog-ng will log to any other destination; but the pipe destination
> does not seem to be working.
> 
> How can i observe this process?  I can't figure out how to "see" -ng's
> attempts to log to the pipe destination.  The destination definition in
> question is
> 
> -----
> # Local MySQL desination
> destination d_mysql {
>         pipe("/var/run/mysql.pipe"
>                 template("INSERT INTO logs (host, facility, priority,
> level, tag, timestamp, program, msg, seq)
>                 VALUES (
> '$HOST','$FACILITY','$PRIORITY','$LEVEL','$TAG','$ISODATE','$PROGRAM','$MSG','$SEQ'
> );\n")
>         template_escape(yes)
>         );
> };
> -----
> 
> exactly as it has been for months.  The log directives are
> 
> -----
> # Sending everything to the MySQL table...
> log {
>         source(s_all);
>         destination(d_mysql);
> };
> 
> # ... and send incoming logs there as well.
> log {
>         source(stunnel);
>         destination(d_mysql);
> ###     destination(df_messages);  was testing.
> };
> -----
> 
> As you can see by the commented line above, i had remote logs going to
> local messages dest: that worked fine.  Instant results.
> 
> Any help would be appreciated, as i'm kind of at a loss.
> 
> 
> 
> 
_______________________________________________
syslog-ng maillist  -  syslog-ng at lists.balabit.hu
https://lists.balabit.hu/mailman/listinfo/syslog-ng
Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

- --
Rob Munsch
Solutions For Progress IT
www.solutionsforprogress.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFBxdGBvBcJFK6xYURAo2TAJ9eigG68J1xVZJOKSZpN3F37E62HQCcDpjJ
alTxr+tUbK3EEhSYs5FxyBs=
=Wh7r
-----END PGP SIGNATURE-----