On Mon, Oct 03, 2005 at 08:44:38AM +0200, Roberto Nibali wrote:

Nate Campi wrote:

...

I built 1.9.5 and eventlog-0.2.3+20050116+1856 on Solaris 8. It wouldn't start with command line arguments like "--pidfile=/foo/syslog-ng.pid", I had to change it to "-p /foo/syslog-ng.pid". The usage message it spits out claims that the --long syntax is supported.

Funny, what's the exact error message, if there is any? And do other long options work? The needed code (unless I'm interpreting it in a wrong way) is there. It could only be that popt is not linked into your binary.

Just my 2 cents ...

I'm sure you're right, I'll follow up with this and make sure. Thanks for looking at it. -- Nate "Habit is habit, and not to be flung out of the window by any man, but coaxed downstairs a step at a time." - Pudd'nhead Wilson's Calendar(1894) - Samuel Clemens

Hello,

I'd rather change to using libpopt (although it is another external dependency, hmm), it generates usage and help screens nicely, so that's sure I won't forget to add them there myself.

Two options IMHO: 1. Use the GLib internal parser [Gg]_option*. I don't exactly know if this depends on popt, since I have never used it before myself. 2. Use the glibc internally provided command line parser called argp which superseeds getopt. More information and documentation can be found in the glibc manual: http://www.gnu.org/software/libc/manual/html_node/Argp.html#Argp It's a sexy interface ;), so I'm in favour of option 2. Should be portable, never tested it though. YMMV. Cheers, Roberto Nibali, ratz -- ------------------------------------------------------------- addr://Kasinostrasse 30, CH-5001 Aarau tel://++41 62 823 9355 http://www.terreactive.com fax://++41 62 823 9356 ------------------------------------------------------------- terreActive AG Wir sichern Ihren Erfolg -------------------------------------------------------------

On Mon, Oct 10, 2005 at 07:13:51AM -0700, Evan Rempel wrote:

This is a little off topic for syslog-ng, but I bring it up here because syslog-ng has 90% of the code already written to accomplish this.

I need a tool to get data into my syslog-ng architecture. There are a number of programs that log to files or that standard output only. I can use tools like chronolog to manage the rollover of the files but that does not get the data into syslog-ng. I can pipe the data into "logger -pxxx.yyy -tag zzz" but this sends all of the messages with the same priority.

I would like a tool that can filter the input using regular expressions and then syslog the message to a specified facility.level and tag.

I wrote a tool a while back called "relogger.pl" which takes an existing syslog data file and re-injects each line back into syslog. It was meant for testing logsurfer or anything else that parsed syslog data, so you could have a "test" stream of log data sent in real-time. It's basically the same idea as logger, but it parses the data and can do things like figure out the delay between two messages and simulate the delay when re-logging the message. It almost does what you want - except right now it only reads syslog-formatted files. You'd have to teach it how to deal with normal text files, and then add parsing for the data in question - maybe add a command line option like: "--facilitymatch /PATTERN/=crit" (if data matches PATTERN, syslog it with facility "crit"). relogger.pl is attached. -- Ed

Check out looperng - perhaps it will help do what you need... http://muthanna.com/looperng/ -----Original Message----- From: Evan Rempel [mailto:erempel@uvic.ca] Sent: Thursday, October 13, 2005 1:25 AM To: Syslog-ng users' and developers' mailing list Subject: [syslog-ng] Re: logger-ng needed I know I'm not supposed to reply to my own posts... After a lot more thought, I think that all I really want to do is ask for one new source to be added, and one new destination to be added. source input { stdin(); }; to obtain data from the standard input. destination syslog.userinfo { syslog( facility(user) priority(info) tag(myApp) template(xxx)); }; to write messages into the local syslog mechanism. I realize that this would cause syslog-ng to process each of these messages twice, but the source of the messages do not yet have a datestamp, facility, priority, tag or host associated with them (ie they are not syslog format yet). This is the same as messages comming from a pipe. It would be nice to regexp match these messages an syslog them with appropriate facility.priority information, not to mention a meaningfull tag. I realize that this option could result in a circular path for a message, but with some care, the messages from a given source (file, pipe, standard input etc) can be passed to syslog which could in tern be loged to a file locally, or transfered to a central logger. This would be sufficient to get most, if not all, log messages fom all applications into a single syslog architecture. Apache and mysql are the big two, but Tivoli Storage manager is also a problem at our site. Thanks for the consideration. Evan. On Mon, 10 Oct 2005, Evan Rempel wrote:

Date: Mon, 10 Oct 2005 07:13:51 -0700 (PDT) From: Evan Rempel <erempel@UVic.CA> To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: logger-ng needed

This is a little off topic for syslog-ng, but I bring it up here because syslog-ng has 90% of the code already written to accomplish this.

I need a tool to get data into my syslog-ng architecture. There are a number of programs that log to files or that standard output only. I can use tools like chronolog to manage the rollover of the files but that does not get the data into syslog-ng. I can pipe the data into "logger -pxxx.yyy -tag zzz" but this sends all of the messages with the same priority.

I would like a tool that can filter the input using regular expressions and then syslog the message to a specified facility.level and tag.

Syslog-ng has all of the code for reading from a variety of sources. Syslog-ng has all of the code to filter based on regular expressions All that needs to be added is applying the facility.level, tag and syslogging the message.

I would like the developers of syslog-ng to consider writing a logger-ng to go along with it. A sample configuration file is included below just so that I get all of my thoughts into this one e-mail. If no configuration file was specified, then it could oporate just like logger, or it could simply be a new tool rather than a replacement.

Thanks for a great syslog deamon.

Evan Rempel

------------------ example configuration file ------------------------------- source my_app { unix-stream("/tmp/logpipe" tag(my_apps_name)); }; source second_app { file("/var/log/http_access" tag(second_app_name)); };

filter find_errors(match(*error*); }

destination user.info { facility(user); level(info); }; destination user.error { facility(user); level(error); }; destination local4.info { facility(local4); level(info); }; destination local4.error { facility(local4); level(error); }; # perhaps the destinations are predefined as I think they will always be # the syslog priorities.

log { source(my_app); filter(find_errors); destination(user.error); }; log { source(my_app); destination(user.info); flags(fallback);

log { source(second_app); filter(find_errors); destination(local4.error); }; log { source(second_app); destination(local4.info); flags(fallback);

# the fallback option would only have the scope for the given source. Each # source would require its own fallback flag.

------------------ end example configuration file ---------------------------

_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

...

When skimming through the old code I extract that the regexp will be applied to all hostnames matching in a log message. Dumb question: what's the use case of this and why isn't there a general regexp function?

It's for solaris-style network messages that leave out the hostname, but then send a badly formatted message (I know this is from a Linux host but bear with me):

<13>Oct 7 14:03:28 device eth0 entered promiscuous mode

The above message should be easy to figure out that there's no hostname because the second field doesn't have a colon like a message from postfix would have (I'm just arbitrarily assigning 13 as the PRI):

<13>Oct 7 14:22:56 postfix/smtpd[9753]: lost connection after RCPT from 1Cust4795.an3.chi30.da.uu.net[63.26.50.187]

But what happens if you have a program name on a solaris host that has a space in it?

<13>Oct 7 14:22:56 ctld 8.9[123]: this is a dumb message

...then syslog-ng will assume that the hostname is ctld - when that's not right, the program name is "ctld 8.9". Using bad_hostnames() we can tell syslog-ng which strings our site sends that really aren't hostnames.

This should be in the man page ;). So how do you set bad_hostname in your example? bad_hostname("ctld")? But in this case you better not have a host named ctld.

Simple as that. Comes in handy at every site for things like "last message repeated xx times", I'd imagine. When you use the $HOST macro this becomes critical to avoid using the wrong hostnames.

I need to check out how we do the syslog configuration for our customers.

Make sense?

Thanks, Nate. Yes it does, but does it handle all possible cases? Maybe the Pareto principle applies ... Thanks and regards, Roberto Nibali, ratz -- echo '[q]sa[ln0=aln256%Pln256/snlbx]sb3135071790101768542287578439snlbxq'|dc

Yeah, but then you wouldn't even think of this option in that case.

True.

At least one of the sample configurations on my site has one:

bad_hostname("^(ctld.|cmd|tmd|last)$");

...

...

Make sense? Thanks, Nate. Yes it does, but does it handle all possible cases? Maybe the Pareto principle applies ...

If you're saying that this is not important, maybe it's not to you, but people using the really common destination like this:

I don't mean to imply that this is not important, I just like to discuss the correctness of the implementation at this point. Your explanation before rendered the necessity for such a feature perfectly clear.

destination hosts { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/$FACILITY$YEAR$MONTH$DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); };

...will see all kinds of crazy hostname directories that don't match any host. Perhaps they should not use keep_hostname and only use IPs - which sounds great.

We also use such destination, it's a typical "config" for central loghost servers. I normally don't configure our projects so I'm not too fluent with the options. I'm certain we use this option as well in our configs.

BUT WAIT - that means "ctld" will be tossed and you'll permanently only have "8.9" as the program name for all your "ctld 8.9" processes. Same goes for other "bad hostnames".

I need to think a bit about this, maybe we find an alternative solution.

If I remember correctly this is the reason Bazsi was convinced to create the bad_hostname option, he saw that information was actually lost when you tried to "do the right thing."

The fruits of this labour can be found in src/log.c:parse_log_msg(). Thanks for your patience and yes, it's definitely missing in 1.9.x. Regards, Roberto Nibali, ratz -- echo '[q]sa[ln0=aln256%Pln256/snlbx]sb3135071790101768542287578439snlbxq'|dc