replacing part of prog name with hostname
I have syslog-ng 1.5.24 on solaris 8, reading from /etc/.syslog_door and I have a log entry like this: Dec 31 13:48:15 larry 6.0[8704]: [ID 702911 local0.warning] [0] Can't stat file in FlushFile [news/PointCast]: No such file or directory ...but the program name was sent from the app was: "ctlds 6.0[8704]:" and syslog-ng replaced the first part of the messed up program name with the host's name. In the next version, can syslog-ng "learn" that it don't get a hostname from solaris ever and that the entire text coming in is actually the log message? I'm losing information this way. I wonder how many other apps split up the program name and lose data - most people would never know as end users. -- Nate Campi http://www.campin.net An Emacs reference mug is what I want. It would hold ten gallons of coffee. -- Steve VanDevender And, no doubt, have a lid that could only be removed with an obscure finger combination requiring both hands. (Ctrl-Alt-Meta-X gimme-the-damn-coffee) -- William Beegle
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi , On Tuesday 31 December 2002 23:05, you wrote:
I have syslog-ng 1.5.24 on solaris 8, reading from /etc/.syslog_door and I have a log entry like this:
Dec 31 13:48:15 larry 6.0[8704]: [ID 702911 local0.warning] [0] Can't stat file in FlushFile [news/PointCast]: No such file or directory
...but the program name was sent from the app was: "ctlds 6.0[8704]:" and syslog-ng replaced the first part of the messed up program name with the host's name.
In the next version, can syslog-ng "learn" that it don't get a hostname from solaris ever and that the entire text coming in is actually the log
I looked at my logs from Solaris 8 systems using standard solaris syslog and with the exception of some kernel messages the hostname is perfectly ok. Both the 'HOST' and 'FULLHOST' macros in the syslog-ng config file return the proper host name. The only problem are some kernel messages in which the hostname indeed contains the first one or two words of the message as does the 'PROGRAM' field. But even then I found that the 'FULLHOST' macro returns the correct host in its last field (like 'Error/<hostname>'). Possibly this has to do with some awkward way the kernel writes its messages.
message? I'm losing information this way. I wonder how many other apps split up the program name and lose data - most people would never know as end users.
- -- Sander de Boer - -- My public key is available at http://www.sanderscorner.com/file/pgp-pubkey.txt -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.7 (GNU/Linux) iD8DBQE+E3DIhNL8OKskREIRAtLrAJ0Wgs8QSXCiuVQjhSp1IMpZc3rapgCg7Amz Qu/Y/0Ult7I5FJLL4uvfUI4= =k6V0 -----END PGP SIGNATURE-----
On Wed, Jan 01, 2003 at 11:50:33PM +0100, Sander de Boer wrote:
, On Tuesday 31 December 2002 23:05, you wrote:
I have syslog-ng 1.5.24 on solaris 8, reading from /etc/.syslog_door and I have a log entry like this:
Dec 31 13:48:15 larry 6.0[8704]: [ID 702911 local0.warning] [0] Can't stat file in FlushFile [news/PointCast]: No such file or directory
...but the program name was sent from the app was: "ctlds 6.0[8704]:" and syslog-ng replaced the first part of the messed up program name with the host's name.
In the next version, can syslog-ng "learn" that it don't get a hostname from solaris ever and that the entire text coming in is actually the log
I looked at my logs from Solaris 8 systems using standard solaris syslog and with the exception of some kernel messages the hostname is perfectly ok. Both the 'HOST' and 'FULLHOST' macros in the syslog-ng config file return the proper host name.
The problem is when part of the program name looks like the hostname. Since you should never actually get a hostname when getting input from the syscall, syslog-ng should not behave this way, IMO. -- Nate Campi http://www.campin.net "The mind is everything. What you think you become." - Buddha
On Tue, Dec 31, 2002 at 02:05:34PM -0800, Nate Campi wrote:
I have syslog-ng 1.5.24 on solaris 8, reading from /etc/.syslog_door and I have a log entry like this:
Dec 31 13:48:15 larry 6.0[8704]: [ID 702911 local0.warning] [0] Can't stat file in FlushFile [news/PointCast]: No such file or directory
...but the program name was sent from the app was: "ctlds 6.0[8704]:" and syslog-ng replaced the first part of the messed up program name with the host's name.
In the next version, can syslog-ng "learn" that it don't get a hostname from solaris ever and that the entire text coming in is actually the log message? I'm losing information this way. I wonder how many other apps split up the program name and lose data - most people would never know as end users.
the problem is ctlds sends a space in the program name tag, thus syslog-ng interprets 'ctlds' as hostname and '6.0' as program name. as keep_hostname() is set to no it rewrites originating host name. Try setting keep_hostname() to yes, it will not touch the hostname then. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
On Thu, Jan 02, 2003 at 10:43:45AM +0100, Balazs Scheidler wrote:
the problem is ctlds sends a space in the program name tag, thus syslog-ng interprets 'ctlds' as hostname and '6.0' as program name. as keep_hostname() is set to no it rewrites originating host name.
Try setting keep_hostname() to yes, it will not touch the hostname then.
Right but "ctlds" isn't the hostname, larry is. My logs are wrong either way - I either lose half my program name or get the wrong hostname. Either way I lose. Arg, I actually wrote my own syslog daemon to relay and rewrite all these last year. The only way to fix this with syslog-ng would be to add a feature like options { bad_hostnames("ctlds","last"); }; When syslog-ng sees either of these: Jan 2 15:06:47 ctlds 6.0[11718]: [0] Request error (500): Template processing error Jan 2 15:06:47 last message repeated 123 times It knows to actually shift the message over one place to the right and stick the value of the $FULLHOST_FROM macro in there. Even if I tried templating out the message on my own syslog-ng will still think that "ctlds" or "last" isn't part of the message and it'll get lost. -- Nate Campi http://www.campin.net Radioactive cats have 18 half-lives.
On Thu, Jan 02, 2003 at 03:21:56PM -0800, Nate Campi wrote:
On Thu, Jan 02, 2003 at 10:43:45AM +0100, Balazs Scheidler wrote:
the problem is ctlds sends a space in the program name tag, thus syslog-ng interprets 'ctlds' as hostname and '6.0' as program name. as keep_hostname() is set to no it rewrites originating host name.
Try setting keep_hostname() to yes, it will not touch the hostname then.
Right but "ctlds" isn't the hostname, larry is. My logs are wrong either way - I either lose half my program name or get the wrong hostname. Either way I lose.
Arg, I actually wrote my own syslog daemon to relay and rewrite all these last year. The only way to fix this with syslog-ng would be to add a feature like
options { bad_hostnames("ctlds","last"); };
When syslog-ng sees either of these:
Jan 2 15:06:47 ctlds 6.0[11718]: [0] Request error (500): Template processing error Jan 2 15:06:47 last message repeated 123 times
It knows to actually shift the message over one place to the right and stick the value of the $FULLHOST_FROM macro in there. Even if I tried templating out the message on my own syslog-ng will still think that "ctlds" or "last" isn't part of the message and it'll get lost.
the last messages case should be covered by the function parse_log_msg() it recognizes message repetitions and takes the complete message as message with no host or program name: /* Now, try to tell if it's a "last message repeated" line */ if (left >= sizeof(repeat_msg_string) && !memcmp(src, repeat_msg_string, sizeof(repeat_msg_string) - 1)) { ; /* It is. Do nothing since there's no hostname or program name coming. */ } I will think about the bad hostname feature. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
On Thu, Jan 02, 2003 at 10:43:45AM +0100, Balazs Scheidler wrote:
the problem is ctlds sends a space in the program name tag, thus syslog-
ng
interprets 'ctlds' as hostname and '6.0' as program name. as keep_hostname() is set to no it rewrites originating host name.
Try setting keep_hostname() to yes, it will not touch the hostname then.
Right but "ctlds" isn't the hostname, larry is. My logs are wrong either way - I either lose half my program name or get the wrong hostname. Either way I lose.
Arg, I actually wrote my own syslog daemon to relay and rewrite all these last year. The only way to fix this with syslog-ng would be to add a feature like
options { bad_hostnames("ctlds","last"); };
When syslog-ng sees either of these:
Jan 2 15:06:47 ctlds 6.0[11718]: [0] Request error (500): Template processing error Jan 2 15:06:47 last message repeated 123 times
It knows to actually shift the message over one place to the right and stick the value of the $FULLHOST_FROM macro in there. Even if I tried templating out the message on my own syslog-ng will still think that "ctlds" or "last" isn't part of the message and it'll get lost.
Better is to implement source templates. This way you can precisely describe input line, so if you know your source never appends host name, you just omit this from template. Something like source s_stream { unix-stream("/dev/log" max-connections(10)); template(DATE PROG[PID]:... );}; Cheers -andrey
On Sat, Jan 04, 2003 at 02:55:51PM +0300, Borzenkov Andrey wrote:
It knows to actually shift the message over one place to the right and stick the value of the $FULLHOST_FROM macro in there. Even if I tried templating out the message on my own syslog-ng will still think that "ctlds" or "last" isn't part of the message and it'll get lost.
Better is to implement source templates. This way you can precisely describe input line, so if you know your source never appends host name, you just omit this from template. Something like
source s_stream { unix-stream("/dev/log" max-connections(10)); template(DATE PROG[PID]:... );};
You missed the fact that before you ever get around to templating, part of the program name is *already* lost. It's too late for that. -- Nate Campi http://www.campin.net "Don't go around saying the world owes you a living. The world owes you nothing. It was here first." - Samuel Clemens
On Sat, Jan 04, 2003 at 12:50:30PM -0800, Nate Campi wrote:
On Sat, Jan 04, 2003 at 02:55:51PM +0300, Borzenkov Andrey wrote:
It knows to actually shift the message over one place to the right and stick the value of the $FULLHOST_FROM macro in there. Even if I tried templating out the message on my own syslog-ng will still think that "ctlds" or "last" isn't part of the message and it'll get lost.
Better is to implement source templates. This way you can precisely describe input line, so if you know your source never appends host name, you just omit this from template. Something like
source s_stream { unix-stream("/dev/log" max-connections(10)); template(DATE PROG[PID]:... );};
You missed the fact that before you ever get around to templating, part of the program name is *already* lost. It's too late for that.
he meant 'source templates' to specify how to parse messages. while that would be interesting it is less than trivial. I'm trying to hack a bad_hostname() feature right now. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
hi, I try to install a central loghost on Solaris9 server. My first approch is to have syslog-ng only on loghost, all other machines ( some Solaris servers + tens of Linux workstations & servers) run own syslog which is configured to send everything to the loghost - which I understand is the most recommended practice. ====(in syslog.conf)=== *.debug @loghost === In this approach I like that I don't have to change software on all machines (exchange from syslog -> syslog-ng) , but I don't like that there is anormous amount of unnecessery information send to the loghost. My question is : isn't it better to filter messages locally, and send to the loghost information of some importance? Do I risk to loose some important info, by sending all of *.warn and higer to loghost? How can I get rid of repeated lines in incoming messages? regards Longina -- Longina Przybyszewska, system programmer Dept. of Math. & Comp. Sci. - IMADA University of Southern Denmark, Odense Campusvej 55,DK-5230 Odense M, Denmark tel: +45 6550 2359 - http://www.imada.sdu.dk email: longina@imada.sdu.dk --
On Mon, Jan 06, 2003 at 10:24:27AM +0100, Balazs Scheidler wrote:
On Sat, Jan 04, 2003 at 12:50:30PM -0800, Nate Campi wrote:
On Sat, Jan 04, 2003 at 02:55:51PM +0300, Borzenkov Andrey wrote:
It knows to actually shift the message over one place to the right and stick the value of the $FULLHOST_FROM macro in there. Even if I tried templating out the message on my own syslog-ng will still think that "ctlds" or "last" isn't part of the message and it'll get lost.
Better is to implement source templates. This way you can precisely describe input line, so if you know your source never appends host name, you just omit this from template. Something like
source s_stream { unix-stream("/dev/log" max-connections(10)); template(DATE PROG[PID]:... );};
You missed the fact that before you ever get around to templating, part of the program name is *already* lost. It's too late for that.
he meant 'source templates' to specify how to parse messages. while that would be interesting it is less than trivial.
I'm trying to hack a bad_hostname() feature right now.
I'm finished, Nate can you test if this patch is good for your problem? Usage: new global option named bad_hostname(), expects a regular expression which should match all bad hostnames: options { bad_hostname("^ctld$"); }; It is currently a global option, and I don't think it will become a per-source option in 1.5.x. I only tested it on Linux. Index: ChangeLog =================================================================== RCS file: /var/cvs/syslog-ng/syslog-ng/ChangeLog,v retrieving revision 1.67 diff -u -r1.67 ChangeLog --- ChangeLog 5 Dec 2002 16:23:50 -0000 1.67 +++ ChangeLog 6 Jan 2003 12:42:54 -0000 @@ -1,3 +1,19 @@ +2003-01-06 Balazs Scheidler <bazsi@balabit.balabit> + + * af*.c: updated to call make_log_reader according to the latest + interface change + + * src/log.c (make_log_info): expect a new argument (bad_hostname), + (parse_log_msg): check if the hostname matches bad_hostname, and + if it does do not interpret it as a hostname + + * src/sources.c (make_log_reader): new argument, a regular + expression which matches bad hostnames + +2002-12-18 Balazs Scheidler <bazsi@balabit.balabit> + + * configure.in: bumped version number to 1.5.24 + 2002-12-05 Balazs Scheidler <bazsi@balabit.balabit> * src/afinet.c (inet_address_setip): check addr if it is NULL Index: src/affile.c =================================================================== RCS file: /var/cvs/syslog-ng/syslog-ng/src/affile.c,v retrieving revision 1.56 diff -u -r1.56 affile.c --- src/affile.c 30 Oct 2002 19:28:11 -0000 1.56 +++ src/affile.c 6 Jan 2003 12:42:54 -0000 @@ -162,7 +162,7 @@ if (do_open_file(self->name, flags, -1, -1, -1, -1, -1, -1, 0, &fd)) { lseek(fd, 0, SEEK_END); self->src = io_read(make_io_fd(cfg->backend, fd, ol_string_use(self->name)), - make_log_reader(0, self->prefix, cfg->log_msg_size, self->pad_size, cfg->check_hostname ? LF_CHECK_HOSTNAME : 0, c), + make_log_reader(0, self->prefix, cfg->log_msg_size, self->pad_size, cfg->check_hostname ? LF_CHECK_HOSTNAME : 0, cfg->bad_hostname, c), NULL); self->res = REMEMBER_RESOURCE(cfg->resources, &self->src->super.super); return ST_OK | ST_GOON; Index: src/afinet.c =================================================================== RCS file: /var/cvs/syslog-ng/syslog-ng/src/afinet.c,v retrieving revision 1.21 diff -u -r1.21 afinet.c --- src/afinet.c 5 Dec 2002 16:23:50 -0000 1.21 +++ src/afinet.c 6 Jan 2003 12:42:54 -0000 @@ -89,13 +89,13 @@ notice("AF_INET client connected from %S, port %i\n", inet->ip, inet->port); io_read(self->client, - make_log_reader(0, NULL, cfg->log_msg_size, 0, cfg->check_hostname ? LF_CHECK_HOSTNAME : 0, c), + make_log_reader(0, NULL, cfg->log_msg_size, 0, cfg->check_hostname ? LF_CHECK_HOSTNAME : 0, cfg->bad_hostname, c), make_afsocket_source_close_callback(self)); } else { /* SOCK_DGRAM */ io_read(self->client, - make_log_reader(1, NULL, cfg->log_msg_size, 0, cfg->check_hostname ? LF_CHECK_HOSTNAME : 0, c), + make_log_reader(1, NULL, cfg->log_msg_size, 0, cfg->check_hostname ? LF_CHECK_HOSTNAME : 0, cfg->bad_hostname, c), make_afsocket_source_close_callback(self)); } Index: src/afstreams.c =================================================================== RCS file: /var/cvs/syslog-ng/syslog-ng/src/afstreams.c,v retrieving revision 1.14 diff -u -r1.14 afstreams.c --- src/afstreams.c 21 Aug 2002 14:03:50 -0000 1.14 +++ src/afstreams.c 6 Jan 2003 12:42:54 -0000 @@ -77,6 +77,7 @@ (name stream_fd) (super nonblocking_fd) (vars + (bad_hostname special-struct regex_t #f regfree) (pipe object log_handler))) */ @@ -145,7 +146,7 @@ length = eol - bol; if (length) { - li = make_log_info(length, bol, NULL, 0); + li = make_log_info(length, bol, NULL, 0, NULL); li->pri = pri; HANDLE_LOG(self->pipe, li); } @@ -160,6 +161,7 @@ struct nonblocking_fd *io_stream_get(struct io_backend *backend, int fd, + UINT8 *hostname_re, struct log_handler *pipe) { NEW(stream_fd, f); @@ -169,6 +171,10 @@ f->super.read = stream_read_callback; f->super.want_read = 1; f->pipe = pipe; + if (hostname_re == NULL) + regcomp(&self->bad_hostname, "^$", REG_NOSUB | REG_EXTENDED); + else + regcomp(&self->bad_hostname, hostname_re, REG_NOSUB | REG_EXTENDED); return &f->super; } @@ -220,7 +226,7 @@ close(fd); return ST_FAIL | ST_QUIT; } - self->stream_fd = io_stream_get(cfg->backend, fd, c); + self->stream_fd = io_stream_get(cfg->backend, fd, cfg->bad_hostname, c); REMEMBER_RESOURCE(cfg->resources, &self->stream_fd->super); Index: src/afunix.c =================================================================== RCS file: /var/cvs/syslog-ng/syslog-ng/src/afunix.c,v retrieving revision 1.24 diff -u -r1.24 afunix.c --- src/afunix.c 4 Sep 2002 14:52:25 -0000 1.24 +++ src/afunix.c 6 Jan 2003 12:42:54 -0000 @@ -51,7 +51,7 @@ CAST(afsocket_source_connection, self, c); io_read(self->client, - make_log_reader(!!(self->owner->flags & AFSOCKET_DGRAM), NULL, cfg->log_msg_size, 0, cfg->check_hostname ? LF_CHECK_HOSTNAME : 0, c), + make_log_reader(!!(self->owner->flags & AFSOCKET_DGRAM), NULL, cfg->log_msg_size, 0, cfg->check_hostname ? LF_CHECK_HOSTNAME : 0, cfg->bad_hostname, c), make_afsocket_source_close_callback(self)); return ST_OK | ST_GOON; Index: src/center.c =================================================================== RCS file: /var/cvs/syslog-ng/syslog-ng/src/center.c,v retrieving revision 1.19 diff -u -r1.19 center.c --- src/center.c 25 Aug 2001 13:11:48 -0000 1.19 +++ src/center.c 6 Jan 2003 12:42:54 -0000 @@ -154,6 +154,7 @@ } next_connection: + ; } } Index: src/cfg-grammar.y =================================================================== RCS file: /var/cvs/syslog-ng/syslog-ng/src/cfg-grammar.y,v retrieving revision 1.56 diff -u -r1.56 cfg-grammar.y --- src/cfg-grammar.y 21 Aug 2002 14:03:50 -0000 1.56 +++ src/cfg-grammar.y 6 Jan 2003 12:42:54 -0000 @@ -75,7 +75,7 @@ /* option items */ %token KW_FLAGS KW_CATCHALL KW_FALLBACK KW_FINAL -%token KW_FSYNC KW_MARK_FREQ KW_SYNC_FREQ KW_STATS_FREQ KW_CHAIN_HOSTNAMES KW_KEEP_HOSTNAME KW_CHECK_HOSTNAME +%token KW_FSYNC KW_MARK_FREQ KW_SYNC_FREQ KW_STATS_FREQ KW_CHAIN_HOSTNAMES KW_KEEP_HOSTNAME KW_CHECK_HOSTNAME KW_BAD_HOSTNAME %token KW_LOG_FIFO_SIZE KW_LOG_MSG_SIZE %token KW_TIME_REOPEN KW_TIME_REAP KW_USE_TIME_RECVD %token KW_USE_DNS KW_USE_FQDN KW_GC_BUSY_THRESHOLD @@ -575,6 +575,7 @@ | KW_CHAIN_HOSTNAMES '(' yesno ')' { configuration->chain_hostnames = $3; } | KW_KEEP_HOSTNAME '(' yesno ')' { configuration->keep_hostname = $3; } | KW_CHECK_HOSTNAME '(' yesno ')' { configuration->check_hostname = $3; } + | KW_BAD_HOSTNAME '(' STRING ')' { cfg_set_bad_hostname($3); } | KW_USE_TIME_RECVD '(' yesno ')' { configuration->use_time_recvd = $3; } | KW_USE_FQDN '(' yesno ')' { configuration->use_fqdn = $3; }; | KW_USE_DNS '(' yesno ')' { configuration->use_dns = $3; }; Index: src/cfg-lex.l =================================================================== RCS file: /var/cvs/syslog-ng/syslog-ng/src/cfg-lex.l,v retrieving revision 1.25 diff -u -r1.25 cfg-lex.l --- src/cfg-lex.l 21 Aug 2002 14:03:50 -0000 1.25 +++ src/cfg-lex.l 6 Jan 2003 12:42:54 -0000 @@ -61,6 +61,7 @@ { "use_fqdn", KW_USE_FQDN }, { "use_dns", KW_USE_DNS }, { "check_hostname", KW_CHECK_HOSTNAME }, + { "bad_hostname", KW_BAD_HOSTNAME }, { "gc_threshold", KW_GC_BUSY_THRESHOLD }, { "gc_busy_threshold", KW_GC_BUSY_THRESHOLD }, { "gc_idle_threshold", KW_GC_IDLE_THRESHOLD }, Index: src/cfgfile.c =================================================================== RCS file: /var/cvs/syslog-ng/syslog-ng/src/cfgfile.c,v retrieving revision 1.39 diff -u -r1.39 cfgfile.c --- src/cfgfile.c 26 Apr 2002 09:43:54 -0000 1.39 +++ src/cfgfile.c 6 Jan 2003 12:42:54 -0000 @@ -125,6 +125,11 @@ configuration->dir_perm = perm; } +void cfg_set_bad_hostname(char *bad_hostname) +{ + configuration->bad_hostname = bad_hostname; +} + struct persistent_info * make_persistent_info(struct ol_string *name, struct ol_object *o, @@ -320,6 +325,7 @@ self->dns_cache_expire = 3600; self->dns_cache_expire_failed = 60; self->log_msg_size = 2048; + self->bad_hostname = NULL; if ((cfg = fopen(name, "r")) != NULL) { lex_init(cfg); res = yyparse(); Index: src/cfgfile.h =================================================================== RCS file: /var/cvs/syslog-ng/syslog-ng/src/cfgfile.h,v retrieving revision 1.25 diff -u -r1.25 cfgfile.h --- src/cfgfile.h 18 Oct 2002 12:31:08 -0000 1.25 +++ src/cfgfile.h 6 Jan 2003 12:42:54 -0000 @@ -57,6 +58,7 @@ (use_fqdn simple UINT32) (use_dns simple UINT32) (check_hostname simple UINT32) + (bad_hostname pointer UINT8) (create_dirs simple UINT32) (uid simple int) (gid simple int) @@ -129,10 +131,7 @@ void cfg_set_dir_owner(char *uid); void cfg_set_dir_group(char *gid); void cfg_set_dir_perm(int perm); - - - - +void cfg_set_bad_hostname(char *bad_hostname); struct syslog_config *make_syslog_config(const char *name, struct io_backend *backend); Index: src/filters.c =================================================================== RCS file: /var/cvs/syslog-ng/syslog-ng/src/filters.c,v retrieving revision 1.16 diff -u -r1.16 filters.c --- src/filters.c 4 Feb 2002 16:07:50 -0000 1.16 +++ src/filters.c 6 Jan 2003 12:42:54 -0000 @@ -34,6 +34,7 @@ #include <sys/socket.h> #include <netinet/in.h> #include <arpa/inet.h> +#include <string.h> #define CLASS_DEFINE #include "filters.h.x" Index: src/log.c =================================================================== RCS file: /var/cvs/syslog-ng/syslog-ng/src/log.c,v retrieving revision 1.28 diff -u -r1.28 log.c --- src/log.c 18 Oct 2002 12:31:08 -0000 1.28 +++ src/log.c 6 Jan 2003 12:42:54 -0000 @@ -41,7 +41,7 @@ static char aix_fwd_string[] = "Message forwarded from "; static char repeat_msg_string[] = "last message repeated"; -static void parse_log_msg(struct log_info *lm, UINT32 length, UINT8 *data, UINT8 *prefix) +static void parse_log_msg(struct log_info *lm, UINT32 length, UINT8 *data, UINT8 *prefix, regex_t *hostname_re) { unsigned char *src; unsigned int left, pri, oldleft; @@ -153,12 +153,15 @@ else { /* If we haven't already found the original hostname, look for it now. */ + char hostname_buf[256]; + int dst; oldsrc = src; oldleft = left; + dst = 0; while (left && *src != ' ' && *src != ':' - && *src != '[') { + && *src != '[' && dst < sizeof(hostname_buf) - 1) { if (lm->flags & LF_CHECK_HOSTNAME && !((*src >= 'A' && *src <= 'Z') || (*src >= 'a' && *src <= 'z') || @@ -168,11 +171,13 @@ *src == '@' || *src == '/')) { break; } + hostname_buf[dst++] = *src; src++; left--; } - - if (left && *src == ' ') { + hostname_buf[dst] = 0; + if (left && *src == ' ' && + (!hostname_re || regexec(hostname_re, hostname_buf, 0, NULL, 0))) { /* This was a hostname. It came from a syslog-ng, since syslogd doesn't send hostnames. It's even better then the one @@ -263,13 +268,13 @@ } } -struct log_info *make_log_info(UINT32 length, UINT8 *msg, UINT8 *prefix, UINT32 flags) +struct log_info *make_log_info(UINT32 length, UINT8 *msg, UINT8 *prefix, UINT32 flags, regex_t *hostname_re) { struct log_info *self; NEW_SPACE(self); self->flags = flags & LF_USER_FLAGS; - parse_log_msg(self, length, msg, prefix); + parse_log_msg(self, length, msg, prefix, hostname_re); self->use_cnt = 1; self->recvd = time(NULL); return self; Index: src/log.h =================================================================== RCS file: /var/cvs/syslog-ng/syslog-ng/src/log.h,v retrieving revision 1.18 diff -u -r1.18 log.h --- src/log.h 21 Aug 2002 14:03:50 -0000 1.18 +++ src/log.h 6 Jan 2003 12:42:54 -0000 @@ -30,6 +30,7 @@ #include "io.h" #include <sys/time.h> +#include <regex.h> struct syslog_config; struct persistent_config; @@ -89,7 +90,7 @@ struct log_info *log_info_use(struct log_info *msg); void log_info_free(struct log_info *msg); -struct log_info *make_log_info(UINT32 length, UINT8 *data, UINT8 *prefix, UINT32 flags); +struct log_info *make_log_info(UINT32 length, UINT8 *data, UINT8 *prefix, UINT32 flags, regex_t *badhostname_re); struct log_info *make_internal_message(UINT32 pri, UINT32 length, UINT8 *data); struct log_info *make_mark_message(void); Index: src/sources.c =================================================================== RCS file: /var/cvs/syslog-ng/syslog-ng/src/sources.c,v retrieving revision 1.36 diff -u -r1.36 sources.c --- src/sources.c 28 Oct 2002 08:33:30 -0000 1.36 +++ src/sources.c 6 Jan 2003 12:42:54 -0000 @@ -42,6 +42,7 @@ #include "sources.c.x" #include "nscache.h" + /* CLASS: (class (name log_reader) @@ -54,6 +55,7 @@ (max_log_line simple UINT32) (pad_size simple UINT32) (msg_flags simple UINT32) + (bad_hostname special-struct regex_t #f regfree) (next object log_handler))) */ @@ -65,7 +67,7 @@ { struct log_info *logmsg; - logmsg = make_log_info(length, data, self->prefix, self->msg_flags); + logmsg = make_log_info(length, data, self->prefix, self->msg_flags, &self->bad_hostname); if (addrlen) { logmsg->saddr = sockaddr2address_info(addrlen, addr); } @@ -155,6 +157,7 @@ UINT32 max_log_line, UINT32 pad_size, UINT32 msg_flags, + UINT8 *hostname_re, struct log_handler *next) { NEW(log_reader, self); @@ -167,7 +170,10 @@ self->pad_size = pad_size; self->msg_flags = msg_flags; self->buffer = ol_space_alloc(self->max_log_line); - + if (hostname_re == NULL) + regcomp(&self->bad_hostname, "^$", REG_NOSUB | REG_EXTENDED); + else + regcomp(&self->bad_hostname, hostname_re, REG_NOSUB | REG_EXTENDED); return &self->super; } Index: src/sources.h =================================================================== RCS file: /var/cvs/syslog-ng/syslog-ng/src/sources.h,v retrieving revision 1.16 diff -u -r1.16 sources.h --- src/sources.h 21 Aug 2002 14:03:50 -0000 1.16 +++ src/sources.h 6 Jan 2003 12:42:54 -0000 @@ -66,6 +66,7 @@ UINT32 max_log_line, UINT32 pad_size, UINT32 msg_flags, + UINT8 *bad_hostname, struct log_handler *next); struct log_source_group *make_source_group(const char *name, struct log_source_driver *drvs); -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
hi, my syslog-ng daemon dies unexpectedly. There is a lot of following messages in the log: salieri syslog-ng[28199]: STATS: dropped 0 No other signs of fail. Longina -- Longina Przybyszewska, system programmer Dept. of Math. & Comp. Sci. - IMADA University of Southern Denmark, Odense Campusvej 55,DK-5230 Odense M, Denmark tel: +45 6550 2359 - http://www.imada.sdu.dk email: longina@imada.sdu.dk --
On Mon, Jan 06, 2003 at 02:18:14PM +0100, Longina Przybyszewska wrote:
hi, my syslog-ng daemon dies unexpectedly. There is a lot of following messages in the log:
salieri syslog-ng[28199]: STATS: dropped 0
No other signs of fail.
it is not a sign of failure. syslog-ng reports that it dropped this amount of messages since the last STATS message. which version are you running? can you create a backtrace of the crash? -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
-- Longina Przybyszewska, system programmer Dept. of Math. & Comp. Sci. - IMADA University of Southern Denmark, Odense Campusvej 55,DK-5230 Odense M, Denmark tel: +45 6550 2359 - http://www.imada.sdu.dk email: longina@imada.sdu.dk -- On Mon, 6 Jan 2003, Balazs Scheidler wrote:
On Mon, Jan 06, 2003 at 02:18:14PM +0100, Longina Przybyszewska wrote:
hi, my syslog-ng daemon dies unexpectedly. There is a lot of following messages in the log:
salieri syslog-ng[28199]: STATS: dropped 0
No other signs of fail.
it is not a sign of failure. syslog-ng reports that it dropped this amount of messages since the last STATS message.
which version are you running? can you create a backtrace of the crash?
syslog-ng 1.5.24 what you mean by "create a backtrace of the crash" - I checked log file and there is no other messages from syslog-ng than 'STATS: dropped 0'. The most prabably it died after massiv restart of syslogd on all our Linxes - as these are the last messages in syslog === ... Jan 5 04:02:00 ulrica syslogd 1.3-3: restart. ... == Longina -- Longina Przybyszewska, system programmer Dept. of Math. & Comp. Sci. - IMADA University of Southern Denmark, Odense Campusvej 55,DK-5230 Odense M, Denmark tel: +45 6550 2359 - http://www.imada.sdu.dk email: longina@imada.sdu.dk --
On Mon, Jan 06, 2003 at 02:48:54PM +0100, Longina Przybyszewska wrote:
On Mon, 6 Jan 2003, Balazs Scheidler wrote:
On Mon, Jan 06, 2003 at 02:18:14PM +0100, Longina Przybyszewska wrote:
hi, my syslog-ng daemon dies unexpectedly. There is a lot of following messages in the log:
salieri syslog-ng[28199]: STATS: dropped 0
No other signs of fail.
it is not a sign of failure. syslog-ng reports that it dropped this amount of messages since the last STATS message.
which version are you running? can you create a backtrace of the crash?
syslog-ng 1.5.24
what you mean by "create a backtrace of the crash" - I checked log file and there is no other messages from syslog-ng than 'STATS: dropped 0'.
The most prabably it died after massiv restart of syslogd on all our Linxes - as these are the last messages in syslog === ... Jan 5 04:02:00 ulrica syslogd 1.3-3: restart. ... ==
attach gdb to the process, reproduce the crash and issue a 'bt' command. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
participants (5)
-
Balazs Scheidler
-
Borzenkov Andrey
-
Longina Przybyszewska
-
Nate Campi
-
Sander de Boer