April 2001 - syslog-ng - lists.balabit.hu

AIX and repeated message formats - fix?
by Jon Marks 04 Jun '01

04 Jun '01

Earlier I posted a message asking about whether syslog-ng had standard equipment for dealing with a couple of strange message types: 1) Messages forwarded by AIX syslogd which contain the string "Message forwarded from [hostname]: " ... and 2) Messages of the form: "last message repeated _n times" I haven't received any responses. I've read traffic in the mailing list archives (e.g. December 2000) regarding the topic, but I haven't found any solutions. If I've missed something vital, please help me and let me know about it.. I certainly apologize if I haven't been observant enough to skim the answers from the available resources. I've taken a stab at solving the problem myself, and I thought I'd share it with all of you. Those of you who are so much more familiar with the source than I, please comment about whether you find I might have broken other functionality or adversely affected performance. I'm glad for you suggestions, or if this can help somebody else. For my installation, being able to handle AIX messages is particularly important. My solution is just a simple and straightforward refactoring in log.c/parse_log_msg(). A patch against the 1.4.11 source distribution is attached to this message. I've tested it only on AIX, but I think it ought to work elsewhere since it only adds standard library stuff. (Basically, a couple of memcmp's). Some commented-out debugging output (to stderr) is there, too, if you'd want to enable it. I'll describe the intended effects of this patch: 1) Messages forwarded by AIX systems are recognized, and the source hostname is taken from the string appended by AIX syslogd. For example, MMM DD HH:MM:SS Message forwarded from <hostname>: ... will use <hostname> for the source address. "Message" isn't taken as the source host. 2) If the message is forwarded by AIX syslogd but originally of syslog-ng origin, it'll have the form MMM DD HH:MM:SS Message forwarded from <hostname>: source@hostname ... In this case, "source@hostname" will be used as the source hostname (so that syslog-ng's source info is preserved). 3) If the message doesn't contain a hostname because it's of the form MMM DD HH:MM:SS last message repeated *n* times then it won't try to extract one (so it won't improperly use 'last'). The hostname will just come later when in sources.c it's gotten from the message source socket or wherever. 4) If the message is a "last message repeated ..." which was *also* forwarded by an AIX syslogd, it'll get the hostname extracted from AIX's "Message forwarded from ..." message. I'm no expert in the syslog protocol/format, and I can see there is variablilty in implementation. So I'm certain there are plenty of cases that are left unaddressed. If you don't find this patch helpful in general, perhaps its useful for some common cases. One thing that's not addressed: When a "last message repeated ..." line is received, it's certainly helpful to know *which* was the last line-- the one that got repeated. If you're outputting to files by PROGRAM, then no program name is inherently available from this kind of message itself. And syslog-ng isn't keeping track of what PROGRAM logged last from a particular source host. So these "last message repeated ..." messages won't fall into the same logs as the lines for the PROGRAMs which produced the repeated messages. No simple solution is readily apparent to me... Also, please note that "keep_hostname(yes)" is important, otherwise the AIX sender of forwarded messages is used anyway-- despite that the real origin is available. If you use chain_hostnames(yes), then this info ought to be preserved anyway. Again, please comment and critique! Thanks very much. -- Jonathan Marks Systems Administrator, Production Systems Group Computing and Communication Services Office University of Illinois at Urbana-Champaign

3 5

[PATCH] chroot and drop root priviledges
by Tommi Virtanen 02 May '01

02 May '01

# This is for the Debian bug tracking system, see http://bugs.debian.org/94898 forwarded 94898 syslog-ng(a)lists.balabit.hu thanks Here's a patch to implement -C, -u and -g command line options, for chrooting, changing uid and gid, respectively. My syslog-ng is happily chrooted into /var/log/syslog-ng, and runs as syslogng:syslogng. Notes: - it still needs write access to the logs; I just chowned the tree to syslogng - it binds to /dev/log etc before chroot/userid switch, so it still works; however, if you change sources and try to reload config, it probably won't have access to open the new sources. - if you store the config outside the chroot, reload won't work. - I had to change the child->parent notify mechanism, as the old kill(getppid(), SIGTERM) trick didn't work after the uid switch. Now it has a pipe open to parent, parent waits on read(). I took the opportunity to make it signal success to the parent by writing a nil byte; parent changes exit value based on whether it saw the nil or not. diff -Naur syslog-ng-1.5.4.original/src/main.c syslog-ng-1.5.4/src/main.c --- syslog-ng-1.5.4.original/src/main.c Wed Nov 29 13:21:31 2000 +++ syslog-ng-1.5.4/src/main.c Sun Apr 22 21:36:14 2001 @@ -38,6 +38,8 @@ #include <errno.h> #include <string.h> #include <getopt.h> +#include <pwd.h> +#include <grp.h> #include "main.c.x" @@ -186,13 +188,29 @@ return 0; } +#define READ 0 +#define WRITE 1 void go_background() { pid_t pid; - + int pipefd[2]; + char buf; + + if (pipe(pipefd) < 0) { + werror("Cannot pipe(), (%z)\n", strerror(errno)); + exit(1); + } pid = fork(); if (pid == 0) { - int fd = open(pidfilename, O_CREAT | O_WRONLY | O_NOCTTY | O_TRUNC, 0600); + int fd; + if (pipefd[WRITE]!=1) { + if (dup2(pipefd[WRITE], 1) < 0) { + werror("Cannot set pipe as stdout, (%z)\n", strerror(errno)); + exit(1); + } + (void) close(pipefd[WRITE]); + } + fd = open(pidfilename, O_CREAT | O_WRONLY | O_NOCTTY | O_TRUNC, 0600); if (fd != -1) { struct ol_string *pid_s = c_format("%i\n", getpid()); write(fd, pid_s->data, pid_s->length); @@ -205,10 +223,13 @@ werror("Cannot fork(), (%z)\n", strerror(errno)); exit(1); } - signal(SIGTERM, sig_term); - while (sigtermrecvd == 0) - pause(); - exit(0); + + if (read(pipefd[READ], &buf, 1) == 1 + && buf=='\0') { + exit(0); + } else { + exit(1); + } } void usage(void) @@ -221,6 +242,9 @@ " -f <fname>, --cfgfile=<fname> Set config file name, default=" PATH_SYSLOG_NG_CONF "\n" " -V, --version Display version number (" PACKAGE " " VERSION ")\n" " -p <fname>, --pidfile=<fname> Set pid file name, default=" PATH_PIDFILE "\n" + " -C <dir>, --chroot=<dir> Chroot to directory\n" + " -u <user>, --user=<user> Switch to user\n" + " -g <group>, --group=<group> Switch to group\n" #ifdef YYDEBUG " -y, --yydebug Turn on yacc debug messages\n" #endif @@ -233,10 +257,45 @@ extern int yydebug; #endif +void nametouidgid(const char* user, uid_t *uid, gid_t *gid) +{ + struct passwd* pw; + pw = getpwnam(user); + + if (pw) { + *uid=pw->pw_uid; + if (gid) + *gid=pw->pw_gid; + } +} + +void nametogid(const char* group, gid_t *gid) +{ + struct group* gr; + gr = getgrnam(group); + + if (gr) { + *gid=gr->gr_gid; + } +} + +int dropprivileges(uid_t uid, gid_t gid) +{ + if ( setgid(gid) != 0 || setuid(uid) != 0 ) { + return -1; + } else { + return 0; + } +} + + int main(int argc, char **argv) { int do_fork = 1; int opt; + char *chrootdir = NULL; + uid_t uid = 0; + gid_t gid = 0; NEW(syslog_backend, backend); struct option syslog_ng_options[] = { @@ -246,6 +305,9 @@ { "verbose", no_argument, NULL, 'v' }, { "help", no_argument, NULL, 'h' }, { "version", no_argument, NULL, 'V' }, + { "chroot", required_argument, NULL, 'C' }, + { "user", required_argument, NULL, 'u' }, + { "group", required_argument, NULL, 'g' }, #ifdef YYDEBUG { "yydebug", no_argument, NULL, 'y' }, #endif @@ -257,7 +319,7 @@ gc_idle_threshold = 100; gc_busy_threshold = 3000; - while ((opt = getopt_long(argc, argv, "f:p:dvhyV", syslog_ng_options, NULL)) != -1) { + while ((opt = getopt_long(argc, argv, "f:p:dvhC:u:yV", syslog_ng_options, NULL)) != -1) { switch (opt) { case 'f': strncpy(cfgfilename, optarg, sizeof(cfgfilename)); @@ -265,6 +327,21 @@ case 'p': strncpy(pidfilename, optarg, sizeof(pidfilename)); break; + case 'C': + if (!optarg[0]) + usage(); + chrootdir = optarg; + break; + case 'u': + nametouidgid(optarg, &uid, gid ? NULL : &gid); + if (uid == 0 || gid == 0) + usage(); + break; + case 'g': + nametogid(optarg, &gid); + if (gid == 0) + usage(); + break; case 'd': debug_flag++; break; @@ -286,6 +363,8 @@ } } + if (uid == 0 && gid != 0) + werror("Cannot use -g without -u"); init_backend(&backend->super); backend->configuration = make_syslog_config(cfgfilename, &backend->super); @@ -301,17 +380,28 @@ } if (!CONFIG_INIT(backend->configuration, NULL)) { werror("Error initializing configuration, exiting.\n"); - if (do_fork) kill(getppid(), SIGTERM); return 2; } + if (chrootdir) { + if (chroot(chrootdir) < 0) { + werror("Error chrooting, exiting.\n"); + return 3; + } + } + if (uid) { + if (dropprivileges(uid, gid) < 0) { + werror("Error in setuid/setgid, exiting.\n"); + return 4; + } + } if (do_fork) { set_error_internal(); close(0); - close(1); close(2); setsid(); - kill(getppid(), SIGTERM); + write(1, "", 1); + close(1); } notice("syslog-ng version %z starting\n", VERSION); return main_loop(backend); -- tv@{{hq.yok.utu,havoc,gaeshido}.fi,{debian,wanderer}.org,stonesoft.com} unix, linux, debian, networks, security, | First snow, then silence. kernel, TCP/IP, C, perl, free software, | This thousand dollar screen dies mail, www, sw devel, unix admin, hacks. | so beautifully.

2 1

RE: [syslog-ng]Syslog messages are getting combined...
by Scott, Joshua 02 May '01

02 May '01

Here are a few lines from syslog that are combined... Apr 26 13:46:20 dnsserver OK" proto=ht<27>named[10268]: check_hints: no A records for nccnt01.jacobs.com class 1 in hints Apr 26 13:46:20 dnsserver 0 OK" proto<29>named[10268]: check_root: 1 root servers after query to root server < min Apr 26 13:57:40 dnsserver - Bryan, TX\\\"<p<29>named[9492]: starting (/etc/named.conf). named 8.2.3-REL After the time and the hostname there is some extra data that comes from my firewalls syslog messages. Then comes named and it's data. -----Original Message----- From: Balazs Scheidler [mailto:bazsi@balabit.hu] Sent: Friday, April 27, 2001 1:59 AM To: syslog-ng(a)lists.balabit.hu Subject: Re: [syslog-ng]Syslog messages are getting combined... On Thu, Apr 26, 2001 at 01:47:17PM -0400, Scott, Joshua wrote: > Has anyone ever had an issue where sometimes you get the information from > one syslog message combined with another syslog message? Every one in a > while I get some of my firewall syslog messages combined with messages from > my DNS servers. This causes my scripts to fail since there is invalid data > in the log message. Can anyone shed some light for me? Thank you very > much! Could you send me some examples. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng

2 1

Does syslog-ng have a message character limit?
by Brian Leveille 02 May '01

02 May '01

I have a device sending syslog messages that are getting truncated to 250 characters. Anyone else seen this with syslog-ng? Brian Leveille ------------------- Senior Internetworking Engineer DefendNet Solutions, Inc. http://www.defendnet.com

3 2

Future plans and ideas for syslog-ng...
by Nick DeClario 30 Apr '01

30 Apr '01

Hey guys, I was wondering what ideas and improvements are in the works for syslog-ng? Any thoughts or plans for including encryption in a future release? Regards, -Nick

1 0

syslog for proxy
by Cafferata Cristiano 27 Apr '01

27 Apr '01

hi, i'm looking for a program for nt platform able to receive syslogs from a proxy server .... it's the first time that i talk about syslog and it's the first time that i have to use it ..... please help me... thanks a lot and sorry for my english . bye * * * * * * * * * * * * * * * Cristiano Cafferata CC4191-RIPE ESPLIT Spa Servizio Tecnico Cristiano.Cafferata(a)esplit.it Via SS Giacomo e Filippo 19/6 16122 - Genova - Italy Phone +39 010881825 Fax +39 010 882743 * * * * * * * * * * * * * * *

1 0

Syslog messages are getting combined...
by Scott, Joshua 27 Apr '01

27 Apr '01

Has anyone ever had an issue where sometimes you get the information from one syslog message combined with another syslog message? Every one in a while I get some of my firewall syslog messages combined with messages from my DNS servers. This causes my scripts to fail since there is invalid data in the log message. Can anyone shed some light for me? Thank you very much! Joshua Scott Jacobs Engineering

3 2

Secure syslog-ng.
by Scott A. McIntyre 27 Apr '01

27 Apr '01

Hi, I was looking for a way to have a secured form of syslog-ng and the most obvious way that came up (without playing IPSEC games) was to use SSH with remote port forwarding. After mucking about with the configurations for an hour (apparently, destination and source descriptors take different syntax for specifying a network connection -- annoyingly undocumented) I got things working more or less as I'd like...however, I did notice that after a few messages sent through this way OpenSSH would die (the main daemon itself, not the one doing the port forwarding). I know this may seem unrelated and the version of OpenSSH (2.3.0p1) wasn't the most up to date (now upgraded) but this was repeatable behaviour. Has anyone else tried such a setup and noticed anything curious about how things interact? Any other solutions for a secured implementation of syslog-ng would also be appreciated. I think that this method will serve me well, after I get the configurations all tweaked and tuned to operate properly. Regards, Scott

2 1

RE: [syslog-ng]Syslog messages are getting combined...
by Andrew Fort 27 Apr '01

27 Apr '01

There was a bug in syslog-ng in one of the 1.4.x releases that caused this to happen. I dont have this prob any longer with 1.5.3, and from memory it was fixed in 1.4.10 (check the CHANGELOG perhaps?) -afort >Hi Joshua, > >I installed the lastest version of syslog-ng on our central >loghost last days. >I have the same problem. Our firewall logs get meshed with the >Catalyst Logs >etc. >Well... I havent figured out the error yet, but I think there >may be a problem >when u use same filters for different sources (What we do.. >cos the Cisco IOS >Msg'es are all the same :-) >We do a first a hostcheck [ host("hostname") ] and then a >contentcheck [ >match("<cisco-msg'es>") ]. >If this all doesnt match the stuff goes into the DEFAULT log file. > >Gotta investigate it tomorrow :) > >Bye > >"Scott, Joshua" wrote: > >> Has anyone ever had an issue where sometimes you get the >information from >> one syslog message combined with another syslog message? >Every one in a >> while I get some of my firewall syslog messages combined >with messages from >> my DNS servers. This causes my scripts to fail since there >is invalid data >> in the log message. Can anyone shed some light for me? >Thank you very >> much! >> >> Joshua Scott >> Jacobs Engineering

1 0

Re: [syslog-ng]Linefeed in incoming messages
by Michael.Garvin＠carolinabroadband.com 26 Apr '01

26 Apr '01

In doing some further debugging to the hack I posted to work around the logging problems with the Cisco VPN3000 I discovered some changes that need to be made to not break TCP-based logging. Here's the updated code, with the existing prefix line for positioning: closure->pos += n; start = closure->buffer + 1; eol = closure->buffer + closure->pos - 1; while (start++ < eol) if ((start[-1] == 0x0a) && (start[0] != 0x3c)) start[-1] = ' '; Sorry about that... -- Michael Garvin mailto:mgarvin@carolinabroadband.com

1 0