November 2004 - syslog-ng - lists.balabit.hu

FAQ-seeding: chroot jail procedure for Syslog-ng
by Michael D. (Mick) Bauer 24 Jan '05

24 Jan '05

Hi, all. In researching/writing the Syslog-ng coverage for the new edition of my book, I've encountered a total lack of published info, anywhere, on how to create a Syslog-ng chroot jail. So in the interests of seeding the FAQ (or at least Google), here's a procedure that works for me. Naturally, anyone should feel free to correct anything I've gotten wrong! Any and all feedback is appreciated. ************************************************ How To Create A Chroot Jail For Syslog-ng 1.6x ************************************************ 1. su to root if you're not root already 2. Create an unprivileged group-account for syslog-ng, e.g., by adding the following line to /etc/group: syslogng:x:77: 3. Create an unprivileged system account for syslog-ng, e.g., via the following command: useradd -d /var/syslog-ng-jail -g syslogng -r syslogng (Note that in Linux, the "-r" flag tells useradd that this will be a system account, causing useradd to automatically set the account's shell to /bin/false and to choose an appropriately low value for its UID.) 4. Create the jail: mkdir -p /var/syslog-ng-jail/var/log (Our actual changed root will be /var/syslog-ng-jail, but we may as well create the var/log subdirectory at the same time) 5. At this point the whole jail should be owned by root:root, which is cool so long as the changed-root-directory itself (/var/syslog-ng-jail) is "other-executable," e.g., rwxr-xr-x. But syslog-ng *will* need to create/write files in the jail's var/log subdirectory, so we need to tweak the latter's group-ownership and -permissions, like so: chgrp syslogng /var/syslog-ng-jail/var/log chmod g+wx /var/syslog-ng-jail/var/log 6. That's it! We may now start syslog-ng like this: syslog-ng -C /var/syslog-ng-jail -u syslogng -g syslogng The syslog-ng process will still read its config from /etc/syslog-ng/syslog-ng.conf (not /var/syslog-ng-jail/etc/...), but immediately after that it will chroot itself to the specified jail. Note, however, that the paths you specify in syslog-ng.conf "file()" statements should all be relative to the changed root. E.g., use file("/var/log/messages"), *not* file("/var/syslog-ng-jail/var/log/messages"). Any path you specify in syslog-ng.conf will end up with "/var/syslog-ng-jail" prepended to it. Naturally, there's nothing to stop you from dropping the "var/log/" subdirectory altogether, and simply specify, e.g., file("/messages") as a destination (resulting in things being written to /var/syslog-ng-jail/messages, a less unwieldy path). ******* So far I haven't noticed that anything else needs to be added to the chroot jail (e.g., stuff from /dev or /etc), but if anyone knows differently please speak up! Regards, Mick P.S. Baszi, you really ought to add "-u" & "-g" to the syslog-ng manpage. After all, running anything in a chroot jail as root is futile, no? :-) P.P.S. BTW, 1.6x rocks! /-------------------------------------------------\ | Michael D. (Mick) Bauer | | Security Editor, Linux Journal | | Dir. of Value-Subtracted Svcs., Wiremonkeys.org | \-------------------------------------------------/

6 12

re-inserting syslog messages for testing
by Ed Ravin 18 Jan '05

18 Jan '05

I've written a perl script that takes as input a file with syslog messages in it, and re-sends them into syslog on the current host. I use this for test environments, especially when I'm testing things with logsurfer and have a known logfile that should generate certain results when parsed by logsurfer. Are there any other tools that do this? If not, I'll release mine after cleaning it up a bit, but there's no point in adding more cruft to the world if Someone Else has done a better job. -- Ed

2 2

What facility do the 'STATS' messages come in on?
by Ben Nelson 18 Jan '05

18 Jan '05

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I am able to see my internal() STATS messages only by sending all messages (unfiltered) to a single file. I'm not sure which of my filters may be dropping them. So, which syslog facility do those messages happen under? Thanks, - --Ben -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFBY1Ld3cL8qXKvzcwRAg3eAJ0a/Vxz810kkCqXzizKTag9Yb05zACg1r/X rZpVs4HsMSAlO3Xygs80JYI= =PKRl -----END PGP SIGNATURE-----

4 3

Problems with Netscreen log entries
by Paul Mindeman 06 Jan '05

06 Jan '05

Running sylog-ng 1.6.4 on Solaris 9 Log entries from my UNIX devices log fine. Log entries from my Netscreen devices seem to be missing the end of line terminator, as the entries run together in the log file. The default syslog daemon was able to handle these entries fine. Any ideas on how to fix this? The options in the syslog-ng.conf file are: options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; Thanks -- Paul Mindeman BTInet Systems Administrator 701-355-5587 mindeman(a)btinet.net

3 2

FYI: Performance
by Jay Guerette 03 Dec '04

03 Dec '04

( I hope I don't offend anyone with an attachment, it's only 8k. ) My largest single-server syslog-ng implementation currently handles over 13M lines per day, totalling about 1.8Gb per day. I've only recently been able to gather this data by creating a process to count incoming lines, sum their lengths, and report via syslog at 1 minute intervals. See attached graph ( if the attachment survived ). I added this configuration before all other entries to make sure it sees everything: <syslog-ng.conf> destination syslog-perf { program(syslog-perf); }; log { source(syslog); destination(syslog-perf); }; </syslog-ng.conf> I originally tried this in Perl, then Bash, but neither could keep up with the incoming messages. This works like a champ. It compiles on Linux. The output format is specific to my syslog-to-rrd implementation, but you get the idea. It is suitable for an installation that is assured of at least 1 message for each reporting interval! <syslog-perf.c> #include <stdio.h> #include <time.h> #include <syslog.h> #define BUFFER_SIZE 8192 #define REPORT_INTERVAL 60 void main(void) { char buf[BUFFER_SIZE]; long count, bytes; time_t lastupdate; lastupdate = time(NULL); while (fgets(buf, BUFFER_SIZE, stdin)) { count++; bytes += (strlen(buf) - 1); if (time(NULL) > (lastupdate + REPORT_INTERVAL)) { openlog("127.0.0.1", LOG_NDELAY, LOG_LOCAL3); syslog(LOG_INFO, "Syslog-ng\\Lines=%d Syslog-ng\\Bytes=%d", count, bytes); closelog(); lastupdate += REPORT_INTERVAL; count = 0; bytes = 0; } } } </syslog-perf.c>

7 12

RES: [syslog-ng]php-syslog-ng
by Eduardo Oliveira Scricco (Service) 30 Nov '04

30 Nov '04

> why dont you use the latest version ? When I tried to install syslog-ng-1.6.2-1 (the lastest compiled version I've found for Suse), I had this error: error: failed dependencies: libc.so.6(GLIBC_2.3) is needed by syslog-ng-1.6.2-1 So, I tri to upgrade glibc, but... error: failed dependencies: glibc = 2.2.5 is needed by lsb-runtime-1.2-37 glibc = 2.2.5 is needed by glibc-devel-2.2.5-213 glibc = 2.2.5 is needed by glibc-locale-2.2.5-213 libdb.so.2 is needed by db-utils-4.0.14-194 libdb.so.2 is needed by gnome-libs-1.4.1.7-134 libdb.so.2 is needed by gdk-pixbuf-0.18.0-108 libdb.so.2 is needed by xchat-1.8.10-59 libdb.so.2 is needed by eel-1.0.2-509 libdb.so.2 is needed by gnome-print-0.36-210 libdb.so.2 is needed by libglade-0.16-835 libdb.so.2 is needed by medusa-0.5.1-644 libdb.so.2 is needed by rpm-3.0.6-418 libdb.so.2 is needed by bonobo-1.0.20-221 libdb.so.2 is needed by bonobo-conf-0.14-505 libdb.so.2 is needed by susewm-3.6.0-159 libdb.so.2 is needed by kdebase3-SLES-8-73 libdb.so.2(GLIBC_2.0) is needed by db-utils-4.0.14-194 libdb.so.2(GLIBC_2.0) is needed by gnome-libs-1.4.1.7-134 libdb.so.2(GLIBC_2.0) is needed by rpm-3.0.6-418 libdb.so.2(GLIBC_2.0) is needed by susewm-3.6.0-159 As I don't want to upgrade all these items, I prefeer not to install it. > > * app-admin/syslog-ng > Latest version available: 1.6.5-r1 > Latest version installed: 1.6.5-r1 > Size of downloaded files: 387 kB > Homepage: http://www.balabit.com/products/syslog_ng/ > Description: syslog replacement with advanced filtering features > License: GPL-2 > > i got php-syslog-ng working perfectly, i even hacked it to my > needs.. its great. > > Petre Daniel > System & Network Administrator > Romania Data Systems Pitesti > http://www.rdspt.ro > Phone: +40-348-400401

2 1

php-syslog-ng
by Eduardo Oliveira Scricco (Service) 30 Nov '04

30 Nov '04

Hi all, Using syslog-ng 1.4.14 at Suse Linux 8.1 I'm trying to run php-syslog-ng (a syslog webviewer), but it does not work. For it works, is needed to log to a database, as MySQL. PHP-Syslog-ng website have on step-by-step script, that I used. When I try to run syslog-ng, it complains about this line (in syslog-ng.conf is at one line): destination d_mysql { pipe("/tmp/mysql.pipe" template("INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; log { source(net); destination(d_mysql); }; I believe this "template" isn't supported in this syslog version. Is that true? The problem is I couldn't upgrade it to version 1.5 or higher, because of libs dependencies... Have one of you logged to mySQL using syslog-ng 1.4.14? Thanks a lot, Eddie

2 1

syslog-ng chrooted vs timezone
by Horvath Gabor Kalman 30 Nov '04

30 Nov '04

Hi! I have a problem with syslog-ng running in chroot: Nov 17 15:45:02 burp /USR/SBIN/CRON[26238]: ... Nov 17 14:47:42 burp syslog-ng[16074]: syslog-ng version 1.6.4 going down Nov 17 14:47:47 burp syslog-ng[26331]: syslog-ng version 1.6.4 starting Nov 17 14:47:47 burp syslog-ng[26331]: io.c: do_read: read() failed (errno 1), Operation not permitted Nov 17 14:48:27 burp syslog-ng[26331]: syslog-ng version 1.6.4 going down It seems that all messages that are generated internally within syslog-ng are logged with false timstamps. The system is debian testing. I am testing syslog-ng in chroot before deploying it on our production syslog server. :) Good thing I did this test. The second start in the log snippet was done by hand. Not from the init script. I am also interested to know what that io.c message is. It does not seem to affect syslog-ng. I had another problem: when logrotate rotated the logs the sent a reload to the chrooted syslog-ng process, it kept the syslog open. So after that was deleted it kept happily logging there not into the newly created file. I works just fine if I send restart instead of reload. Thanks in advance. g.

2 5

compiling error syslog-ng 1.6.5
by Lopez Fernandez, Jose 30 Nov '04

30 Nov '04

Hi, I`m compiling syslog-NG 1.6.5 but i get this error: configure:2159: gcc --version </dev/null >&5 gcc (GCC) 3.4.2 Copyright (C) 2004 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. configure:2162: $? = 0 configure:2164: gcc -v </dev/null >&5 Reading specs from /usr/local/lib/gcc/sparc-sun-solaris2.8/3.4.2/specs Configured with: ../configure --with-as=/usr/ccs/bin/as --with-ld=/usr/ccs/bin/ld --disable-nls Thread model: posix gcc version 3.4.2 configure:2167: $? = 0 configure:2169: gcc -V </dev/null >&5 gcc: `-V' option must have argument configure:2172: $? = 1 configure:2195: checking for C compiler default output file name configure:2198: gcc conftest.c >&5 ld: fatal: file values-Xa.o: open failed: No such file or directory ld: fatal: File processing errors. No output written to a.out collect2: ld returned 1 exit status configure:2201: $? = 1 ideas? help? :-) Greetings, Jose Lopez Dieses Dokument ist vertraulich und ausschliesslich fuer den Adressaten bestimmt. Falls Sie diese E-Mail versehentlich bekommen haben, informieren Sie uns bitte unverzueglich und loeschen Sie diese Nachricht von Ihrem Computer. Jegliche Art von Reproduktion, Verbreitung, Vervielfaeltigung, Modifikation, Verteilung und/oder Publikation dieser E-Mail Nachricht ist untersagt. Die in dieser E-Mail enthaltenen Angaben und Erklaerungen sind unverbindlich. Haftungsansprueche des Empfaengers jeglicher Art werden ausgeschlossen. Die GZS schliesst ausser fuer den Fall von Vorsatz oder grober Fahrlaessigkeit die Haftung fuer jeglichen Verlust oder Schaeden durch virenbefallene Software oder E-Mails aus. ----------------------------------------------------------------------------------------------------------- This message contains confidential information and is intended only for the named individual. If you are not the named addressee, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this message in error and delete this e-message from your system. No reliance may be placed on this message without written confirmation of its contents from an authorized representative. GZS accepts no liability for loss or damage caused by software viruses except in case of gross negligence or willful behaviour.

1 0

Re: [Fwd: [syslog-ng]feature request to Stats ouput]
by Jay Guerette 29 Nov '04

29 Nov '04

No comment on the potential issue.... just a 'me too'. I'd love to see a 'processed' count. My syslog server is processing about 10M lines/day, by my best estimate, but would like to know more precisely. A byte count would be nice, too; currently about 1.2GB day. I'd like to generate RRD graphs, if I could get that data. I tried writing a couple programs to do line & byte counts, and set it up as a 'program' destination, but it would sometimes drop stuff when it looked away from it's input long enough to write the stats to a log. Probably when the there was a burst well above the ~ 115 messages/sec average. User-specified destinations & macro/template formating for stats output would be fantastic! > Stats: dropped is great, would this be better?: > Stats: x processed y buffered/queued z dropped > > * I'm seeing syslog-ng process sizes in 20-30MB and am wondering if my > queue sizes should be upped.

2 1