- syslog-ng - lists.balabit.hu

Re: tracelogs
by Balazs Scheidler 07 Apr '00

07 Apr '00

On Mon, Mar 27, 2000 at 04:33:18PM +0200, Gero Krueger wrote: > Hi Balazs ! > > HP has sent me a tracing tool. So I recorded a session > where syslog-ng listens on /dev/log and forwards that > to a remote host via tcp. > With the first logentry syslog-ng gets from /dev/log he > produces that huge unterminated output. > > I'm not really sure if this tracing-tool records the > information you need - just have a look at it and > let me know. > > > Thank you very much for your help. Sorry for the long time it took me to answer, but I'm really out of time. I've checked the trace you sent me, and the problem is that the info I need is missing: 3191: read(4 [/dev/log], 0x40019888, 1024) = 1024 The contents of the buffer is not displayed. It's strange that HPUX says that 1024 bytes could be successfully read, something should terminate that string, the question is what it is? -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 url: http://www.balabit.hu/pgpkey.txt

1 0

Expanding $PROGRAM and $FACILIITY macros with empty log messges
by Jason Edgecombe 06 Apr '00

06 Apr '00

hi all, i'm running syslog-ng-1.4.2 and I am having a problem with macro expansion an blank log message causes the problem. here is an excerpt from my logs: note, mathlog-pc is our central logging server, it is also running syslog-ng (same version) but is UDP forwarding all logs to the log server Apr 5 08:28:40 src@fret340f-pc/fret340f-pc Apr 5 08:28:40 src@mathlog-pc syslog-ng[1793]: Cannot open file /var/log/syslog-ng/2000/04/05/fret340f-pc/p/ for writing (Is a directory) Apr 5 08:28:40 src@fret340f-pc/fret340f-pc syslog-ng[4481]: Cannot open file /var/log/syslog-ng/2000/04/05/fret340f-pc/ for writing (Is a directory) Any help on this is appreciated. Thanks, Jason

1 0

RE: [syslog-ng] complex filtering
by Hamilton, Andrew Mr. 06 Apr '00

06 Apr '00

I've also had problems with the "and not" filtering in syslog-ng. I ended up having to do a kind of a DeMorgan method and change the filtering around a little. I got it to do what I wanted but not quite the way I expected. filter f_messages { level(info..warn) and not facility(local2,kern,auth); }; The above didn't work the way I expected it should have. I ended up doing filter f_messages { level(info..warn) and facility(mail,news,etc..); }; This seemed to work ok. I'm not sure if syslog-ng is evaluating the "and not" construct the way you'd expect. Of course it may be that I just wasn't using it correctly as well. Drew > -----Original Message----- > From: Balazs Scheidler [SMTP:bazsi@balabit.hu] > Sent: Wednesday, April 05, 2000 5:08 PM > To: syslog-ng(a)venus.terrasoft.hu > Subject: Re: [syslog-ng] complex filtering > > > > > Im currently building our loghost syslog-ng conf file and encounter a > problem. How can I filter message so that only not-alredy-routed line will > be routed/filtered again ? > > > > For example, I am trying to send ssh line to a dedicated log file, since > sshd send its message to the deamon facility and that ( generic) daemon > facility are send to a generic file, a given line appereas in both file > and that waste my disk space :-). > > > > I am trying to solve that by adding "and not filter(f_ssh)" in my dameon > filter but it does nothing ( not even syntax error message). > > > > Is there any error ? Is there another way ? > > > > Thanks for any help. > > The filter way should work, if it doesn't, it's a bug. There's a feature > you > may use here. You can define default log statements: > > log { source(src); filter(DEFAULT); destination(dst); }; > > This is a catchall statement, and should catch all messages which were not > accepted any of the previous statements. > > -- > Bazsi > PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C > 8EB1 > url: http://www.balabit.hu/pgpkey.txt > > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > http://lists.balabit.hu/mailman/listinfo/syslog-ng

2 1

program
by Przemek Bak 05 Apr '00

05 Apr '00

Hi, I have following lines in syslog-ng.conf filter aaa { match ("aaa"); }; destination aaa { program ("/usr/bin/mail -s WARN user"); }; log { source(src); filter (aaa); destination(aaa); }; When I run logger: # logger aaa it doesn't always work. Why ? I just want to have lines of messages sent to me by mail. przemol

1 0

syslog-ng and empty lines
by Balazs Scheidler 05 Apr '00

05 Apr '00

Many of you reported that syslog-ng emits empty loglines beginning with 1.4.2. This patch should fix the problem: diff -u -r1.23.2.2 sources.c --- sources.c 2000/03/24 10:57:28 1.23.2.2 +++ sources.c 2000/04/05 05:54:05 @@ -118,7 +118,8 @@ length = end - start; - do_handle_line(closure, length, start, salen ? (abstract_addr *) &sabuf : NULL , salen); + if (length) + do_handle_line(closure, length, start, salen ? (abstract_addr *) &sabuf : NULL , salen); start = end + 1; #if 0 -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 url: http://www.balabit.hu/pgpkey.txt

1 0

Collecting Logs on Loghost
by Wolfgang Braun 05 Apr '00

05 Apr '00

Greetings I'm writing my thesis on the subject of collecting and analysing logfiles. i'm in the information gathering and i'm interested in your opinions on the topic. the primary idea is that hosts log all their syslog (and maybe other files like http-logs) to a dedicated loghost where the log messages are inserted into a DB. additionally there should be a filter program that scans the message stream for certain patterns and triggers actions (information gathering like backfinger, ripe data etc). things to be evaluated now are o how much net load does syslog create? one pitfall could be that logging to the extreme could have an impact on network performance making it impossible to transfer all logging information. my view (not yet supported by numbers) is that the impact on net performance should be rather low, but i'm setting up a test environment to packet count. since i assume thet remote logging is common practice i'm interested in your experiences. o how much data does a host generate? this is of course dependant on which services are running on the machine and the usage of the services. question is how much data a DB (e.g. PostgreSQL) can handle (real time queries should be possible). this will be taken care of in the test environment, too. o common tools i'm aware of several analyzing tools (swatch, logsurfer ... from freshmeat). i like the logsurfer approach on creating contexts (to gather information from different sources into a small report on specified events). are there more tools like that that do a good job? some unsorted ideas: + doing long term analysis via the DB interface, to maybe trace long term patterns (slow portscans or door knob testing come to mind) + central administration of configurations and pushing of configurations to 'agents' running on each host + default filter configurations to handle standard services and their logs (bind, sendmail, ftpd), creation of regular reports as said, i'd be very interested on your thoughts on the topic, your experiences and maybe references to papers/tools. wolfi -- Internet ist das Ding mit den Kabeln. Usenet ist das Ding mit den Menschen. -- Hubert Partl

2 1

syslog-ng under HP-UX 10.20
by Gero Krueger 28 Mar '00

28 Mar '00

Hello snort-users ! I tried to compile syslog-ng-1.4.2 under HP-UX 10.20 (using gcc 2.8). During the linker process, I realized that there is no snprintf (and vsnprintf) function in the libc. Does anyone know a solution for this ? -- < Gero Krueger | DeTeCSM > < Tel. : +49-541-333-2488 | IVS Osnabrueck > < Fax : +49-541-333-2940 | Hannoversche Str. 6-8 > < mailto: krueger(a)obk.telekom.de | 49084 Osnabrueck >

4 9

Re: syslog-ng
by Balazs Scheidler 25 Mar '00

25 Mar '00

> > Great program from what I have read so far. Just installed it. I was > curious if there were any plans to add an option to syslog-ng.conf > under DESTINATIONS: > > * mail <email address> - Emails specified logs to user. you can easily achieve this using the program() destination. Write a simple shell script: #!/bin/sh while read line; do echo $line | mail root(a)domain.com done -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 url: http://www.balabit.hu/pgpkey.txt

1 0

syslog and syslog-ng together
by Gero Krueger 23 Mar '00

23 Mar '00

Hello snort-users ! I've got another interesting issue here: I set up syslog-ng on a solaris-2.6-box running together with the standard syslog. syslog-ng only forwards everything (/dev/log) to another loghost via tcp - this works fine. All messages are logged locally and on the remote loghost. Trying this under Linux 2.2.13, syslog and syslog-ng end up in some kind of race condition (messages are spread between them). Is there any way to get these two working at the same time like under solaris ? -- < Gero Krueger | DeTeCSM > < Tel. : +49-541-333-2488 | IVS Osnabrueck > < Fax : +49-541-333-2940 | Hannoversche Str. 6-8 > < mailto: krueger(a)obk.telekom.de | 49084 Osnabrueck >

1 0

syslog-ng 1.4.1
by Balazs Scheidler 20 Mar '00

20 Mar '00

Hi, I've released syslog-ng 1.4.1. This time I announce it to this list only, and if no bugs show up in two-three days, the announcement goes up to freshmeat. I've fixed a couple of bugs in this release: * fixed a filter evaluation bug (not in 'or' and 'and') * configuration file reading with syntax errors * pipe() cleanup This driver readds the requirement for GNU flex. I need a way to reinitialize lex buffers, and I know how to do that only with GNU flex (yyreinit(), patches to support other lex versions are welcome) -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 url: http://www.balabit.hu/pgpkey.txt

3 2

syslog-ng on AIX 4.x
by Ohrberg, Mick 16 Mar '00

16 Mar '00

Dear syslog-ng users, Is there anyone out there who has successfully installed and used syslog-ng on an AIX machine? I use it in Solaris 2.6, and it works great! As a Solaris-user, I really don't know where to look for the syslog-equivalent in AIX. Syslog-ng compiles great under AIX, but using the logger doesn't give me the expected result... Anyone? Thanks in advance! -- Mick Öhrberg EMTS System Administrator, UMB (816)860-1064

1 0

sendmail with syslog-ng
by Simon Yang 15 Mar '00

15 Mar '00

Hi folks, I installed syslog-ng 1.3.14 under Solaris 2.6 and used the following config file source local { sun-streams("/dev/log" door("/etc/.syslog_door")); internal(); }; destination syslog { file("/var/tmp/syslog.trc"); program("/usr/lib/sendmail john(a)company.com"); }; log { source(local); destination(syslog); }; I expect that emails should only be sent to john(a)company.com when an error message occurs. With the current implementation of the syslog-ng, only one messages is sent out containing the whole syslog.trc that is defined in the config file when the syslog-ng is started. Obviously, something is wrong. Is this a feature or bug with syslog-ng or the config file was not configured correctly. Any help will be greatly appreciated.

1 0

Odd error
by Hamilton, Andrew Mr. 15 Mar '00

15 Mar '00

Hi, I'm relatively new to syslog-ng and have been working with it constantly over the last couple of weeks trying to get it to work with my systems. For the most part I have had no problems, or very few but now I am getting an odd error. I am running it on a Sun Solaris 5.6 and have run it on other systems identical to this system but I am getting this error: Error creating AF_INET socket (Error 0) Error initializing configuration, exiting. I've checked my config file and it doesn't appear to be messed up. Any ideas? Thanks Drew

2 1

RE: syslog-ng digest, Vol 1 (Cu)167 - 4 msgs
by Nick Hindley 10 Mar '00

10 Mar '00

This message has been automatically generated. I am out of the office from 6 March 2000 for approximately two weeks. To contact me before my return, please resend your message to nick.hindley(a)ukgateway.net

1 0

Small log entries are discarded in 1.4.0
by Jeffrey W. Baker 09 Mar '00

09 Mar '00

Hi there, I have found another serious bug in the 1.4.0 syslog-ng code. Actually the bug is in libol. The log is configured to read log entries from a named pipe, write them to a local file, and forward them to a remote machine which writes them to a file. The input testing program is writing log entries which are four bytes: three bytes of integers and a carraige return. The problem is that syslog-ng is throwing away most of these entries. It reads 1024 bytes from the input pipe. 1024 bytes is 256 log entries. It then takes 100 of these entries and puts them in the log queue. These 100 are logged properly, and then another 1024 bytes are read from the pipe. The remaining 156 log entries from the original read are simply overwritten. I believe the solution would be to write log entries into the queue until the input buffer is empty. Any other way will result in the loss of log entries. Perhaps I will have a patch for this later. Also, I found another small bug in src/sources.c. On line 128, memcpy is used to copy memory between overlapping regions. At least on Linux, memcpy should not be used to copy between overlapping regions. memmove() should be used instead. Cheers, Jeffrey

2 1

no kernel messages
by Martin Krejcirik 09 Mar '00

09 Mar '00

I'm new to syslog-ng, maybe this question was asked before, but I do not get logged any messages from kernel (packet logger for example). Tried both 1.2.4 and 1.4.0. For some reason logger -p kern.info works. Klogd is running. If I stop it, kernel logs to console. config file: # Global Options options { sync(0); }; # Source dirvers source src_int { internal(); }; source src { unix-stream("/dev/log"); }; # log everything to a console tty destination d_tty { file("/dev/tty12"); }; log { source("src"); source("src_int"); destination("d_tty"); }; _____________________________ Martin Krejcirik <mk(a)krej.cz>

2 1

RE: syslog-ng digest, Vol 1 (Cu)166 - 5 msgs
by Nick Hindley 09 Mar '00

09 Mar '00

This message has been automatically generated. I am out of the office from 6 March 2000 for approximately two weeks. To contact me before my return, please resend your message to nick.hindley(a)ukgateway.net

1 0

RE: [syslog-ng] syslog-ng & forwarding
by Hamilton, Andrew Mr. 08 Mar '00

08 Mar '00

> > I am using syslog-ng 1.4.0rc3 at this time (I do have 1.4.0 but haven't > > implemented it yet), on Solaris 7 from a Sun SPARC machine. I am trying > to > > forward my logs from one machine through another and log them on a third > > machine. There is a reason I do this and it is necessary that I do it > this > > way. However, my question is this: > > I assume your logs are transferred to a loghost behind a chain of > firewalls. [Hamilton, Andrew] Yes they are. > > > How do I keep it from putting the middle machine's hostname on it? > > > > I assumed that chain_hostnames(no) option would do it but it doesn't > work > > the way I would have expected. When I used chain_hostnames(no) I got > > something like: > > > > "Mar 8 08:35:41 mhost host hamilton: Testing..." > > Why is there 3 names? there should be only "mhost hamilton: testing..." [Hamilton, Andrew] The final destination is using regular syslogd which it won't when I am done testing. > > > > > When I used chain_hostnames(yes) I got something like: > > > > "Mar 8 08:38:17 mhost host/host hamilton: Testing..." > > > > Any ideas? Or is this just not possible? > > Are all your hosts using syslog-ng, or some of them use the native > syslogd? [Hamilton, Andrew] Yes the last one is using syslogd until I get done testing. They will all use syslog-ng when I am done. > > The proper behaviour is: > > 1) chain_hostnames(off) > the hostname the message was received from is put in the hostname part > of > the message (if it contained one it is replaced) so on your inner > loghost > you'll find the name of your last hop. > 2) chain_hostnames(on) > hostnames are chained, which means that on each hop the hostname the > message was received from is appended to the chain. The first hostname > is > the originator. So a hostname like this: > bzorp/bzorp/balabit > > [Hamilton, Andrew] That is the behavior I expected but didn't quite get. It just didn't dawn on me that the syslogd would add another hostname. > means, that the message was generated on bzorp, was received from bzorp > to > balabit, and it was received from balabit at its final destination. > > A keep_hostname() option would be a useful addition, but it's not > implemented yet. > [Hamilton, Andrew] That would be useful and thanks for the prompt response. Drew

1 0

syslog-ng & forwarding
by Hamilton, Andrew Mr. 08 Mar '00

08 Mar '00

I am using syslog-ng 1.4.0rc3 at this time (I do have 1.4.0 but haven't implemented it yet), on Solaris 7 from a Sun SPARC machine. I am trying to forward my logs from one machine through another and log them on a third machine. There is a reason I do this and it is necessary that I do it this way. However, my question is this: How do I keep it from putting the middle machine's hostname on it? I assumed that chain_hostnames(no) option would do it but it doesn't work the way I would have expected. When I used chain_hostnames(no) I got something like: "Mar 8 08:35:41 mhost host hamilton: Testing..." When I used chain_hostnames(yes) I got something like: "Mar 8 08:38:17 mhost host/host hamilton: Testing..." Any ideas? Or is this just not possible? Andrew Hamilton mailto:hamiltona@hq.5sigcmd.army.mil

2 1

RE: syslog-ng digest, Vol 1 (Cu)165 - 9 msgs
by Nick Hindley 08 Mar '00

08 Mar '00

This message has been automatically generated. I am out of the office from 6 March 2000 for approximately two weeks. To contact me before my return, please resend your message to nick.hindley(a)ukgateway.net

1 0

Can I use destinate to all user?
by Tomohiro Yamauchi 07 Mar '00

07 Mar '00

Hi, I am running syslog-ng 1.4.0rc3 on Solaris7 (SPARC). Solaris's syslog can use '*' action. e.g. kern.emerg * This means "Kernel emergency messages are forwarded to all logged-in users." I want to use the same action. Please tell me how to do it. -- Tomohiro Yamauchi handy(a)nid.co.jp

3 4

Re: [syslog-ng] syslog-ng dropping log messages forwarded w/TCP
by Jeffrey W. Baker 07 Mar '00

07 Mar '00

You wrote: >Probably the speed of your network is less than required. How fast are >you >sending those messages? syslog-ng can do with quite high loads (2G logs >a >day), but it still cannot widen your network bandwidth. Note that log >connections in syslog-ng (e.g. a log statement) is not flowcontrolled. >This >means that messages are continously read even if they have not yet been >flushed to the destination. The reason behind this is to prevent >syslog-ng >to become the bottleneck. > I don't think we have been quite clear on the nature of the problem. We are sending messages at a very high rate over a named pipe to the local syslog-ng, and these messages are being forwarded via tcp to a remote machine, and also are logged in a local file. Our network is switched 100 Mbps, and the machines at each end are 450 MHz and 600 MHz Intel Pentium III with 3Com 3c905B NICs. It seems unlikely that our network is the problem. The exact sequence of events is that our program is started which makes 100000 log entries in a tight loop. After the program exits, the local log file has all 100000 entries, but the remote log file has only about 25000 entries. No amount of waiting increases the number of entries in the remote log. The entries in the remote log are not sequentially numbered. For example, they skip from 95432 to 96109. The intervening log entries are simply dropped. This is why we believe the problem is in syslog-ng. At any rate it is possible that James has found the problem. Apparently the return value of write is not being corretly interpreted when zero bytes were written. Perhaps James will have a patch soonish. Regards and thanks for your software, Jeffrey Baker >I may add an option to log statements to stop reading from the source if >the >destination fifo is full. This is however not trivial, since several >sources >can log to the same destination. I plan something like this: > >log { source(src); options(flow_control(yes)); destination(dst); };

3 2

RE: syslog-ng digest, Vol 1 (Cu)164 - 4 msgs
by Nick Hindley 07 Mar '00

07 Mar '00

This message has been automatically generated. I am out of the office from 6 March 2000 for approximately two weeks. To contact me before my return, please resend your message to nick.hindley(a)ukgateway.net

1 0

missing syslog-ng.conf
by Sebastian Sanchez 06 Mar '00

06 Mar '00

Hi list: I' ve just compiled syslog-ng in a Linux box running RedHat 6.0. Everything worked fine, except for one thing: there is no /etc/syslog-ng directory and no syslog-ng.conf file. Where can I find a sample configuration file ? Thank you very much. Sebastian Sanchez Security Operator se_sanchez(a)crm.com.ar

4 3

pipe cleanup
by Balazs Scheidler 04 Mar '00

04 Mar '00

Hi, I've began working on 1.5.0 and cleaned up pipe support. Thanks for the idea about O_RDWR pipes for "Klaus Groenhoej" <kgr(a)kmd.dk>. No need to check for spuriously closed pipes now. I've backported the patch to 1.4.0 and is available attached to this mail. This adds no additional functionality, and 1.4.0 should be o.k. as well, so I don't release 1.4.1 immediately, rather I wait for feedback on 1.4.0. Index: affile.c =================================================================== RCS file: /home/bazsi/cvs/syslog-ng/src/affile.c,v retrieving revision 1.30 diff -u -r1.30 affile.c --- affile.c 2000/03/03 14:43:20 1.30 +++ affile.c 2000/03/04 19:56:34 @@ -179,7 +179,6 @@ (vars (owner object affile_dest) (dest object io_fd) - (dest_buf object abstract_buffer) (reap simple "struct callout *") (expanded_fname string) (prev object affile_dest_writer) @@ -190,42 +189,30 @@ do_init_dest_writer(struct log_handler *c, struct syslog_config *cfg, struct persistent_config *persistent) { CAST(affile_dest_writer, self, c); - int fd; + int fd, flags; - if (self->dest) - self->dest->buffer = NULL; - if (!self->dest_buf) - self->dest_buf = make_pkt_buffer_ext(self->owner->sync_freq, - MAX(self->owner->super.log_fifo_size + 1, - self->owner->sync_freq)); - self->dest = NULL; - if (do_open_file(self->expanded_fname, O_WRONLY | O_CREAT | O_APPEND | O_NOCTTY | O_NONBLOCK, self->owner->perm, self->owner->flags & AFFILE_CREATE_DIRS, self->owner->dir_perm, &fd)) { - struct resource_node *res; + if (self->owner->flags & AFFILE_PIPE) + flags = O_RDWR | O_NOCTTY | O_NONBLOCK; + else + flags = O_WRONLY | O_CREAT | O_APPEND |O_NOCTTY | O_NONBLOCK; + if (do_open_file(self->expanded_fname, flags, self->owner->perm, self->owner->flags & AFFILE_CREATE_DIRS, self->owner->dir_perm, &fd)) { chown(self->expanded_fname->data, self->owner->uid, self->owner->gid); - self->dest = io_write(make_io_fd(cfg->backend, fd), - self->dest_buf, - NULL); - res = REMEMBER_RESOURCE(cfg->resources, &self->dest->super.super); - self->dest->super.close_callback = - make_driver_reinit_on_close(c, res, cfg->time_reopen, cfg); + self->dest = + io_write(make_io_fd(cfg->backend, fd), + make_pkt_buffer_ext(self->owner->sync_freq, + MAX(self->owner->super.log_fifo_size + 1, + self->owner->sync_freq)), + NULL); + REMEMBER_RESOURCE(cfg->resources, &self->dest->super.super); if ((self->owner->flags & AFFILE_NO_EXPAND) == 0) self->reap = io_callout(cfg->backend, TIME_REAP_DEST, make_affile_dest_reaper(self)); /* self->attempts = 0; */ } else { - if (errno == ENXIO) { - /* opening the write side of the fifo failed */ - /* self->attempts++; */ - io_callout(cfg->backend, cfg->time_reopen, make_driver_reinit(c, cfg)); - if ((self->owner->flags & AFFILE_NO_EXPAND) == 0) - self->reap = io_callout(cfg->backend, TIME_REAP_DEST, make_affile_dest_reaper(self)); - } - else { - werror("Cannot open file %S for writing (%z)\n", self->expanded_fname, strerror(errno)); - return ST_FAIL | ST_QUIT; - } + werror("Cannot open file %S for writing (%z)\n", self->expanded_fname, strerror(errno)); + return ST_FAIL | ST_QUIT; } return ST_OK | ST_GOON; @@ -235,8 +222,8 @@ { CAST(affile_dest_writer, self, c); - if (self->dest_buf) { - A_WRITE_STRING(&self->dest_buf->super, c_format("%S %S %S\n", msg->date, msg->host, msg->msg)); + if (self->dest) { + A_WRITE_STRING(&self->dest->buffer->super, c_format("%S %S %S\n", msg->date, msg->host, msg->msg)); if (self->reap) io_callout_set_timeout(self->reap, TIME_REAP_DEST); } -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 url: http://www.balabit.hu/pgpkey.txt

1 0