January 2007 - syslog-ng - lists.balabit.hu

Feeding data into MYSQL
by Chris Killian 04 Jan '07

04 Jan '07

All I have a script that takes the data collected by syslog-ng and inputs into mysql database, which is then viewable using something like phpsyslog. However although this works for all hosts there is one that it wont do. I can see the data coming in if and get added to the syslog log file, however will not go into the data. COuld this be down to the format that the data is received in. Anyone got any ideas? Regards CK

2 1

RE: [syslog-ng] I/O error occurred while reading; fd='4',error='Operation not permitted (1)'
by Justin Randall 04 Jan '07

04 Jan '07

This cleared a few things up. Should have noticed this earlier when looking at the return code for the "read" system call: "read(4, 0x55ad20, 8192) = -1 EPERM (Operation not permitted)" This is indicative of access control restrictions. If you consider the file trying to be read ("/proc/kmsg") and it's permissions (only readable as root) it makes sense why you are getting permission errors when trying to as user "syslogng". You are also experiencing other permission related errors on certain files: /var/log/syslog /var/log/messages /dev/xconsole File attribute changes are failing due to only being modifiable as root, but being modified as user "syslogng". If you want to verify this claim, try running the daemon as root, and you should no longer see this problem. You will likely have to tune file access/location suitable to your secutiry and/or logging requirements. Regards, Justin. -----Original Message----- From: Matt Miller [mailto:syslog-ng@mattmillersf.fastmail.fm] Sent: Wed 1/3/2007 12:16 AM To: Syslog-ng users' and developers' mailing list Cc: Justin Randall Subject: RE: [syslog-ng] I/O error occurred while reading; fd='4',error='Operation not permitted (1)' > send the full output from strace without filtering specific system calls? Attached. Thanks for your help.

3 3

Re: Request for help with a filter
by G.W. Haywood 03 Jan '07

03 Jan '07

Hi there, On Wed, 3 Jan 2007 Marlier, Ian wrote: > ... need to get rid of a couple of error messages that appear on > a single host [snip] [and only those messages, defined as] > host("hostX") AND (match("ERROR 1: error 1 text") or match("ERROR 2: error 2 text")) > [snip] > > filter f_errors1 { not host("hostX") and not (match("ERROR 1:") or match("ERROR 2")); }; I think you need: not ( host("hostX") and ( match("ERROR 1:") or match("ERROR 2:") ) ) > filter f_errors2 { host("hostX") and match("ERROR 1:") or match("ERROR 2"); }; > ... not filter(f_errors1) ... This is also incorrect. You need f_errors2 { host("hostX") and (match("ERROR 1:") or match("ERROR 2")); }; and ... not filter(f_errors2) ... which I assume was just a typo. You really have to be careful with this stuff, it can bring your server down if you get it wrong. > (Perhaps more to the point, what am I totally messing up? :-)) Your logic. You need to read about the precedence of operators. Have a look at deMorgan's theorem too. But your ideas are sound. :) -- 73, Ged.

1 0

RE: [syslog-ng] I/O error occurred while reading; fd='4',error='Operation not permitted (1)'
by Justin Randall 03 Jan '07

03 Jan '07

You are correct in that fd 4 is /proc/kmsg as this can be verified in the line: open("/proc/kmsg", O_RDONLY|O_NONBLOCK|O_NOCTTY) = 4 The returned value for the call to "open" is the file descriptor. Calling "chroot" in between calls to "open" and "read" is not a problem. I've written another application which does without problems. I think however, that the path used when opening the file should be changed to the full path rather than the chroot'd absolute path. Since the call to "open" is done to "/proc/kmsg" before the chroot, this falls outside of the jail's access. Try changing the path to "/var/local/chroot/syslogng/proc/kmsg" and see if this resolves the issue. I hope this helps, Justin. -----Original Message----- From: Matt Miller [mailto:syslog-ng@mattmillersf.fastmail.fm] Sent: Tuesday, January 02, 2007 5:18 PM To: Syslog-ng users' and developers' mailing list Subject: RE: [syslog-ng] I/O error occurred while reading; fd='4',error='Operation not permitted (1)' > > running in a jail via the "-C" option, but ... syslog-ng[19522]: > > I/O error occurred while reading; fd='4', error='Operation not > > permitted (1)' > > Looking at that attached output, my best guess at what's opened on > descriptor 4 is /proc/kmsg. Could the problem be that /proc/kmsg is opened before the chroot, but then read after the chroot? _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 2

RE: [syslog-ng] I/O error occurred while reading; fd='4',error='Operation not permitted (1)'
by Justin Randall 03 Jan '07

03 Jan '07

Hey again, Could you actually send the full output from strace without filtering specific system calls? It would be helpful to see the full deal as to what's going on with the thread creation, and the calls to "close" on certain file descriptors. Thanks, Justin. -----Original Message----- From: Justin Randall Sent: Tuesday, January 02, 2007 6:32 PM To: Syslog-ng users' and developers' mailing list Subject: RE: [syslog-ng] I/O error occurred while reading;fd='4',error='Operation not permitted (1)' You are correct in that fd 4 is /proc/kmsg as this can be verified in the line: open("/proc/kmsg", O_RDONLY|O_NONBLOCK|O_NOCTTY) = 4 The returned value for the call to "open" is the file descriptor. Calling "chroot" in between calls to "open" and "read" is not a problem. I've written another application which does without problems. I think however, that the path used when opening the file should be changed to the full path rather than the chroot'd absolute path. Since the call to "open" is done to "/proc/kmsg" before the chroot, this falls outside of the jail's access. Try changing the path to "/var/local/chroot/syslogng/proc/kmsg" and see if this resolves the issue. I hope this helps, Justin. -----Original Message----- From: Matt Miller [mailto:syslog-ng@mattmillersf.fastmail.fm] Sent: Tuesday, January 02, 2007 5:18 PM To: Syslog-ng users' and developers' mailing list Subject: RE: [syslog-ng] I/O error occurred while reading; fd='4',error='Operation not permitted (1)' > > running in a jail via the "-C" option, but ... syslog-ng[19522]: > > I/O error occurred while reading; fd='4', error='Operation not > > permitted (1)' > > Looking at that attached output, my best guess at what's opened on > descriptor 4 is /proc/kmsg. Could the problem be that /proc/kmsg is opened before the chroot, but then read after the chroot? _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 1

Request for help with a filter
by Marlier, Ian 02 Jan '07

02 Jan '07

Hi, all -- I'm trying to get a little bit of help getting a filter to work. I've looked around a little bit, but haven't found a clear indication as to priority order and the like. I've got a network source set up for syslog-ng, that accepts messages from a whole bunch of different machines: source netsrc { udp(ip("0.0.0.0") port(514)); }; All of the messages that I'm interested in come from a specific application, which is configured to log everything on local5: filter f_myapp { facility(local5); }; I've got a general filter set up that sends these messages to a single log file: destination central { file("/opt/central/central"); }; log { source(netsrc); filter(f_myapp); destination(central); }; All of the above is working fine. Now, I have need to get rid of a couple of error messages that appear on a single host. BUT, I need to do this in such a way as to avoid masking those errors if they should appear somewhere else. So, basically, I need to add a second filter that eliminates from consideration anything where: host("hostX") AND (match("ERROR 1: error 1 text") or match("ERROR 2: error 2 text")) I can think of a couple of different ways to do this, but I'm not positive if either syntax will work, or if there's another that'll work better, or what...: First attempt: source netsrc { udp(ip("0.0.0.0") port(514)); }; destination central { file("/opt/central/central"); }; filter f_myapp { facility(local5); }; filter f_errors1 { not host("hostX") and not (match("ERROR 1:") or match("ERROR 2")); }; log { source(netsrc); filter(f_myapp); filter(f_errors1); destination(central); }; Second attempt: source netsrc { udp(ip("0.0.0.0") port(514)); }; destination central { file("/opt/central/central"); }; filter f_myapp { facility(local5); }; filter f_errors2 { host("hostX") and match("ERROR 1:") or match("ERROR 2"); }; log { source(netsrc); filter(f_myapp); not filter(f_errors1); destination(central); }; Will either of these work as written? What are the alternate methods that might work better? (Perhaps more to the point, what am I totally messing up? :-)) Thanks! - Ian

1 0

RE: [syslog-ng] I/O error occurred while reading; fd='4', error='Operation not permitted (1)'
by Justin Randall 02 Jan '07

02 Jan '07

With the lack of the "-C" allowing the 4th file descriptor to be read indicates that the path to this file (or link) is not accessible from the jail. There are a few tools you could use to figure out which file is generating this log message. A good approach could involve starting the daemon through strace, and looking for the system "read" which results in a return of EACCESS. The first argument should be "4" representing the process' 4th file descriptor. A typical invocation would look like "strace -f -e trace=open,read <Syslog-NG CMD>", replacing <Syslog-NG CMD> with the appropriate command + arguments. If you are using Solaris you would use the truss utility. It's invocation would be similar: "truss -f -t open,read <Syslog-NG CMD>". You may also be able to find the problem simply verifying all the files in your "source" directives in your Syslog-NG configuration file. I hope this helps, Justin. -----Original Message----- From: Matt Miller [mailto:syslog-ng@mattmillersf.fastmail.fm] Sent: Tuesday, January 02, 2007 1:02 PM To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] I/O error occurred while reading; fd='4', error='Operation not permitted (1)' I seem to have 2.0.0 running in a jail via the "-C" option, but when I start the service I see the following in /var/local/chroot/syslogng/var/log/syslog: Jan 2 16:48:19 xpc1 syslog-ng[19522]: syslog-ng starting up; version='2.0.0' Jan 2 16:48:19 xpc1 syslog-ng[19522]: I/O error occurred while reading; fd='4', error='Operation not permitted (1)' The daemon does seem to continue to run fine, since I am seeing facilities logging their activity to /var/local/chroot/syslogng/var/log/syslog. However, that second line above seems to indicate a problem. When I run syslog-ng without the "-C" option I don't get this second line. Am I missing something in my jail? Is the jailing process documented? _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

2 2

I/O error occurred while reading; fd='4', error='Operation not permitted (1)'
by Matt Miller 02 Jan '07

02 Jan '07

I seem to have 2.0.0 running in a jail via the "-C" option, but when I start the service I see the following in /var/local/chroot/syslogng/var/log/syslog: Jan 2 16:48:19 xpc1 syslog-ng[19522]: syslog-ng starting up; version='2.0.0' Jan 2 16:48:19 xpc1 syslog-ng[19522]: I/O error occurred while reading; fd='4', error='Operation not permitted (1)' The daemon does seem to continue to run fine, since I am seeing facilities logging their activity to /var/local/chroot/syslogng/var/log/syslog. However, that second line above seems to indicate a problem. When I run syslog-ng without the "-C" option I don't get this second line. Am I missing something in my jail? Is the jailing process documented?

1 0