[Bug 218] New: To include local*.* facility. level in the source of syslog-ng
https://bugzilla.balabit.com/show_bug.cgi?id=218 Summary: To include local*.* facility.level in the source of syslog-ng Product: syslog-ng Version: 3.2.x Platform: Other OS/Version: Solaris Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi@balabit.hu ReportedBy: jk_kathi1@yahoo.com Type of the Report: --- Estimated Hours: 0.0 We have oracle audits configured to send updates to local0.info facility.level and the existing syslog.conf ( solaris syslgod ) has a feature to include them in local6.info /var/log/syslog We are replacing syslogd with syslog-ng and need to replicate this property I have searched the syslog-ng documentation and there is not mention of including local6.info in the source section of syslog-ng.conf. I can use it in the filter section , but the source itself is not getting updated so filter option may be unused. I can source a file but the audit logs does not write to a file , they write to proprietary syslog's facility.level Is there a solution , i have hit a road block Any information , documentation of how i can include the facility.level into syslog-ng would be helpful source s_local { system(); sun-streams("/dev/log" door("/etc/.syslog_door")); }; # Filters #filter f_local6 { facility(local6); }; Do i include any other keywords apart from system in the source s_local line Thanks -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=218 --- Comment #1 from Balazs Scheidler <bazsi@balabit.hu> 2013-02-02 21:44:17 --- I think you misunderstand how syslogd works. That line in syslogd means that messages tagged facility "local6", severity "info" or above, should be written into /var/log/syslog. It is essentially a filter. You can do the same with syslog-ng, but you can also ask syslog-ng to send everything into a file regardless of filters. Here's a sample: # source omitted for brevity source s_local { ... }; filter f_oracle { facility(local6); } destination d_oracle { file("/var/log/oracle.log"); }; log { source(s_local); filter(f_oracle); destination(d_oracle); }; But if you want to put everything to the same file, you can completely omit the filter part: source s_local { ... }; destination d_syslog { file("/var/log/syslog"); }; log { source(s_local); destination(d_syslog); }; Basically, the "log" statement tells syslog-ng what actions to perform on messages coming from a (set of) sources. They establish connections between input and output channels of syslog-ng, potentially with additional processing. BTW: your source declaration is redundant, system() is a source that expands to the system specific local log transport of the underlying OS, on Solaris it expands to the stuff you wrote there. BTW/2: syslog-ng has a nice documentation, you might want to start with section 2.2, which outlines the basic processing model of syslog-ng. Hope this helps. source s_local { system(); sun-streams("/dev/log" door("/etc/.syslog_door")); }; -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=218 --- Comment #2 from Balazs Scheidler <bazsi@balabit.hu> 2013-02-02 21:44:52 --- One further note, it might be easier to subscribe/post to the mailing list; there are lots of helpful folks there to help you out. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=218 --- Comment #3 from kathi <jk_kathi1@yahoo.com> 2013-02-03 01:42:42 --- Sorry if i omitted the entire syslog-ng.conf in my earlier mail , the log statement is there The issue is syslog-ng is not capturing the local6.info facility messages. So even if I give the filter it is of no use This is my syslog-ng conf source s_local { system(); sun-streams("/dev/log" door("/etc/.syslog_door")); }; destination d_messages { file("/var/adm/messages"); }; log { source(s_local); destination(d_messages); }; Same issue in another system where the websphere logs are written using local0.info , syslog-ng is not able to capture that. I was of the view that system() in the source line should capture all the messages including local6 . but it is not so. Is there any other keyword to be included in the source section that can capture local*.* as well Thank you Kathiresan -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=218 --- Comment #4 from Balazs Scheidler <bazsi@balabit.hu> 2013-02-03 06:49:55 --- (In reply to comment #3)
Sorry if i omitted the entire syslog-ng.conf in my earlier mail , the log statement is there The issue is syslog-ng is not capturing the local6.info facility messages. So even if I give the filter it is of no use
This is my syslog-ng conf
source s_local { system(); sun-streams("/dev/log" door("/etc/.syslog_door")); }; destination d_messages { file("/var/adm/messages"); }; log { source(s_local); destination(d_messages); };
Same issue in another system where the websphere logs are written using local0.info , syslog-ng is not able to capture that.
I was of the view that system() in the source line should capture all the messages including local6 . but it is not so. Is there any other keyword to be included in the source section that can capture local*.* as well
your impression is correct, everything including local* should be included, assuming the applications are indeed using the local /dev/log interface (eg the syslog API) to submit messages. It may happen that they use some kind of network transport instead and send messages to localhost, port 514. To validate either, you can use tcpdump to check for network packets, or truss on the application as they submit new messsages. In any case you shouldn't need the sun-streams() part in your source as system() should be expanded to exactly that. To confirm that, you should run syslog-ng with --preprocess-into=/tmp/foo.conf argument and check what the preprocessed config looks like (there system is already expanded) hth -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=218 --- Comment #5 from kathi <jk_kathi1@yahoo.com> 2013-02-03 15:46:15 --- The application is running on a non-global zone which shares /usr /var /lib FS from the global zone the syslog-ng is installed on the global zone and saame is available on the non-global zone since the FS are shared I have edited the /usr/local/etc/syslog-ng.conf ( since /usr is shared it is avilable on the non-global zone ) syslog-ng binaries run fine. I am not sure since the app is running on the non-global zone if this scenario would work -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=218 --- Comment #6 from Balazs Scheidler <bazsi@balabit.hu> 2013-02-03 19:10:27 --- ah, so zones are in the mix. good to know. do you run syslog-ng in all the zones, or just the global one? does a single syslogd process (in the global zone) suffice in this case? -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=218 --- Comment #7 from kathi <jk_kathi1@yahoo.com> 2013-02-04 02:31:06 --- syslog-ng runs on the global zone and one sparse zone
From the global zone when i run ps -efa i see 4 syslog-ng processes running 2 for global zone and 2 for the sparse zone one common syslog-ng.conf file in /usr/local/etc
syslog-ng on the sparse zone uses the same syslog-ng.conf file. The application is running on the sparse zone which is configured to write the logs to local6.info Any other info i would be happy to provide. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=218 --- Comment #8 from Balazs Scheidler <bazsi@balabit.hu> 2013-02-04 06:36:50 --- does logger command work in the sparse zone? can you run syslog-ng in debug mode to see if it actually receives anything? also, can you confirm that the door file is not shared between the zones? unning syslog-ng in debug mode: # syslog-ng -Fedv it'll stay in the foreground, and write all internal and debug messages to the console. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=218 --- Comment #9 from kathi <jk_kathi1@yahoo.com> 2013-02-04 19:59:29 --- Syslog-ng debug output from the Sparse zone bash-3.00# /usr/local/sbin/syslog-ng -Fedv Trying to open module; module='syslogformat', filename='/usr/local/lib/syslog-ng/libsyslogformat.so' Trying to open module; module='basicfuncs', filename='/usr/local/lib/syslog-ng/libbasicfuncs.so' Trying to open module; module='afsocket', filename='/usr/local/lib/syslog-ng/libafsocket.so' Trying to open module; module='affile', filename='/usr/local/lib/syslog-ng/libaffile.so' Trying to open module; module='afprog', filename='/usr/local/lib/syslog-ng/libafprog.so' Trying to open module; module='afuser', filename='/usr/local/lib/syslog-ng/libafuser.so' Trying to open module; module='dbparser', filename='/usr/local/lib/syslog-ng/libdbparser.so' Trying to open module; module='csvparser', filename='/usr/local/lib/syslog-ng/libcsvparser.so' Trying to open module; module='afstreams', filename='/usr/local/lib/syslog-ng/libafstreams.so' Starting to read include file; filename='/usr/local/etc/scl.conf', depth='1' Global value changed; define='scl-root', value='/usr/local/share/include/scl' Global value changed; define='include-path', value='/usr/local/etc:/usr/local/share/include' Starting to read include file; filename='/usr/local/etc/modules.conf', depth='2' Global value changed; define='autoload-compiled-modules', value='0' Trying to open module; module='syslogformat', filename='/usr/local/lib/syslog-ng/libsyslogformat.so' Attempted to register the same plugin multiple times, ignoring; context='format', name='syslog' Trying to open module; module='basicfuncs', filename='/usr/local/lib/syslog-ng/libbasicfuncs.so' Attempted to register the same plugin multiple times, ignoring; context='template-func', name='echo' Attempted to register the same plugin multiple times, ignoring; context='template-func', name='grep' Attempted to register the same plugin multiple times, ignoring; context='template-func', name='if' Trying to open module; module='afsocket', filename='/usr/local/lib/syslog-ng/libafsocket.so' Attempted to register the same plugin multiple times, ignoring; context='source', name='unix-stream' Attempted to register the same plugin multiple times, ignoring; context='destination', name='unix-stream' Attempted to register the same plugin multiple times, ignoring; context='source', name='unix-dgram' Attempted to register the same plugin multiple times, ignoring; context='destination', name='unix-dgram' Attempted to register the same plugin multiple times, ignoring; context='source', name='tcp' Attempted to register the same plugin multiple times, ignoring; context='destination', name='tcp' Attempted to register the same plugin multiple times, ignoring; context='source', name='tcp6' Attempted to register the same plugin multiple times, ignoring; context='destination', name='tcp6' Attempted to register the same plugin multiple times, ignoring; context='source', name='udp' Attempted to register the same plugin multiple times, ignoring; context='destination', name='udp' Attempted to register the same plugin multiple times, ignoring; context='source', name='udp6' Attempted to register the same plugin multiple times, ignoring; context='destination', name='udp6' Attempted to register the same plugin multiple times, ignoring; context='source', name='syslog' Attempted to register the same plugin multiple times, ignoring; context='destination', name='syslog' Trying to open module; module='affile', filename='/usr/local/lib/syslog-ng/libaffile.so' Attempted to register the same plugin multiple times, ignoring; context='source', name='file' Attempted to register the same plugin multiple times, ignoring; context='source', name='pipe' Attempted to register the same plugin multiple times, ignoring; context='destination', name='file' Attempted to register the same plugin multiple times, ignoring; context='destination', name='pipe' Trying to open module; module='afprog', filename='/usr/local/lib/syslog-ng/libafprog.so' Attempted to register the same plugin multiple times, ignoring; context='source', name='program' Attempted to register the same plugin multiple times, ignoring; context='destination', name='program' Trying to open module; module='afuser', filename='/usr/local/lib/syslog-ng/libafuser.so' Attempted to register the same plugin multiple times, ignoring; context='destination', name='usertty' Trying to open module; module='dbparser', filename='/usr/local/lib/syslog-ng/libdbparser.so' Attempted to register the same plugin multiple times, ignoring; context='parser', name='db-parser' Trying to open module; module='csvparser', filename='/usr/local/lib/syslog-ng/libcsvparser.so' Attempted to register the same plugin multiple times, ignoring; context='parser', name='csv-parser' Finishing include; filename='/usr/local/etc/modules.conf', depth='2' Starting to read include file; filename='/usr/local/share/include/scl/system/plugin.conf', depth='2' Trying to open module; module='confgen', filename='/usr/local/lib/syslog-ng/libconfgen.so' Finishing include; filename='/usr/local/share/include/scl/system/plugin.conf', depth='2' Starting to read include file; filename='/usr/local/share/include/scl/pacct/plugin.conf', depth='2' Finishing include; filename='/usr/local/share/include/scl/pacct/plugin.conf', depth='2' Starting to read include file; filename='/usr/local/share/include/scl/syslogconf/plugin.conf', depth='2' Trying to open module; module='confgen', filename='/usr/local/lib/syslog-ng/libconfgen.so' Finishing include; filename='/usr/local/share/include/scl/syslogconf/plugin.conf', depth='2' Finishing include; filename='/usr/local/etc/scl.conf', depth='1' Trying to open module; module='afstreams', filename='/usr/local/lib/syslog-ng/libafstreams.so' Attempted to register the same plugin multiple times, ignoring; context='source', name='sun-streams' Trying to open module; module='afstreams', filename='/usr/local/lib/syslog-ng/libafstreams.so' Attempted to register the same plugin multiple times, ignoring; context='source', name='sun-streams' Finishing include; content='source confgen system', depth='1' Syslog connection established; fd='6', server='AF_INET(10.62.61.4:514)', local='AF_INET(0.0.0.0:0)' Running application hooks; hook='1' Running application hooks; hook='3' syslog-ng starting up; version='3.2.4' Incoming log entry; line='<38>Feb 4 12:55:08 sshd[25832]: [ID 800047 auth.info] Accepted keyboard-interactive for wasadm from 10.58.97.235 port 61300 ssh2\x0a' Initializing destination file writer; template='/var/adm/messages.0', filename='/var/adm/messages.0' Incoming log entry; line='<38>Feb 4 12:55:08 sshd[25832]: [ID 800047 auth.info] Accepted keyboard-interactive for wasadm from 10.58.97.235 port 61300 ssh2\x0a' It writes other events like sshd logins but not application events that is using local0.info I confirmed that the door file (/etc/.syslog_door ) is not shared between the global and zone /etc is not inherited between the zones. Kathi -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=218 --- Comment #10 from Balazs Scheidler <bazsi@balabit.hu> 2013-02-04 20:46:29 --- hmm, it shouldn't depend on the facility itself. I suspect the culprit is somewhere between the applications and syslog-ng. can you run 'logger -p local6.info foo' to check that sshd is indeed using /dev/log and not the network connection accepted right after startup? it should also confirm that syslog-ng writes that message to the expected log file. if all is fine, then I'm afraid something must be wrong with the applications that perform logging, probably a configuration issue, perhaps some security module? it often happens on Linux, but AFAIK Solaris has some labeled security stuff, that might interfere with logging here. If that's the case you can troubleshoot that by running the application under truss or attach to the running instance using truss -p. The important thing is to enable enough detail in the truss log. There the opening of /dev/log should be visible, and everything the app does with the fd in question should be interesting. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=218 --- Comment #11 from kathi <jk_kathi1@yahoo.com> 2013-02-04 23:35:21 --- This is what i tried 1) installed syslog-ng 3.2.4 on a stand alone box , configured syslog-ng.conf to filter local6.info messages using source s_local { system(); sun-streams("/dev/log" door("/var/run/syslog_door")); }; source s_syslogng { internal(); }; destination d_syslogng { file("/var/log/syslogng.log"); }; destination d_messages { file("/var/adm/messages.0"); }; destination d_app { file("/var/log/app.log"); }; filter f_local6 { facility(22); }; log { source(s_local); destination(d_messages); }; log { source(s_local); filter (f_local6); destination(d_app); }; log { source(s_syslogng); destination(d_syslogng); }; Works fine , i see local6.info messages logged to /var/log/app.log by the application ( Application here is Websphere which uses log4j properties to write to local6.info) When i introduce this config into a Solaris ZONE environment ( global zone and 1 sparse zone /lib, /platform, ,/sbin , /usr ) and try syslog-ng except for local6.info everthing else works fine . When disable syslog-ng and enable the default syslogd of solaris i see local6.info messages recorded in /var/log/app.log local6.info /var/log/app.log in the /etc/syslog.conf file When the sparse zone is running syslog-ng in debug mode i tried the logger command and here are the results bash-3.00# logger -p local6.info "foo" Incoming log entry; line='<182>Feb 4 16:10:59 admin: [ID 702911 local6.info] foo\x0a' Filter rule evaluation begins; filter_rule='f_local6' Filter node evaluation result; filter_result='match', filter_type='facility' Filter rule evaluation result; filter_result='match', filter_rule='f_local6' Incoming log entry; line='<182>Feb 4 16:13:01 admin: [ID 702911 local6.info] foo\x0a' Filter rule evaluation begins; filter_rule='f_local6' Filter node evaluation result; filter_result='match', filter_type='facility' Filter rule evaluation result; filter_result='match', filter_rule='f_local6' Incoming log entry; line='<182>Feb 4 16:13:01 admin: [ID 702911 local6.info] foo\x0a' Filter rule evaluation begins; filter_rule='f_local6' Filter node evaluation result; filter_result='match', filter_type='facility' Filter rule evaluation result; filter_result='match', filter_rule='f_local6' But the application logs fail to be written in the /var/log/app.log Probably because syslog-ng is never been tried in a Solaris zoned environment -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=218 --- Comment #12 from Balazs Scheidler <bazsi@balabit.hu> 2013-02-05 00:12:52 --- the PE guys use zones to test on Solaris. I don't get why logger works while the app doesn't. are you sure that log4j isn't using network transport that syslog-ng is not configured to listen for? also, I already mentioned that the system() driver should be enough. You don't need the sun-streams() part -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=218 --- Comment #13 from kathi <jk_kathi1@yahoo.com> 2013-02-14 22:14:03 --- log4j on application side is configured to log events using syslog local0.info facility .it is not using network transport. Anyways since local0.info does not work for now we have directly given the app log file path in the source source s_file { file("/opt/appp/log/access.log"); }; and defined the destination and log section appropirately syslog-ng now polls this log file for new events. But yes still curious why events do not getting logged using local0.info from inside a Solaris container. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=218 --- Comment #14 from Balazs Scheidler <bazsi@balabit.hu> 2013-02-14 22:35:55 --- like I said earlier, the facility is not relevant. is's just numeric data within the message. the issue is probably related to something else. but to troubleshoot that we would have to truss the sender application -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
participants (1)
-
bugzilla＠bugzilla.balabit.com