Hi, This is probably the last thing you care about during the release candidate phase but a single bit of extra functionality would be greatly useful for those using the multicast logging feature. What I am planning on doing is having something like the following in my syslog-ng.conf file which is fantastic as to add logging groups I just add extra udp() lines to s_multicastGeneric and nothing else is needed. At the moment I would need separate source/destination/log entries per multicast group address, which is upsetting and results in a hideous configuration file. ==================== source s_multicastGeneric { ## squid # access udp(ip(239.194.251.0) port(514)); # cache udp(ip(239.194.251.32) port(514)); }; destination df_multicastGeneric { file("/var/log/syslog-ng/mulitcastGeneric/$DESTIP/$R_YEAR$R_MONTH$R_DAY.log"); }; log { source(s_multicastGeneric); destination(df_multicastGeneric); flags(final); }; ==================== I have actually had a go at patching syslog-ng to do this, however my patching effort is getting out of control. My method is to add the IP_PKTINFO (or IP_RECVDSTADDR) socket option to the multicast receive section in afinet_setup_socket() but thats about as far as I have gotten. I'm still working on trying to produce a patch myself but I am hoping someone on this list might have a better idea on how to cleanly get the information needed for $DESTIP to function from macros.c without doing anything major to the sourcecode. Whilst I am in my 'moon-on-a-stick' request list, a $HOST_TO in addition to $DESTIP would be handy too. :) As I said, I'm working on a patch at my end and will post it here once done, but its unlikely to be pretty... :) Cheers Alex
Hi, Time to reply to my own email.... Alexander Clouter <ac56@soas.ac.uk> [20061019 10:25:38 +0100]:
[snipped]
What I am planning on doing is having something like the following in my syslog-ng.conf file which is fantastic as to add logging groups I just add extra udp() lines to s_multicastGeneric and nothing else is needed. At the moment I would need separate source/destination/log entries per multicast group address, which is upsetting and results in a hideous configuration file.
==================== source s_multicastGeneric { ## squid # access udp(ip(239.194.251.0) port(514)); # cache udp(ip(239.194.251.32) port(514)); };
destination df_multicastGeneric { file("/var/log/syslog-ng/mulitcastGeneric/$DESTIP/$R_YEAR$R_MONTH$R_DAY.log"); };
log { source(s_multicastGeneric); destination(df_multicastGeneric); flags(final); }; ====================
[snipped]
As I said, I'm working on a patch at my end and will post it here once done, but its unlikely to be pretty... :)
well I said it was going to be nasty, the patch has been attached for peer review. It really does things in probably a manner that is going to make Balazs break down into tears. The patch enables you to do the above so that you can have a group of machines all logging to the same multicast address and all their messages appear in the same logfile. To add new *groups* of machines you simply add the single extra multicast udp() line to s_multicastGeneric you want to listen out for. This is as opposed to maintaining a collection of source address IP's to filter under which can become un-managable if your groups become large in the number of hosts they contain. The only other way I thought I could implement the patch, that I could think of, was to make my admendments in log_source_group_queue() however[1] I thought it would be more efficient to put it in the macros.c file so that the conversion of the binary address into something human readable would only occur if $DESTIP[2] is evaluated. Anyway, it "Works for Me(tm)" and something like this in syslog-ng would be very helpful. Obviously we are in the release candidate stage so I don't expect it for version 2.0 :) I look forward to the feedback, flames, onslaught... Cheers Alex [1] also I could not find out how to get to the original socket that had been read from easily [2] I chose not to bother with $HOST_TO (look up the PTR record for the destination multicast group) as I though it would be best to get this looked at first and I can live without $HOST_TO 'support'
On Fri, 2006-10-20 at 17:19 +0100, Alexander Clouter wrote:
Hi,
Time to reply to my own email....
Alexander Clouter <ac56@soas.ac.uk> [20061019 10:25:38 +0100]:
[snipped]
What I am planning on doing is having something like the following in my syslog-ng.conf file which is fantastic as to add logging groups I just add extra udp() lines to s_multicastGeneric and nothing else is needed. At the moment I would need separate source/destination/log entries per multicast group address, which is upsetting and results in a hideous configuration file.
It took me a while to understand what you want to accomplish, and to be honest the only reason I understood it is that you attached a patch. :) The reason I did not understand your original post was that I can't remember all defined macro names in syslog-ng and $DESTIP in your configuration snipped simply did not jump out at first. :) In fact the patch is not that horrible :) and the idea is sane. It is also useful when someone has a listener bound to '0.0.0.0' and logs are to be put on a receiving-interface specific file. Thanks for your contribution, I'll see to adding a DESTIP support, once I have dealt with the current blockers for 2.0.0. -- Bazsi
Hi, Balazs Scheidler <bazsi@balabit.hu> [20061021 21:26:38 +0200]:
On Fri, 2006-10-20 at 17:19 +0100, Alexander Clouter wrote:
Hi,
Time to reply to my own email....
Alexander Clouter <ac56@soas.ac.uk> [20061019 10:25:38 +0100]:
[snipped]
What I am planning on doing is having something like the following in my syslog-ng.conf file which is fantastic as to add logging groups I just add extra udp() lines to s_multicastGeneric and nothing else is needed. At the moment I would need separate source/destination/log entries per multicast group address, which is upsetting and results in a hideous configuration file.
It took me a while to understand what you want to accomplish, and to be honest the only reason I understood it is that you attached a patch. :)
heh heh.
The reason I did not understand your original post was that I can't remember all defined macro names in syslog-ng and $DESTIP in your configuration snipped simply did not jump out at first. :)
In fact the patch is not that horrible :) and the idea is sane. It is also useful when someone has a listener bound to '0.0.0.0' and logs are to be put on a receiving-interface specific file.
One thing to bear in mind, I am pretty sure that using getsockname() will only return the *binding* address, so you might actually only get '0.0.0.0' back from a socket thats bound to 0.0.0.0. Its going to show you no more than what you would gain looking at the 'local address' coloumn when you run 'netstat -lnut'. The getsockname() method works for me as I explictly tell syslog-ng to listen on a particular address, I bind with 'udp(ip(239.123.120.5) port(514))', so calling getsockname() on that socket returns '239.123.120.5'. This is not so helpful for others unless they also type specific multiple udp()/tcp() commands into their configuration, however getsockname() *very* cross platform compatible. In practice its not a bad limitation as you would hope the syslog-ng logging servers out there are on static IPs :) The alternative approach is to use the Linux specific socket option IP(V6)_PKTINFO[1], or for *BSD people its IP_RECVDSTADDR (for both IPv4 and IPv6) if I am right, and then instead of calling recvfrom() you use recvmsg() and you get a lot of UDP header information.
Thanks for your contribution, I'll see to adding a DESTIP support, once I have dealt with the current blockers for 2.0.0.
Cheers, that would be great if you could. Have a good weekend Alex [1] http://groups.google.com/group/linux.kernel/msg/e5df1c1bf1fa7aea?dmode=sourc...
-- Bazsi
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
participants (2)
-
Alexander Clouter
-
Balazs Scheidler