snmptrapd to syslog-ng 3.1
My interest is in network device syslog and traps. I'm trying to receive traps, and then process them in Simple Event Correlator (SEC). I've got SEC working fine with standard remote syslog. After reading everything I could find, I found a discussion from 2008 about losing the source hostname when sending the trap to syslog. I'm trying the source program method, and eliminating multi-line traps. As I understand that syslog-ng is looking for Standard Out from the program, I specified -f in snmptrapd to stop forking, and -Lo to send output to standard output. I'm getting nothing in my d_debug file. Any suggestions? There is also a web page with a filter and rewrite recipe for traps. Not sure why I need this if I am sending the trap directly to SEC. See https://lists.balabit.hu/pipermail/syslog-ng/2008-November/012200.html And http://bazsi.blogs.balabit.com/2008/11/syslog-ng-3-0-and-snmp-traps/ -=Dan=- ========= syslog-ng.conf ================= # options { long_hostnames(off); use_dns(yes); use_fqdn(no); keep_hostname(yes); owner("root"); group("adm"); perm(0640); stats_freq(0); bad_hostname("^gconfd$"); ts_format(iso); flush_lines(100); log_fetch_limit(100); log_fifo_size(2048); dir_perm(0755); }; source s_program { program("/usr/sbin/snmptrapd -a -f -Lo --disableAuthorization=yes", flags(no-multi-line)); }; destination d_debug { file("/var/log/syslog-ng-debug" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); }; destination d_sec { program("/usr/local/bin/sec -input=\"-\" -conf=/usr/local/etc/sec.conf" flags(no-multi-line) ); }; log { source(s_program); destination(d_sec); destination(d_debug); flags(flow-control); };
Hi Dan, when I do something like this, I usually take a different approach, I use a script called from snmptrapd to write the log message to a socket. If you use something different than /dev/log (e.g. /dev/log.snmp), it's also easier to filter on the message. I am not sure if SEC has pre-defined rules for traps, but if not, this is probably easier to handle of you are not using a LOT of traps. If it's just port-security violations and link up/down messages on a moderately sized network, it should work fine. So in my setup this usually looks like: /etc/snmp/snmp.conf mibdirs +/usr/share/snmp/mibs/:/etc/snmp/mibs/ mibs +ALL mibwarninglevel 1 logtimestamp yes printnumericenums no printnumericoids no suffixprinting 0 /etc/snmp/snmptrapd.conf # syslog-ng configuration doNotRetainNotificationLogs yes doNotLogTraps yes snmpTrapdAddr 0.0.0.0:162 authCommunity execute public logOption s 10 outputOption Q traphandle default /usr/local/bin/traptosyslog /usr/local/bin/traptosyslog #!/usr/bin/python import sys, time, socket t = time.strftime('%Y-%m-%dT%H:%M:%S') hostname = None ipaddress = None trap = None oids = [] for line in sys.stdin: if not hostname: hostname = line.strip() elif not ipaddress: ipaddress = line.strip() else: (n, v) = line.split('=', 1) if n.strip() == "SNMPv2-MIB::snmpTrapOID.0": (base, real) = v.strip().split('::', 1) trap = real else: if n.find('::') > 0: (base, real) = n.strip().split('::', 1) oids.append("%s='%s'" % (real, v.strip('\'"\n '))) else: oids.append("%s='%s'" % (n.strip(), v.strip('\'"\n '))) oids.reverse() if hostname == "<UNKNOWN>": b = ipaddress.find('[') e = ipaddress.find(']') if e > 0 and b > 0: hostname = ipaddress[b+1:e] else: hostname = ipaddress sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) sock.connect("/dev/log.snmp") sock.sendall("%s %s snmptrap: %s; %s" % (t, hostname, trap, ', '.join(oids))) sock.close() /etc/syslog-ng/syslog-ng.conf [...] source snmp { unix-stream( "/dev/log.snmp" keep_timestamp(yes) keep_hostname(yes) ); }; [...] This way the hostname and timestamp are preserved, the SNMP trap is nicely collapsed to a single line and formatted according to the matching MIB definition. All MIBs are loaded from /etc/snmp/mibs and if you need to add one more, just drop it into that folder and reload snmptrapd. Balint On 08/17/2011 11:17 PM, Smart, Dan wrote:
My interest is in network device syslog and traps.
I'm trying to receive traps, and then process them in Simple Event Correlator (SEC). I've got SEC working fine with standard remote syslog.
After reading everything I could find, I found a discussion from 2008 about losing the source hostname when sending the trap to syslog.
I'm trying the source program method, and eliminating multi-line traps.
As I understand that syslog-ng is looking for Standard Out from the program, I specified --f in snmptrapd to stop forking, and --Lo to send output to standard output. I'm getting nothing in my d_debug file. Any suggestions?
There is also a web page with a filter and rewrite recipe for traps. Not sure why I need this if I am sending the trap directly to SEC.
See https://lists.balabit.hu/pipermail/syslog-ng/2008-November/012200.html
And http://bazsi.blogs.balabit.com/2008/11/syslog-ng-3-0-and-snmp-traps/
-=Dan=-
========= syslog-ng.conf =================
# options { long__hostnames_(off); use__dns_(yes); use__fqdn_(no); keep__hostname_(yes); owner("root"); group("_adm_"); perm(0640); stats__freq_(0); bad__hostname_("^_gconfd_$"); ts_format(_iso_); flush_lines(100); log_fetch_limit(100); log__fifo__size(2048); _dir__perm(0755); };
source s_program {
program("/_usr_/_sbin_/_snmptrapd_ -a -f -Lo --_disableAuthorization_=yes", flags(no-_multi_-line));
};
destination d_debug { file("/var/log/_syslog_-_ng_-debug" owner(root) group(root) perm(0600) _dir__perm(0700) create__dirs_(yes)); };
destination d__sec_ { program("/_usr_/local/bin/_sec_ -input=\"-\" -_conf_=/_usr_/local/_etc_/_sec_._conf_" flags(no-_multi_-line) ); };
log {
source(s_program);
destination(d_sec); destination(d_debug);
flags(flow-control);
};
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
From: Balint Kovacs [mailto:balint.kovacs@balabit.com] Sent: Thursday, August 18, 2011 2:35 AM
when I do something like this, I usually take a different approach, I use a script called from snmptrapd to write the >log message to a socket. If you use something different than /dev/log (e.g. /dev/log.snmp), it's also easier to filter >on the message. I am not sure if SEC has pre-defined rules for traps, but if not, this is probably easier to handle of >you are not using a LOT of traps. If it's just port-security violations and link up/down messages on a moderately >sized network, it should work fine.
Balint, Thanks for the configs. I'm guessing that snmptrapd is running in daemon mode. As such, it reads its options from /etc/default/snmpd. What options do you have set for snmptrapd? Thanks -=Dan=-
On 08/19/2011 12:19 AM, Smart, Dan wrote:
Balint, Thanks for the configs. I'm guessing that snmptrapd is running in daemon mode. As such, it reads its options from /etc/default/snmpd.
What options do you have set for snmptrapd?
Thanks -=Dan=- Hi Dan,
yes, you are right, it runs as a daemon. The cmdline is the following: /usr/sbin/snmptrapd -A -LF d /var/log/net-snmpd.log -p /var/run/snmptrapd.pid This way it reads the config files from their default locations. Balint
Balint, Thanks for the configs. Is there a way to debug this to make sure its working? I don't seem to get anything in my syslog-ng-debug file (file not created) I put in the recipe in syslog-ng... source snmp { unix-stream( "/dev/log.snmp" keep_timestamp(yes) keep_hostname(yes) ); }; destination d_debug { file("/var/log/syslog-ng-debug" owner(root) group(adm) perm(0660)); }; log { source(snmp); destination(d_debug); }; ================================================= Here's what I can see....
From netstat unix 6 [ ] DGRAM 77199 3475/syslog-ng /dev/log unix 2 [ ] DGRAM 89815 10562/snmptrapd unix 2 [ ACC ] STREAM LISTENING 77193 /dev/log.snmp udp 0 0 *:snmp-trap *:*
Tcpdump show trap data on interface. Not sure how to debug the unix-stream and syslog-ng. Any pointers? -=Dan=-
Balint, I compiled and installed the 3.7 version of Net-SNMP, and now snmptrapd combo appears to be working. Not sure what it took, but I'm happy with the results. Thanks for your help. -=Dan=-
Hi Dan, I am glad to hear that! Just for the record, there are several points where you could debug this kind of setup: - set "doNotLogTraps no" in /etc/snmp/snmptrapd.conf, this way you see that snmptrapd processes the trap correctly - add some logging to the python script (to a file) so that you see that it has been started by snmptrapd and has received the trap correctly - start syslog-ng in the foreground in debug mode (/usr/sbin/syslog-ng -evtdF) so that it prints incoming log messages to stdout to see if it's a formatting issue or the trap has not been received. Another approach is that instead of syslog-ng start up socat or something alike to listen to that socket to see if the trap arrives. All the best, Balint On 08/19/2011 11:53 PM, Smart, Dan wrote:
Balint, I compiled and installed the 3.7 version of Net-SNMP, and now snmptrapd combo appears to be working. Not sure what it took, but I'm happy with the results. Thanks for your help.
-=Dan=-
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
participants (2)
-
Balint Kovacs
-
Smart, Dan