- syslog-ng - lists.balabit.hu

Backup Logfiles to specific folder of a remote machine
by amy wong 02 Jul '04

02 Jul '04

Hi all, I have recently setup a central syslog-ng server to receives log from cisco router and other server. I am intend to store a backup copy of this log files in a specific folder of a remote machine. Would anyone show me how can I do that? thanks amy

1 0

Backup Logfiles to specific folder of a remote machine
by amy wong 02 Jul '04

02 Jul '04

Hi all, I have recently setup a central syslog-ng server to receives log from cisco router and other server. I am intend to store a backup copy of this log files in a specific folder of a remote machine. Would anyone show me how can I do that? thanks, amy

1 0

Re: [syslog-ng]problem on HPUX with s_tcp snd d_tcp
by Stephan Hendl 02 Jul '04

02 Jul '04

I've just upgraded all version to 1.6.4 and the problem is the same. After a kill -HUP on the hpux-server the clients are going into CLOSE_WAIT state where the server is in the FIN_WAIT_2 state client: tcp 1 0 <client>:55682 <server>:10514 CLOSE_WAIT server tcp 0 0 <server>.10514 <client>.55682 FIN_WAIT_2 After a 'service syslog-ng restart' on all clients they open a new connection to the server and the connection will be established as I expect. I hope that helps.. Stephan

2 1

Re: [syslog-ng]Can't get pipe working
by Stephan Hendl 02 Jul '04

02 Jul '04

Hi Clayton, wie use ist in slyghtly other way and it writes several gigabytes per dy in our mysql database. 1) the syslog-ng configuration destination d_ToSysLogDB { pipe("/tmp/syslog.pipe" owner("root") group("root") perm(0640) template("insert into syslog(host,facility,priority,level,tag,date,time,progra m,msg) values ('$HOST','$FACILITY', '$PRIORITY', '$LEVEL', '$TAG','$YEAR-$MONTH- $DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; log { source(loghost); destination(d_ToSysLogDB); }; 2) there is a script running that takes the data from the pipe and writes them into the databse #!/bin/bash # if [ -e /tmp/syslog.pipe ]; then while [ -e /tmp/syslog.pipe ] do mysql -u syslog --password=<password> syslog -h syslog < /tmp/syslog.pipe done else mkfifo /tmp/syslog.pipe while [ -e /tmp/syslog.pipe ] do mysql -u syslog --password=<password> -h syslog < /tmp/syslog.pipe done fi 3) Keep in mind that a tail -f on a pipe will take all data out of the pipe and you cannot write them into a database. Stephan Dr. Stephan Hendl Systemmanagement ----------------------------------- Landesbetrieb für Datenverarbeitung und Statistik Land Brandenburg Adresse: 14467 Potsdam, Dortustr. 46 Telefon: +49-(0)331 39-471 Fax: +49-(0)331 27548-1187 Mobil: +49-(0)160 90 645 893 EMail: Stephan.Hendl(a)lds.brandenburg.de Internet: http://www.lds-bb.de >>> cdukes(a)cdukes.com 01.07.2004 20:02:57 >>> Hi all, I seem to be having trouble getting a mysql pipe working and thought I would enlist your help. Here's my config: options { use_fqdn(no); keep_hostname(yes); use_dns(no); long_hostnames(off); sync(0); log_fifo_size(1000); # The default action of syslog-ng 1.6.0 is to log a STATS line # to the file every 10 minutes. That's pretty ugly after a while. # Change it to every 12 hours so you get a nice daily update of # how many messages syslog-ng missed (0). stats(43200); }; ############################ # Set up Sources ############################ source src { unix-dgram("/dev/log"); internal(); }; source net { udp(); }; source s_jffnms { unix-dgram("/dev/log"); internal(); udp(); }; # MySQL Logs destination d_jffnms { pipe("/tmp/mysql.pipe" template("INSERT INTO syslog (date, date_logged, host, message) VALUES ('$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', NOW, '$FULLHOST', '$MSG');\n") template-escape(yes)); }; destination jffnms_processing { program ("mysql -u jffnms -pjffnms jffnms < /tmp/mysql.pipe"); }; log { source (s_jffnms); #filter (f_jffnms); #if you use it destination (d_jffnms); }; # End When I restart syslog-ng and tail -f /tmp/mysql.pipe nothing comes in (nor are there any database inserts happening). When I do a tcpdump, I do see the event coming into my server, but it seems as though syslog-ng is just dropping the packet...? Regards, Clayton Dukes CCNA, CCDA, CCNP, CCDP Sr. Network Engineer E Solutions Corp. http://www.esnet.com 813.301.2620 (o) 813.545.7373 (c) _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

Syslog-Mysql-error ( ATTN: Asif Iqbal)
by Venkata Phani Kiran Achanta 02 Jul '04

02 Jul '04

Greetings all, Everything was going well until I got this error while loading the Mysql tables.... DBD::mysql::db do failed: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near '192.168.254.250', 'auth', 'notice', 'notice', '25', '2004-07-01 at ./db-loader.pl line 19, <FH> line 1. And my line 1 in mysql.pipe looks like this INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( '192.168.254.250', 'auth', 'notice', 'notice', '25', '2004-07-01', '15:26:28', 'DNS_NS25','DNS_NS25: NetScreen device_id=DNS_NS25 [No Name]system-notification-00257(traffic): start_time=\"2004-07-01 15:30:26\" duration=2 policy_id=0 service=dns proto=17 src zone=Trust dst zone=Untrust action=Permit sent=99 rcvd=274 src=192.168.254.2 dst=63.209.70.231 src_port=1116 dst_port=53 src-xlated ip=12.28.143.203 port=1116 Any ideas....let me know if you need more info to get me out of this error. Thanks in advance. I have searched thru the archives and found Mr.Asif Iqbal had a similar problem but unfortunately he did not find an answer in this list. Hey Asif, If you are still subscribing to this list and see this message please respond ASAP explaining how you worked around this problem. Thanks much, Venkata Achanta --Open source Rocks.........

1 0

Can't get pipe working
by Clayton Dukes 01 Jul '04

01 Jul '04

Hi all, I seem to be having trouble getting a mysql pipe working and thought I would enlist your help. Here's my config: options { use_fqdn(no); keep_hostname(yes); use_dns(no); long_hostnames(off); sync(0); log_fifo_size(1000); # The default action of syslog-ng 1.6.0 is to log a STATS line # to the file every 10 minutes. That's pretty ugly after a while. # Change it to every 12 hours so you get a nice daily update of # how many messages syslog-ng missed (0). stats(43200); }; ############################ # Set up Sources ############################ source src { unix-dgram("/dev/log"); internal(); }; source net { udp(); }; source s_jffnms { unix-dgram("/dev/log"); internal(); udp(); }; # MySQL Logs destination d_jffnms { pipe("/tmp/mysql.pipe" template("INSERT INTO syslog (date, date_logged, host, message) VALUES ('$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', NOW, '$FULLHOST', '$MSG');\n") template-escape(yes)); }; destination jffnms_processing { program ("mysql -u jffnms -pjffnms jffnms < /tmp/mysql.pipe"); }; log { source (s_jffnms); #filter (f_jffnms); #if you use it destination (d_jffnms); }; # End When I restart syslog-ng and tail -f /tmp/mysql.pipe nothing comes in (nor are there any database inserts happening). When I do a tcpdump, I do see the event coming into my server, but it seems as though syslog-ng is just dropping the packet...? Regards, Clayton Dukes CCNA, CCDA, CCNP, CCDP Sr. Network Engineer E Solutions Corp. http://www.esnet.com 813.301.2620 (o) 813.545.7373 (c)

1 0

Cisco Syslog with Syslog-ng
by Davis, Jay 01 Jul '04

01 Jul '04

From: amy wong amywsp(a)gmail.com Date: Tue, 29 Jun 2004 17:03:14 +0800 Subject: Cisco Log To: bazsi(a)balabit.hu Hi, I'm new in syslog-ng. Currently install syslog-ng-1.4.7. I have got the cisco router monitor, and log can be seen on /var/log/messages but not /var/log/cisco.log. My questions are do the cisco.log file will be created automatically? and would you show me the right way in setting up? source remote {tcp(); udp();}; destination d_cisco { file("/var/log/cisco.log); }; filter f_cisco_info { level(info); }; filter f_cisco_notice { level(notice); }; filter f_cisco_warn { level(warn); }; filter f_cisco_crit { level(crit); }; filter f_cisco_err { level(err); }; log { source(remote); filter(f_cisco_info); destination(d_cisco); }; log { source(remote); filter(f_cisco_notice); destination(d_cisco); }; log { source(remote); filter(f_cisco_warn); destination(d_cisco); }; log { source(remote); filter(f_cisco_crit); destination(d_cisco); }; log { source(remote); filter(f_cisco_err); destination(d_cisco); }; destination hosts { file("/var/log/HOSTS/$HOSTS/$YEAR/$MONTH/$DAY/$FACILITY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)); }; log { source(remote); destination(hosts); }; +++++++++++++++++++++++++++++++ Had to cut and paste First the setup on the Cisco devices. Cisco uses facility 7 for syslog. You need to set the proper severity levels for what you plan on sending to your syslog host. Do not use debug, you will kill the router or switch possibly. YMMV. I took some snippets of our config file here. The filter just puts all levels of syslog into one file which is parsed by netcool. You will have to play around with the severities. Cisco is kind of confused (being nice) on some severities. I think you will end up adjusting and using what you think is critical than using what they think is critical. Remember that you can filter specific messages using syslog-ng i.e. LINK-5-UPDOWN. The 5 is severity. I hope this helps. # The udp and port 514 are defaults but sometimes we change them here for diagnostics. # It is easier to leave in so you don't have to dig through docs to find out how to set the port source s_udpmessages {udp(ip(<IPADDRESSOFLOCALINTERFACETOLISTENON>) port(514));}; #This is a log file for Netcool destination d_mesg { file("/var/log/ncolog"); }; filter f_filter7 { facility(local7) and level(emerg,alert,crit,err,warning,notice,info); }; log { source(s_udpmessages); filter(f_filter7); destination(d_mesg); };

2 1

Re: [syslog-ng]problem on HPUX with s_tcp snd d_tcp
by Stephan Hendl 01 Jul '04

01 Jul '04

maybe the problem is that the server side (central syslog) is hpux. Unfortunately I do not have a linux box for this purpose... With a tcpdump on the server-side you can see that even the syslog-ng daemon is down there are many tcp pakets going from the clients to the server. The clients do not recognise that the server process has died. On client side I have syslog-ng 1.6.0-rc4 where I have on the hpux server side 1.6.4. What can I do with the socket(7) option? I am not familiar with tcp/ip internals ;-((( Stephan >>> lool+syslog(a)via.ecp.fr 01.07.2004 14:52:21 >>> Stephan Hendl <Stephan.Hendl(a)lds.brandenburg.de> - Thu, Jul 01, 2004: > It seems to me that the state 'FIN_WAIT_2' doesn't allow the hpux > syslog-ng to accept data from the linux clients and this state is quite > very long. I could not reproduce this on my Linux 2.6 system: - I killed syslog-ng with SIGTERM, SIGINT, SIGKILL, and SIGHUP while a telnet connection was open to the TCP port; - I restarted syslog-ng as soon as it was dead, and telneted successfully to the same port again, and syslog-ng saw my connection. Could it be a problem on the client side? My TCP connection gets closed when I kill -HUP syslog-ng. Or could it be a socket(7) option missing on the listening TCP socket? I do a setsockopt with SO_REUSEADDR on my sockets prior to binding them, which does not seem to be in syslog-ng. -- Loïc Minier <lool(a)dooz.org> _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

RE: [syslog-ng]use_time_recvd() not working?
by Hall J D (ISeLS) 01 Jul '04

01 Jul '04

Thanks to Michael pointing me at the mail archive I now understand that use_time_recvd() only applies to macros used in filename expansion and logformat templates. So to get the behaviour I was expecting, that use_time_recvd(yes) would effect the "default" template, I need to define a template for my destination. Would template("$DATE $HOST $MSG\n") cover it? While looking through the mail archives I saw mention of many other macros, R_DATE, S_DATE etc., which don't appear in the documentation. Do these still exist and if so what do they do? Are they left out of the documentation for a reason? Thanks again, Jonathan -----Original Message----- From: Trapp, Michael [mailto:michael.trapp@sap.com] Sent: 30 June 2004 16:04 To: Hall J D (ISeLS) Cc: 'syslog-ng(a)lists.balabit.hu' Subject: RE: [syslog-ng]use_time_recvd() not working? hi jonathan, have a look at https://lists.balabit.hu/pipermail/syslog-ng/2002-September/003874.html regards michael -----Original Message----- From: syslog-ng-admin(a)lists.balabit.hu [mailto:syslog-ng-admin@lists.balabit.hu] On Behalf Of Hall J D (ISeLS) Sent: Mittwoch, 30. Juni 2004 16:53 To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng]use_time_recvd() not working? Hello all, I've recently installed Syslog-ng 1.6.2 on a FreeBSD 4.9 to act as my new collector and I can't get the use_time_recvd() option to work properly. No matter if I specify use_time_recvd(yes) or use_time_recvd(no) the messages, from a Cisco PIX firewall, are still getting recorded with the time from the message and not the local time. Is this a know issue, or am I doing something really silly? Below are the relevant bits from my config Thanks, Jonathan options { long_hostnames(off); sync(0); use_time_recvd(yes); create_dirs(yes); dir_perm(0750); }; source net { udp(ip(193.63.147.98) port(514)); tcp(ip(193.63.147.98) port(1740) keep-alive(yes)); }; destination fwall { file("/var/log/firewalls/$HOST.$YEAR.$MONTH.$DAY.log" perm(0640)); }; filter f_pixmsg { match("%PIX"); }; filter f_local0 { facility(local0); }; log { source(net); filter(f_local0); filter(f_pixmsg); destination(fwall); };

1 0

problem on HPUX with s_tcp snd d_tcp
by Stephan Hendl 01 Jul '04

01 Jul '04

Hi all, I have an central syslog host on hpux 11.0 running syslog-ng-1.6.2 for five linux systems (pns[3-7]). The linux systems send syslog data via tcp to port 10514 on the hpux system (pns1). The hpux system listen on that port, gets the data and put them into a file called 'cic.log' as well as to localhost Port 10514, where a ssh-tunnel is listening. The normal behaviour is like root@pns1:/> netstat -a | grep 10514 tcp 0 0 localhost.49297 localhost.10514 TIME_WAIT tcp 0 0 pns1.10514 *.* LISTEN tcp 0 0 localhost.10514 *.* LISTEN tcp 98 0 localhost.10514 localhost.49153 ESTABLISHED tcp 0 0 pns1.10514 pns3.55228 ESTABLISHED tcp 98 0 pns1.10514 pns4.16048 ESTABLISHED tcp 0 0 pns1.10514 pns5.55717 ESTABLISHED tcp 0 0 pns1.10514 pns6.32356 ESTABLISHED tcp 0 0 pns1.10514 pns7.49633 ESTABLISHED tcp 0 98 localhost.49153 localhost.10514 ESTABLISHED root@pns1:/> After a 'kill -HUP $PID' due to confoguration changes of the syslog-ng process on the hpux box the syslog-ng doesn't get anymore data via tcp (the file cic.log as well as the stream to the tunnel are empty). The netstat output is like root@pns1:/var/adm/syslog> netstat -a | grep 514 tcp 0 0 localhost.49618 localhost.10514 TIME_WAIT tcp 0 0 localhost.49587 localhost.10514 TIME_WAIT tcp 0 0 localhost.49619 localhost.10514 ESTABLISHED tcp 0 0 pns1.10514 pns3.55228 FIN_WAIT_2 tcp 0 0 pns1.10514 pns4.16048 FIN_WAIT_2 tcp 0 0 pns1.10514 pns5.55717 FIN_WAIT_2 tcp 0 0 pns1.10514 pns6.32356 FIN_WAIT_2 tcp 0 0 pns1.10514 pns7.49633 FIN_WAIT_2 tcp 0 0 localhost.10514 *.* LISTEN tcp 0 0 pns1.10514 *.* LISTEN tcp 0 0 localhost.10514 localhost.49619 ESTABLISHED root@pns1:/var/adm/syslog> It seems to me that the state 'FIN_WAIT_2' doesn't allow the hpux syslog-ng to accept data from the linux clients and this state is quite very long. Then I inserted the prameter 'keep-alive(no)' into the configuration file of the hpux box but it doesn't help. The only thing that helps is a 'service syslog-ng restart' on every linux client. But that cannot be done autimatically ;-(( and is not the preferred way. Maybe there are some problems in the tcp/ip code of syslog-ng running on hpux? Thanks for helping me! regards Stephan

2 1