July 2004 - syslog-ng - lists.balabit.hu

Help with sorting by host
by phusion 05 Jul '04

05 Jul '04

Hi, I can't figure out why the following won't work. My server config file is at the bottom. I'm trying to get it to log service logs locally on the server to the directory /var/log/ (multiple files) like normal, and also to the directory /storage/logs/$YEAR/$MONTH/$DAY/$HOST/ (multiple files), so all the service logs for each host will go into that specific host directory like host1 or host2. Also, named and sshd log messages to syslog-ng. I'm not sure what the problem is. Let me know what you think. Thanks. # CONFIG FILE # Options # options { long_hostnames(off); sync(0); }; # Sources # source local { unix-dgram("/var/run/log"); internal(); file("/dev/klog"); }; source network { tcp(port(514) max-connections(10) keep-alive(yes)); }; # Destinations # destination localhost { file("/var/log/all.log"); }; destination hosts { file("/storage/logs/$YEAR/$MONTH/$DAY/$HOST" owner(root) group(wheel) perm(0600) dir_perm(0700) create_dirs(yes)); }; # Filters # filter d_named { match("named[\[0-9]+\]"); }; filter d_sshd { match("sshd[\[0-9]+\]") and match("Server listening") or match("Connection from") or match("(Accepted|Failed) password") or match("Connection closed") or match("Closing connection"); }; filter h_host1 { host("host1"); }; filter h_host2 { host("host2"); }; # Logs # log { source(local); destination(localhost); }; log { source(local); filter(h_host1); filter(d_named); filter(d_sshd); destination(hosts); }; log { source(remote); filter(h_host2); filter(d_sshd); destination(hosts); }; __________________________________ Do you Yahoo!? Yahoo! Mail is new and improved - Check it out! http://promotions.yahoo.com/new_mail

1 0

garbled kernel messages
by Balazs Scheidler 04 Jul '04

04 Jul '04

Hi, I discovered a problem recently and I wanted to let you know about it. The Linux kernel may drop bytes from its ringbuffer which causes a garbled message to reach the logs. For example, I enabled logging on my notebook for all loopback traffic, and used ping -f to generate a lot of messages. To avoid syslog-ng's complexity in the picture, I simply used "cat" to read kernel messages: # cat /proc/kmsg > kmsg ## generated about 40000 packets # grep -v '^<4>IN=lo OU' kmsg RC=127.0.0.1 DST=127.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=15957 DF PROTO=ICMP TYPE=8 CODE=0 ID=21848 SEQ=21822 IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00 SRC=127.0.0.1 DST=127.0.0.1 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=11274 DF PROTO=ICMP TYPE=8 CODE=0 ID=22104 SEQ=2604 As you can see both messages missed the syslog header (the string <4>), but messages might be garbled in different ways, in general the linux kernel shifts the ring buffer whenever it is full, without flow control. I also checked the the cat process with "strace" to confirm that the kernel itself returns garbled messages. A workaround is to increase the kernel ring buffer size which is possible using a compile time option in recent kernels, but it is also possible to change by patching the value of LOG_BUF_LEN in the beginning kernel/printk.c to a greater value (must be power of two) However this will not necessarily fix the problem, the kernel can always "outperform" its userspace, e.g. it can generate way more messages than a userspace program can handle. -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1

1 0

error when installing libol on HP UX
by Albert Collazo 03 Jul '04

03 Jul '04

Hi, i'm new to this...and when i hit the "make" command for libol i get the following error: (Bundled) cc: "format.c", line 306: error 1549: Modifiable lvalue required for a ssignment operator. do you know why? and what is the work around? i have libol-0.3.13. thanks!! al _________________________________________________________________ FREE pop-up blocking with the new MSN Toolbar get it now! http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/

2 1

Antw: [syslog-ng]error when installing libol on HP UX
by Stephan Hendl 03 Jul '04

03 Jul '04

Using the newest gcc (3.2.? I think) available from the hp porting centre it worked for me. With the hpux ansi c compiler I had trouble. Alternatively on monday I can send a binary distribution of libol-0.3.13 for hpux 11.0 to you if you want... Please let me know. Stephan Dr. Stephan Hendl Systemmanagement ----------------------------------- Landesbetrieb für Datenverarbeitung und Statistik Land Brandenburg Adresse: 14467 Potsdam, Dortustr. 46 Telefon: +49-(0)331 39-471 Fax: +49-(0)331 27548-1187 Mobil: +49-(0)160 90 645 893 EMail: Stephan.Hendl(a)lds.brandenburg.de Internet: http://www.lds-bb.de >>> nme2nme(a)hotmail.com 03.07.2004 00:52:35 >>> Hi, i'm new to this...and when i hit the "make" command for libol i get the following error: (Bundled) cc: "format.c", line 306: error 1549: Modifiable lvalue required for a ssignment operator. do you know why? and what is the work around? i have libol-0.3.13. thanks!! al _________________________________________________________________ FREE pop-up blocking with the new MSN Toolbar * get it now! http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/ _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

Backup Logfiles to specific folder of a remote machine
by amy wong 02 Jul '04

02 Jul '04

Hi all, I have recently setup a central syslog-ng server to receives log from cisco router and other server. I am intend to store a backup copy of this log files in a specific folder of a remote machine. Would anyone show me how can I do that? thanks amy

1 0

Backup Logfiles to specific folder of a remote machine
by amy wong 02 Jul '04

02 Jul '04

Hi all, I have recently setup a central syslog-ng server to receives log from cisco router and other server. I am intend to store a backup copy of this log files in a specific folder of a remote machine. Would anyone show me how can I do that? thanks, amy

1 0

Re: [syslog-ng]problem on HPUX with s_tcp snd d_tcp
by Stephan Hendl 02 Jul '04

02 Jul '04

I've just upgraded all version to 1.6.4 and the problem is the same. After a kill -HUP on the hpux-server the clients are going into CLOSE_WAIT state where the server is in the FIN_WAIT_2 state client: tcp 1 0 <client>:55682 <server>:10514 CLOSE_WAIT server tcp 0 0 <server>.10514 <client>.55682 FIN_WAIT_2 After a 'service syslog-ng restart' on all clients they open a new connection to the server and the connection will be established as I expect. I hope that helps.. Stephan

2 1

Re: [syslog-ng]Can't get pipe working
by Stephan Hendl 02 Jul '04

02 Jul '04

Hi Clayton, wie use ist in slyghtly other way and it writes several gigabytes per dy in our mysql database. 1) the syslog-ng configuration destination d_ToSysLogDB { pipe("/tmp/syslog.pipe" owner("root") group("root") perm(0640) template("insert into syslog(host,facility,priority,level,tag,date,time,progra m,msg) values ('$HOST','$FACILITY', '$PRIORITY', '$LEVEL', '$TAG','$YEAR-$MONTH- $DAY', '$HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)); }; log { source(loghost); destination(d_ToSysLogDB); }; 2) there is a script running that takes the data from the pipe and writes them into the databse #!/bin/bash # if [ -e /tmp/syslog.pipe ]; then while [ -e /tmp/syslog.pipe ] do mysql -u syslog --password=<password> syslog -h syslog < /tmp/syslog.pipe done else mkfifo /tmp/syslog.pipe while [ -e /tmp/syslog.pipe ] do mysql -u syslog --password=<password> -h syslog < /tmp/syslog.pipe done fi 3) Keep in mind that a tail -f on a pipe will take all data out of the pipe and you cannot write them into a database. Stephan Dr. Stephan Hendl Systemmanagement ----------------------------------- Landesbetrieb für Datenverarbeitung und Statistik Land Brandenburg Adresse: 14467 Potsdam, Dortustr. 46 Telefon: +49-(0)331 39-471 Fax: +49-(0)331 27548-1187 Mobil: +49-(0)160 90 645 893 EMail: Stephan.Hendl(a)lds.brandenburg.de Internet: http://www.lds-bb.de >>> cdukes(a)cdukes.com 01.07.2004 20:02:57 >>> Hi all, I seem to be having trouble getting a mysql pipe working and thought I would enlist your help. Here's my config: options { use_fqdn(no); keep_hostname(yes); use_dns(no); long_hostnames(off); sync(0); log_fifo_size(1000); # The default action of syslog-ng 1.6.0 is to log a STATS line # to the file every 10 minutes. That's pretty ugly after a while. # Change it to every 12 hours so you get a nice daily update of # how many messages syslog-ng missed (0). stats(43200); }; ############################ # Set up Sources ############################ source src { unix-dgram("/dev/log"); internal(); }; source net { udp(); }; source s_jffnms { unix-dgram("/dev/log"); internal(); udp(); }; # MySQL Logs destination d_jffnms { pipe("/tmp/mysql.pipe" template("INSERT INTO syslog (date, date_logged, host, message) VALUES ('$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', NOW, '$FULLHOST', '$MSG');\n") template-escape(yes)); }; destination jffnms_processing { program ("mysql -u jffnms -pjffnms jffnms < /tmp/mysql.pipe"); }; log { source (s_jffnms); #filter (f_jffnms); #if you use it destination (d_jffnms); }; # End When I restart syslog-ng and tail -f /tmp/mysql.pipe nothing comes in (nor are there any database inserts happening). When I do a tcpdump, I do see the event coming into my server, but it seems as though syslog-ng is just dropping the packet...? Regards, Clayton Dukes CCNA, CCDA, CCNP, CCDP Sr. Network Engineer E Solutions Corp. http://www.esnet.com 813.301.2620 (o) 813.545.7373 (c) _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

1 0

Syslog-Mysql-error ( ATTN: Asif Iqbal)
by Venkata Phani Kiran Achanta 01 Jul '04

01 Jul '04

Greetings all, Everything was going well until I got this error while loading the Mysql tables.... DBD::mysql::db do failed: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near '192.168.254.250', 'auth', 'notice', 'notice', '25', '2004-07-01 at ./db-loader.pl line 19, <FH> line 1. And my line 1 in mysql.pipe looks like this INSERT INTO logs (host, facility, priority, level, tag, date, time, program, msg) VALUES ( '192.168.254.250', 'auth', 'notice', 'notice', '25', '2004-07-01', '15:26:28', 'DNS_NS25','DNS_NS25: NetScreen device_id=DNS_NS25 [No Name]system-notification-00257(traffic): start_time=\"2004-07-01 15:30:26\" duration=2 policy_id=0 service=dns proto=17 src zone=Trust dst zone=Untrust action=Permit sent=99 rcvd=274 src=192.168.254.2 dst=63.209.70.231 src_port=1116 dst_port=53 src-xlated ip=12.28.143.203 port=1116 Any ideas....let me know if you need more info to get me out of this error. Thanks in advance. I have searched thru the archives and found Mr.Asif Iqbal had a similar problem but unfortunately he did not find an answer in this list. Hey Asif, If you are still subscribing to this list and see this message please respond ASAP explaining how you worked around this problem. Thanks much, Venkata Achanta --Open source Rocks.........

1 0

Can't get pipe working
by Clayton Dukes 01 Jul '04

01 Jul '04

Hi all, I seem to be having trouble getting a mysql pipe working and thought I would enlist your help. Here's my config: options { use_fqdn(no); keep_hostname(yes); use_dns(no); long_hostnames(off); sync(0); log_fifo_size(1000); # The default action of syslog-ng 1.6.0 is to log a STATS line # to the file every 10 minutes. That's pretty ugly after a while. # Change it to every 12 hours so you get a nice daily update of # how many messages syslog-ng missed (0). stats(43200); }; ############################ # Set up Sources ############################ source src { unix-dgram("/dev/log"); internal(); }; source net { udp(); }; source s_jffnms { unix-dgram("/dev/log"); internal(); udp(); }; # MySQL Logs destination d_jffnms { pipe("/tmp/mysql.pipe" template("INSERT INTO syslog (date, date_logged, host, message) VALUES ('$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', NOW, '$FULLHOST', '$MSG');\n") template-escape(yes)); }; destination jffnms_processing { program ("mysql -u jffnms -pjffnms jffnms < /tmp/mysql.pipe"); }; log { source (s_jffnms); #filter (f_jffnms); #if you use it destination (d_jffnms); }; # End When I restart syslog-ng and tail -f /tmp/mysql.pipe nothing comes in (nor are there any database inserts happening). When I do a tcpdump, I do see the event coming into my server, but it seems as though syslog-ng is just dropping the packet...? Regards, Clayton Dukes CCNA, CCDA, CCNP, CCDP Sr. Network Engineer E Solutions Corp. http://www.esnet.com 813.301.2620 (o) 813.545.7373 (c)

1 0