- syslog-ng - lists.balabit.hu

Extra characters at beginning of line
by Florian Hines 07 Jan '09

07 Jan '09

Hi Everyone, I'm running into an issue where syslog-ng is adding extra characters to beginning of every line. Specifically, "<134>" is getting inserted right before the time stamp: <134>Jan 7 13:06:17 host1 kernel: device eth0 entered promiscuous mode This syslog-ng server is sending traffic to a remote Splunk instance (using TCP, not UDP), at first I though it was Splunk adding the characters but when I did a tcpdump on syslog-ng's outbound connection I found that they where already present. In addition to sending this traffic to Splunk the syslog-ng instance also log's local to a file. The <134> doesn't show up in the local file. Anyone have any ideas where this is coming from ? Thanks! Florian

2 2

Issues with SYSLOG-NG 2.0.10 and relaying via a SSH tunnel
by Dean Thompson 07 Jan '09

07 Jan '09

Hi Everybody, I was wondering whether somebody might be able to help me with a problem which appears to have been discussed before but is still continuing to be a problem for an environment that I am working within. The issue relates to one host forwarding SYSLOG messages to another host via a SSH tunnel. Scenario: ========= <Device> --SYSLOG MESSAGE--> Machine A <--SSH Tunnel--> Machine B Items to note include Machine B setting up a reverse SSH tunnel with Machine A. Machine A has a local port of 5514 that it uses to forward syslog messages to which sents them to machine B. This connectivity is working without any problems. The issue that I am currently experiencing is on Machine B when looking at the logs delivered via Machine A. It would appear that Machine B thinks that all the logs are coming from the IP address of 127.0.0.1. I have checked the configuration of syslog-ng on machine A and machine B and they look as if they are configured correctly. Syslog-NG 2.0.10 on Machine A is configured as follows: ======================================================= options { log_fifo_size(1000); long_hostnames(off); use_dns(no); use_fqdn(no); keep_hostname(yes); check_hostname(yes); log_msg_size(16384); }; source s_network { udp(ip(0.0.0.0) port(514)); tcp(ip(0.0.0.0) port(514) max_connections(1000)); }; destination d_syslogmgr { tcp("127.0.0.1" port(5514)); }; [...] Syslog-NG 2.0.10 on Machine B is configured as follows: ======================================================= options { log_fifo_size(1000); long_hostnames(off); use_dns(no); use_fqdn(no); keep_hostname(yes); check_hostname(yes); log_msg_size(16384); }; [...] Output that is being observed: ============================== Jan 7 12:32:34 127.0.0.1 172.XX.YY.1 3317162: 3339552: Jan 7 12:37:09.608: %LINK-3-UPDOWN: Interface FastEthernet0/7, changed state to up Jan 7 12:32:34 127.0.0.1 172.XX.YY.1 3317163: 3339553: Jan 7 12:37:10.457: %LINK-3-UPDOWN: Interface FastEthernet0/7, changed state to down Jan 7 12:32:34 127.0.0.1 172.XX.YY.1 23:37:07 accept 172.XX.YY.242 > >eth-s2p2c3 src: 192.168.17.48; dst: 192.168.19.167; proto: tcp; rule: 22; product: VPN-1 & FireWall-1; service: 9999; s_port: 3310; Jan 7 12:32:34 127.0.0.1 172.XX.YY.1 23:37:07 accept 172.XX.YY.242 > >eth-s2p4c1 src: XXX.YYY.235.195; dst: 192.168.108.1; proto: udp; rule: 5; product: VPN-1 & FireWall-1; service: 53; s_port: 5707; Jan 7 12:32:34 127.0.0.1 172.XX.YY.1 23:37:07 accept 172.XX.YY.242 > >eth-s2p4c1 src: XXX.YYY.235.197; dst: 192.168.108.1; proto: udp; rule: 5; product: VPN-1 & FireWall-1; service: 53; s_port: 11639; Jan 7 12:32:34 127.0.0.1 172.XX.YY.1 23:37:08 accept 172.XX.YY.242 > >eth-s2p4c1 src: XXX.YYY.235.196; dst: 192.168.108.1; proto: udp; rule: 5; product: VPN-1 & FireWall-1; service: 53; s_port: 1325; Jan 7 12:32:34 127.0.0.1 172.XX.YY.1 8023816: Jan 7 23:37:09.028: %SYS-CFC8-2-MMAP: mmap failed for size 24576 bytes Caller PC 0x72B12BCC errno 12 : ios-base : (PID=20499, TID=23) : -Traceback=(c6lc2-sp-4-dso-b.so+0x1F9AD0) ([19:0]+0x204790) ([19:0]+0x20CEDC) ([16:-2]np.+0x84BD4) ([21:0]+0x855B8) ([9:-3]1-dso-b+0x1FE904) ([9:-3]0-dso-bn+0xD354C) ([20:0]+0xD2658) ([20:0]+0xD29D0) ([9:-3]2-dso-b+0x4E4684) ([19:0]+0x4E465C) Ideally, I would prefer that the 127.0.0.1 was actually the 172.XX.YY.1 IP address, but it appears that there is a re-write of IP addresses going on. I have placed a packet sniffer on the line and I can see that the packet which is actually sent to MACHINE B is correct (including the correct IP addresses). To me it would appear that the issue is in the processing of the message at MACHINE B, but the issue may lie elsewhere. Is there anything obvious with the configurations which would be contributing to this problem, or is my basic understanding of how SYSLOG-NG works when wanting to keep hostnames and IP addresses flawed ? Regards, Dean

3 2

Solaris Client Not sending to Central Log Host
by Zeek Anow 05 Jan '09

05 Jan '09

We are running Solaris 9 on various Sun servers. I need to be able to send logs from the clients to the Central loghost using UDP. Below is a partial Client config file. Note that I don't have a Filter for this. If that is the problem, what kind of filter would I need to send all logs? Any help or suggestions would be appreciated. # GLOBAL OPTIONS ########################################################### options { keep_hostname(yes); chain_hostnames(no); create_dirs(yes); use_time_recvd(yes); use_dns(no); use_fqdn(no); }; # Source source local { sun-streams("/dev/log" door("/etc/.syslog_door")); internal(); }; destination d_centralhost { udp("10.5.2.3") port(514); }; log { source(local); destination (d_centralhost); }; ################################################################## Thanks, Zeek

3 4

underscore issue
by Stefan Varga 05 Jan '09

05 Jan '09

Hi all, I'm using syslog-ng 2.0.9 as a central logging server with destination like this: destination r_messages { file ("/logpool/logs/$DAY$MONTH$YEAR/$HOST_$DAY$MONTH$YEAR$_$HOUR"); }; But I get this: #ls /logpool/logs/05012009 0501200910 HOST is not expanded and _ before $HOUR is missing . What is special about underscore ? While using destination r_messages { file ("/logpool/logs/$DAY$MONTH$YEAR/$HOST-$DAY$MONTH$YEAR$-$HOUR"); }; everything is fine: # ls /logpool/logs/05012009/ 10.96.47.233-05012009-10 I would prefer to use an underscore if I could. Thank you, Stefan

2 3

importing old syslog-ng messages
by Daniel L. Spells Sr. 05 Jan '09

05 Jan '09

Sirs, Is there a way to import /var/log/messages, created by syslog-ng, into syslog-ng to be placed into a database? Daniel

5 11

Syslog-NG, log4j, tabs, whitespace, indent
by John Skopis 31 Dec '08

31 Dec '08

Hello, I believe that the log4j SyslogAppender is sending logs to syslog-ng that begin with a '\t' (or four spaces). The leading whitespace is trimmed either because it's not allowed per the syslog RFC, or for some other (hopefully) good reason. Can I turn this behavior off? an example would be a message: Dec 10 16:17:52 testbox at java.lang.Thread.run(Thread.java:619) which should be Dec 10 16:17:52 testbox \tat java.lang.Thread.run(Thread.java:619) Thanks, John Skopis Systems Administrator

3 7

Behavior when followed files are renamed
by Joe Shaw 30 Dec '08

30 Dec '08

Hi, I couldn't find this anywhere in the online docs. What's the behavior of syslog-ng OSE when a file which is being followed with follow_freq() is renamed? Thanks, Joe P.S. From a git checkout of 3.0, doc/reference doesn't exist... is it gone, or does it get pulled from somewhere else when a tarball is built?

4 7

Problem compiling syslog-ng with enable-spoof-source on AIX 5.2
by CentralRegion IPAdmin 20 Dec '08

20 Dec '08

Hi, I'm trying to compile syslog-ng.2.1.3 after configuring with --enable-spoof-source. libnet is a prereq as I have learned, but I can't find any hint how to install libnet. Latest version of libnet is "0.10.11: released 18/02/2003" and it seems there is no mailinglist available. The code comes with some makefile templates such as "beos.mak, djgpp.mak, freebsd.mak, linux.mak, mingw.mak, rsxntdj.mak", but none of them works with AIX I have used "GNU Make 3.81" and "gcc version 3.3.2" to compile libnet. All are failing. Do you have any ideas how to get it done? Thank You! Werner

3 3

Performance issues
by farouk.s.khawaja＠jpmorgan.com 18 Dec '08

18 Dec '08

I'm finding that my Syslog-NG 2.0.9 OSE is not able to send more than 19 msgs/sec. I've tried setups with/without flow control, but keep coming up with similar performance results. I'm using the 'file' driver to read the data from application log files. I supposed I could use a named pipe, but I rather not create a new tool to write data to the pipe and create other bottlnecks and buffering issues. The application logs I want to follow writes at a pace of ~130msgs/sec during peak usage with average message size of ~410bytes. My destination log location is on NFS. I've verified that it's not the network or NFS that's causing the issues. I whipped up a simple perl script that made a socket connection from the source host to the destination host where syslog-ng was listening. I was able to pump 48320 messages to that remote syslog port that was configured to write the data to a NFS partition. That's ~3200 msgs/sec. So I also know that the destination syslog-ng is working fine. My guess is that the problem lies with the file driver. I'd like to get syslog-ng to be the tool I use to consolidate logs across N number of hosts instead of writing some bootleg code of my own to which I will forever be a slave to maintain. Any assistance you can provide would be appreciated. Source syslog-ng.conf ########################################################## # G L O B A L O P T I O N S ########################################################## options { use_dns (no); use_fqdn (no); create_dirs (yes); log_fifo_size(2100); time_sleep (10); }; ########################################################## # Template ########################################################## template t_clog { template("$MSG\n"); template_escape(no); }; ########################################################## # Sources ########################################################## # ## default source for logging syslog messages source s_internal { internal(); }; # ## source s_ldsol01_file { file("/tmp/ldsol01-clog.log" follow_freq(10) flags(no-parse) log_fetch_limit(1000) log_iw_size(2000) ); }; ########################################################## # Destinations ########################################################## # ## default destination for logging syslog messages destination d_syslog_file { file("/home/csh_apps/twataprd/Logs/SYSLOG-NG/syslog-ng.log"); }; # ## eqny-cshweb2 destination d_ldsol01_tcp { tcp( "10.3.22.43" port(20534) ); }; destination d_ldsol01rej_tcp { tcp("10.3.22.43" port(20535)); }; ########################################################## # logs ########################################################## log { source( s_internal ); destination ( d_syslog_file ); }; log { source( s_ldsol01_file ); destination( d_ldsol01_tcp ); flags(flow-control); }; Generally, this communication is for informational purposes only and it is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. In the event you are receiving the offering materials attached below related to your interest in hedge funds or private equity, this communication may be intended as an offer or solicitation for the purchase or sale of such fund(s). All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates. This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to UK legal entities.

3 2

Ronald Jones is out of the office.
by Ronald Jones 18 Dec '08

18 Dec '08

I will be out of the office starting 12/18/2008 and will not return until 12/29/2008. I will respond to your message when I return. ----------------------------------------- ________________________________________________________ DTCC DISCLAIMER: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify us immediately and delete the email and any attachments from your system. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.

1 0