- syslog-ng - lists.balabit.hu

Problem using tags with syslog-ng 3.1
by Hoenig, Stefan, VF-Group 04 Apr '10

04 Apr '10

Hi all, I got a problem to get the "tags" feature working on our syslog-ng 3.1. I want to collect messages from 3 different files on the source system and want to separate them again on the central logging server. The client configuration looks like this: ------------------------------------------------------------------------ ---------------------------- source s_app { file("/var/log/log1.log"); file("/opt//log/log2.log" tags("log2")); file("/opt/log/log3.log" tags("log3")); }; options { }; destination d_app { tcp("logrelay01.domain.com" port(13074)); }; log { source(s_app); destination(d_app); }; ------------------------------------------------------------------------ ---------------------------- The log relay does nothing than forward the messages to the central logging server with the following config: ------------------------------------------------------------------------ ---------------------------- options { time_sleep(20); log_fifo_size(1000); dns_cache(2000); dns_cache_expire(87600); keep_hostname(yes); }; source s_remote { tcp(ip("0.0.0.0") port(13074)); }; destination remote_tcp { tcp("centrallog01.domain.com" port(13074)); }; log { source(s_remote); destination(remote_tcp); }; ------------------------------------------------------------------------ ---------------------------- On the central logging server I use filters to separate the logfiles again: ------------------------------------------------------------------------ ---------------------------- @version: 3.0 include "/opt/config/syslogng-inc.conf"; options { time_sleep(20); dns_cache(2000); dns_cache_expire(87600); keep_hostname(yes); create_dirs(yes); }; source s_remote { tcp (ip("0.0.0.0") port(13074) keep-alive(yes)); }; ============================================ This is the confoguration in /opt/config/syslogng-inc.conf # Filter filter f_log1 { host("web00(09|10)"); }; filter f_log2 { host("web00(09|10)") and tags("log2"); }; filter f_log3 { host("web00(09|10)") and tags("log3"); }; #Configuration for Destinations destination d_log1 { file("/var/logs/log1/combined.log" perm(0755) dir_perm(0755)); }; destination d_log2 { file("/var/logs/log2/combined.log" perm(0755) dir_perm(0755)); }; destination d_log3 { file("/var/logs/log3/combined.log" perm(0755) dir_perm(0755)); }; # Logfile log1 log { source(s_remote); filter(f_log1); destination(d_log1); }; # Logfile log2 log { source(s_remote); filter(f_log2); destination(d_log2); }; # Logfile log3 log { source(s_remote); filter(f_log3); destination(d_log3); }; ------------------------------------------------------------------------ ---------------------------- Does anybody have an idea, why it does not work as expected. Thanks for any suggestion and/or idea. Best regards Stefan

6 10

Tailing tomcat catalina.out
by BoOnSiOnG 30 Mar '10

30 Mar '10

Hello all, Good day, I have configure syslog-ng to checks catalina.out every second, but syslog-ng will append date time and machine name to the log, I would like to know is it possible to configure syslog-ng do not alter the original log? my config, source s_tomcat_catalina { file("/srv/tomcat/logs/catalina.out" default-facility(local3) default-priority(info)); }; log output, Mar 29 21:43:38 forward Mar: 29, 2010 9:43:37 PM org.apache.coyote.http11.Http11Protocol start Mar 29 21:43:38 forward INFO: Starting Coyote HTTP/1.1 on http-8080 Mar 29 21:43:38 forward Mar: 29, 2010 9:43:37 PM org.apache.jk.common.ChannelSocket init Mar 29 21:43:38 forward INFO: JK: ajp13 listening on /0.0.0.0:8009 Mar 29 21:43:38 forward Mar: 29, 2010 9:43:37 PM org.apache.jk.server.JkMain start Mar 29 21:43:38 forward INFO: Jk running ID=0 time=0/14 config=null Mar 29 21:43:38 forward Mar: 29, 2010 9:43:37 PM org.apache.catalina.startup.Catalina start Mar 29 21:43:38 forward INFO: Server startup in 428 ms Thanks. :)

3 5

queue management
by Eric Snow 29 Mar '10

29 Mar '10

Hi, I am looking to manage my output buffers. I have one destination that sometimes has trouble and I want to be able to manage the messages that got buffered while I fix my flaky destination. Is there any tool I can use to manage those buffered messages? If not, isn't there any exception handling that I can apply to a log such that errors or other failures for that log can be redirected to another destination? I want to avoid clogging up my output queue and especially to avoid losing any messages. Thanks! -eric This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio, Inc. makes no warranty that this email is error or virus free. Thank you.

2 4

3.0.5 & Cisco TCP problems
by d lists 28 Mar '10

28 Mar '10

Hello, After spending the afternoon trying to get this working, I've decided to reach out for some help (tried google - no luck!). I have what I believe is a very simple syslog-ng configuration where I am trying to capture syslog events from Cisco routers via TCP. I first tested via UDP, and all works good. The router establishes a TCP connection with my syslog-ng (viewable via netstat -nap, and in the /var/log/messages): Mar 25 17:07:53 logger syslog-ng[8943]: Syslog connection accepted; fd='11', client='AF_INET(X.X.X.254:56868)', local='AF_INET(X.X.X.5:2002)' On the router I do a "config term" & "exit" to generate a log. tcpdump on the syslog-ng machine shows traffic as expected: 17:18:44.891080 IP X.X.X.254.61241 > 10.240.0.5.2002: . 3679441367:3679441450(83) ack 1811354960 win 4128 17:18:44.891134 IP X.X.X.5.2002> 10.240.0.254.61241: . ack 83 win 5840 And a strace: poll([{fd=6, events=POLLIN}, {fd=5, events=POLLIN}, {fd=3, events=POLLIN}, {fd=8, events=POLLIN}], 4, 456650) = 1 ([{fd=8, revents=POLLIN}]) gettimeofday({1269559238, 85688}, NULL) = 0 read(8, "<189>145: *Mar 25 23:41:37.033: "..., 7739) = 83 read(8, 0x9f94618, 7656) = -1 EAGAIN (Resource temporarily unavailable) gettimeofday({1269559238, 86032}, NULL) = 0 poll([{fd=6, events=POLLIN}, {fd=5, events=POLLIN}, {fd=3, events=POLLIN}, {fd=8, events=POLLIN}], 4, 449113 But nothing ever shows up in the log files! Here's my syslog-ng.conf. Any pointers or ideas? @version: 3.0 options { use_dns(no); }; source t_net { tcp(ip(X.X.X.5) port(2002) keep-alive(yes) ); }; source s_local { internal(); unix-stream("/dev/log"); file("/proc/kmsg" program_override("kernel: ")); }; destination d_messages { file("/var/log/messages"); }; destination d_cisco { file("/var/log/cisco.log"); }; log { source(t_net); destination(d_messages); }; log { source(s_local); destination(d_messages); }; And for what it is worth the relevant lines from my test cisco: logging host X.X.X.5 transport tcp port 2002 And syslog-ng version: syslog-ng 3.0.5 Installer-Version: 3.0.5 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.0#master#f2fd7457627a88eb9602a73a5c16cd8b2434b667 Compile-Date: Dec 7 2009 14:52:42 Enable-Threads: on Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-Sun-Door: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: off Enable-SSL: on Enable-SQL: on Enable-Linux-Caps: on Enable-Pcre: off Thanks in advance!

4 7

processing stops after config change
by Curt Shaffer 26 Mar '10

26 Mar '10

I have a pretty simple configuration file on my syslog-ng-2.1.3.1 running on CentOS 5.4 64bit and 8GB of RAM. I'm sending logs to 3 destinations. One is a mysql database, one is a flat file, and one is to a log correlation engine. All was running fine and out of nowhere one day, I noticed that syslog-ng kept hanging. I would restart the process and it would be fine for a few seconds then crash. I noticed that when I would start it, it would jump to 100% CPU and stay there until crash. I did some log viewing and straced the process and I noticed that logs were getting to the process, but they never got to their destinations. I started disabling one destination at a time until I found the culprit. The problem was with the destination going to the log correlation via udp 514. If I comment out that source and destination, the system runs like a champ to the other destinations. I'm confused as to what happened, because this was working for at least 3 months solid. I am posting the config here, if anyone can see something or has some suggestions, I would appreciate it. #begin syslog-ng.conf source s_all{ unix-stream("/dev/log"); internal(); udp(ip(0.0.0.0) port(514)); }; destination d_lce { udp("xxx.xxx.xxx.xxx"); } log { source(s_all); destination(d_lce); }; Just to reiterate, I do have two other destinations setup and they run just fine if I comment out this destination and log statement. Every time I uncomment this destination and log statement and restart syslog-ng, I reproduce the problem. Any suggestions would be great, and if you need more info, just ask and I can present it. The first thing I wanted to check is that the config looks sane, though again it worked fine for months. Thanks Curt

2 2

forwarding host only gives name, not IP
by Jan Schaumann 24 Mar '10

24 Mar '10

Hello, I have the following setup: +---+ hostA ---+ V | +---> syslog1 ----> syslog2 hostB ---+ That is, hostA, hostB and syslog1 are all logging to syslog1, syslog1 then takes all the messages and forwards them to syslog2. On syslog2, I'd like to get the IP addresses from each of the three hosts it's receiving messages from. syslog2 has "use_dns(no)" as a global option. When I have "keep_hostname(no)" or "chain_hostnames(no)" on syslog2, I only get syslog1's IP address in my logs, even for messages from hostA or hostB. If I put "keep_hostname(yes)" or "chain_hostnames(yes)" on syslog2, I get the IP addresses from hostA or hostB, but from syslog1 I get the short hostname (ie "syslog1"). If I add "use_fqdn(yes)" to syslog1's config, I get "syslog1.full.domain" on syslog2. However, I'd like for messages from syslog1 to arrive with the IP address, not the hostname on syslog2. I'm sure I'm missing something that'll be obvious in hindsight... just what? Thanks! -Jan

1 0

Question RE: Timezone in filename
by Brian Donaldson 23 Mar '10

23 Mar '10

I've configured syslog-ng to include the timezone in the log filename -- but the value used is "07:00". Any way to get "PDT" instead?

2 1

Spoofing SRC of forwarded syslog
by AM M84 23 Mar '10

23 Mar '10

Hi, I want to know if it is possible to spoof(UDP) the original source IP of a message that has already been forwarded. Server1 (Original Source)--> Server2 (Forwarding Syslog Server )--> Server3 (Server Source Spoofing Server1) --> Final Dest. Thanks, AMM

2 2

facility 10 plus?
by fedora fedora 23 Mar '10

23 Mar '10

Hi, I have some odd devices using syslog facility 24, and there is no way I can change it! In this case, is it possible letting syslog-ng use facility 24? right now it gives me, 24 is not supported error and quits. Thanks

3 3

syslog-ng client messages delayed
by mark peters 22 Mar '10

22 Mar '10

Hi, We are running into a issue where syslog-ng clients who are configured to forward to a central syslog-ng destination server that are reasonably chatty are getting there messages delayed at the source (ie. 'tcpdump'ing the client shows it is sending messages from N minutes ago, where N is anything from 5 to 60 minutes on average sometimes more). We have also seen a small percentage of loss at the destination. On these 'delayed' clients we also see the resident memory footprint on the growing. Clients are running 1.6.12 (have tried 2.0.8 also - same symptons). The central syslog-ng server is running 2.0.8. relevant config entries are: syslog-ng client: options { sync (1); time_reopen (10); log_fifo_size (2097152); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; destination d_central { tcp("log2.xyz.com" port(5140)); }; filter f_local5 { facility(local5); }; log { source(s_sys); filter(f_local5); destination(d_central); flags(final); }; syslog-ng central server: options { time_reopen (10); log_fifo_size (2097152); long_hostnames (off); use_dns (yes); dns_cache (yes); dns_cache_expire (28800); use_fqdn (no); create_dirs (yes); dir_perm (0755); perm (0644); keep_hostname (no); use_time_recvd (no); stats_freq(600); flush_timeout(16000); flush_lines(256); time_sleep(80); }; destination d_server { file("/logs/server.$LEVEL.log"); }; filter f_server { facility(local5) and program("server"); }; log { source(s_sys); filter(f_server); destination(d_server); flags(final); }; to give a rough idea of message volume the server is receiving approximately 50 million log lines per day on average any help/information would be appreciated thanks -------------------------------------- Easy + Joy + Powerful = Yahoo! Bookmarks x Toolbar http://pr.mail.yahoo.co.jp/toolbar/

3 4