October 2005 - syslog-ng - lists.balabit.hu

Fwd: [syslog-ng] spoof_source not working
by Chance Ellis 22 Oct '05

22 Oct '05

Ok, I have tried everything you have given men and I am still coming up with nothing... Here is the output of my ldd: ldd /usr/local/sbin/syslog-ng libpthread.so.1 => /usr/lib/libpthread.so.1 libnsl.so.1 => /usr/lib/libnsl.so.1 libsocket.so.1 => /usr/lib/libsocket.so.1 libdoor.so.1 => /usr/lib/libdoor.so.1 libresolv.so.2 => /usr/lib/libresolv.so.2 libxnet.so.1 => /usr/lib/libxnet.so.1 libc.so.1 => /usr/lib/libc.so.1 libdl.so.1 => /usr/lib/libdl.so.1 libmp.so.2 => /usr/lib/libmp.so.2 libthread.so.1 => /usr/lib/libthread.so.1 /usr/platform/SUNW,Sun-Fire-V210/lib/libc_psr.so.1 Notice libxnet. Should I be using it? Can it be causing the problem? How do I tell syslog-ng at compile time to use libnet.a or libnet.so? On 10/14/05, Nate Campi <nate(a)campin.net> wrote: > > Ellis, > > I didn't mention in my email to you that you'll want the libnet.so file > in /tmp/foo - but first make sure you're dynamically liked using the ldd > command: > > -0-[root@duo:masterfiles]# ldd /sbin/syslog-ng > libnsl.so.1 => /lib/libnsl.so.1 (0x40028000) > libresolv.so.2 => /lib/libresolv.so.2 (0x4003c000) > libc.so.6 => /lib/libc.so.6 (0x4004e000) > /lib/ld-linux.so.2 (0x40000000) > > You should see libnet.so in there somewhere if you're dynamically > linked. If you're statically linked then you have to narrow it down with > truss as Bazsi has said. > > Good luck, > Nate > > On Fri, Oct 14, 2005 at 05:35:13PM +0200, Balazs Scheidler wrote: > > On Thu, 2005-10-13 at 15:57 -0400, Chance Ellis wrote: > > > Ok, > > > > > > I copied the the Solaris 8 libnet-config file to the /tmp/foo folder > > > on Solaris 9. I then ran: > > > > > > LD_LIBRARY_PATH=/tmp/foo:$LD_LIBRARY_PATH > > > truss /usr/local/sbin/syslog-ng -f /usr/local/etc/syslog-ng.conf -F & > > > > > > > > > I get the same result... Whenever I apply the spoof_source(yes) to the > > > config I do not get any messages forwarded to the destination. If I > > > remove the spoof_source(yes) messages flow but with the source IP > > > address from the syslog-ng server... > > > > > > The truss output is quite huge! Is there any piece of the truss output > > > that would help me to troubleshoot this? Is libnet-config the only > > > thing I need or do I need something in addition to libnet-config? > > > > libnet-config is not used at all while running syslog-ng, it is a > > build-time script that is invoked to query header location and linking > > information about the libnet library. > > > > libnet is usually linked statically, so there's no point in setting > > LD_LIBRARY_PATH either, you can check which one syslog-ng is using by > > using "ldd" and/or checking if you have a libnet.a file (static), or > > libnet.so file (dynamic), or both. If you have both, the linking > > parameters determine which one is used by syslog-ng, in this case ldd > > should tell you the truth. > > > > To analyze the truss path, you should look for the pattern of the > > message that should be sent out with a spoofed source address, and then > > you should see system calls like recvmsg() when the message is received > > and either send() or write() when it is sent. libnet uses raw sockets to > > send messages, syslog-ng is probably opening AF_INET, SOCK_RAW sockets > > in that case. > > > > Although Solaris uses socket emulation and thus the actual system calls > > you see in the truss output might not be the same as in Linux, this is > > pretty straightforward. By the way, it might be easier to analyze the > > truss output if you are running a syslog-ng instance which does not > > actually deliver a lot of messages, but only a single one, this way the > > truss output will not be so large. > > -- > Nate > > "Often, when I am reading a good book, I stop and thank my teacher. That > is, I used to, until she got an unlisted number." - Unknown 15-year-old > > >

3 4

Queueing of messages if remote loghost is down
by Eric Brown 22 Oct '05

22 Oct '05

I'd like to centralize logs from 2-data centers to my corporate offices via tcp destinations at my data centers and a tcp source at my corporate office. What if my data centers suffer network connectivity problems or I loose power at my corporate office? How long will syslog-ng keep queueing messages before I start loosing data? Will its attempts to retry connecting to my corporate office adversely affect overall system performance of the machine it is running on? Will it recover on its own after a few hours of lost connectivity? Thanks, Eric

1 0

Fwd: Re: [syslog-ng] I need to add more facilities
by Yan M. 22 Oct '05

22 Oct '05

On Fri, 2005-10-21 at 01:03 -0500, Kevin wrote:> On 10/20/05, Yan M. <yannnick_m(a)yahoo.com> wrote:> > searching through the documentation, FAQ, README> > and INSTALL I didnt find anywhere mentionned that I can> > create new facilities.> >> > I read that syslog-ng has the same facilities than syslog in Solaris.> > Technically, syslog-ng has the same facilities as all> standards-compliant syslog daemons, because the facility range is> defined in RFC 3164> > There are only 24 legal values for the facility, and eight legal> values for severity.> > > What I want to do is to have more facilities, with significative names> > like httpd0, httpd1, httpd2, httpd3.....and so on to httpd50.> >> > This is to split message received from my apache VirtualHosts AccessLog and> > ErrorLog> > How would you cause Apache to generate "syslog" messages with> non-standards-conformant facility values, corresponding to your> virtual hosts?> > > > I dont want to filter using regexp, only with facility> > One ugly hack to accomplish what you ask would be to use the standard> facilities and severities to indicate which virtual host sourced the> traffic. So "kern.debug" would be httpd0, kern.info would be httpd1,> and local7.emergency would be httpd191> > It might be easier to just use regex.syslog-ng 2.x allows specifying facility values by number, even stilldifficult to generate non-compliant facility messages in the firstplace.The reason why the facility based filtering failed in the first placewas that no appropriate processes were created to allocate new numbers.-- Bazsi_______________________________________________syslog-ng maillist - syslog-ng@lists.balabit.huhttps://lists.balabit.hu/mailman/listinfo/syslog-ngFrequently asked questions at http://www.campin.net/syslog-ng/faq.html Rethinking about it, I found another way to do it. It is under Solaris8 so I can use pipes, apache ErrorLog and AccessLog can write to pipes. Apache runs in a chroot jail, and ErrorLog's/AccessLog's of all VirtualHosts write to their own unique pipe in the jail. Syslog-ng runs not chrooted and can read the pipes so I defines a source for each pipes in the config I can split logs from apache that way. Thanks for your replies Yan --------------------------------- Yahoo! FareChase - Search multiple travel sites in one click.

1 0

new list: syslog-ng-commit
by Balazs Scheidler 21 Oct '05

21 Oct '05

Hi, For those interested I've created a new list named syslog-ng-commit where information about the commits to my internal syslog-ng 2.0 tree will be published. To subcribe go to https://lists.balabit.hu/mailman/listinfo/syslog-ng-commit/ The number of commits is usually low, you don't need to expect more than 1-5 commits weekly. I hope this list can help you to track the progress towards syslog-ng 2.x. Each message lists commit meta information, changelog entry and a diffstat on the commit. For now the patch will not be available, I'm going to publish my internal GNU arch tree if requested. -- Bazsi

1 0

I need to add more facilities
by Yan M. 21 Oct '05

21 Oct '05

Hi list, searching through the documentation, FAQ, README and INSTALL I didnt find anywhere mentionned that I can create new facilities. I read that syslog-ng has the same facilities than syslog in Solaris. What I want to do is to have more facilities, with significative names like httpd0, httpd1, httpd2, httpd3.....and so on to httpd50. This is to split message received from my apache VirtualHosts AccessLog and ErrorLog I dont want to filter using regexp, only with facility Thank you Yan --------------------------------- Yahoo! FareChase - Search multiple travel sites in one click.

3 2

Unix-stream destination as listen socket
by Peter Nahas 20 Oct '05

20 Oct '05

Hello everyone, I am currently working on a project using syslog-ng-1.9.5 (soon to upgrade to 1.9.6) which requires a syslog-ng destination to be a Unix socket. The problem is that the syslog-ng daemon will always be started before the destination application. So to prevent the loss of messages due to the reconnect timer, the syslog-ng destination Unix socket will have to listen()/accept() for a connection rather than connect() to an external socket. If this enhancement would be used by others, I would like to write this properly. Therefore I have the following questions: A) If this enhancement were written, would it be accepted into the repository? If not, I will simply hack up the current AFUnixDestinationDriver to do what I want. B) Is there any specification document which would assist me in making this enhancement? C) Would this best be done by adding an additional option to unix-stream for destinations or by creating a separate destination driver? D) It appears as if the AFSocketDestinationDriver is not designed to perform listen()/accept(), but the AFSocketSourceDriver is. Would it be possible to set up the AFUnixDestinationDriver to use the AFSocketSourceDriver as a parent, but attach it to a log writer? Thanks in advance, Peter Nahas Software Engineer MRV Communications - InReach division

3 4

Filter based on source?
by Robert.Becker＠motoristsgroup.com 20 Oct '05

20 Oct '05

As with many people I am using syslog-ng to send all my logs to a central source. A bunch of the logs I am sending are from growing log files. I have everything working and quite well, but I have one problem. When syslog-ng builts the log statement it goes in the following format: Oct 20 09:59:24 server_pipe@linuxp17 [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 server_pipe is the name of the source, defined on the client syslog server, this log is coming from. When this log is piped via tcp to the central server I want to be able to filter the server_pipe part and send it to a specific destination. Can this be done? From what I've read the match only works on the message section. I need to filter based the source. Any ideas? Thanks a bunch. -Rob Becker ********************************************************************** The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error or there are any problems please notify the originator immediately. The unauthorized use, disclosure, copying or alteration of this message is strictly forbidden. Motorists Insurance Group will not be liable for direct, special, indirect or consequential damages arising from the alteration of the contents of this message by a third party or as a result of any virus being passed on. **********************************************************************

2 1

Problem with setup of syslog-ng loghost in HA environment
by Berthold Cogel 19 Oct '05

19 Oct '05

Hello! We want to set up a syslog-ng in a HA Cluster (Red Hat Clumanager). The Cluster has two nodes for failover of services with attached shared storage from our SAN. The idea was to use syslog-ng only for the loghost. Local logging on all machines is done by standard syslog and forwarded to the loghost. The loghost uses his own IP which is handed over from one node to the other in case of a failover situation. It is not a problem to get the loghost running. The problem is the logging on the cluster node that is the current loghost. Although syslog doesn't log remote messages and syslog-ng binds to another (virtual) interface. We have an old loghost with syslog, where our systems send parts of their messages. This works, even for the system, syslog-ng is running on. I can see the udp packets with ethereal. But there is no syslog traffic to the second IP or localhost. Only 'external' messages get logged with syslog-ng The system is Red Hat Enterprise Linux 3. syslog-ng is 1.6.8. Thanks in advance Berthold Cogel

2 1

syslog server
by John.Dickinson＠nominet.org.uk 19 Oct '05

19 Oct '05

I have a central syslog server receiving log messages from clients. I want to store the messages on the clients and the server (with different rotation/archiving setting). From what I have seen I will need to have all the syslog configuration on both the client and the server. Is there anyway to simplify this? As an example: I have something like this on the client (along with other filters and destinations) filter f_emerg { level (emerg); }; filter f_alert { level (alert .. emerg); }; filter f_crit { level (crit .. emerg); }; filter f_err { level (err .. emerg); }; filter f_warning { level (warning .. emerg); }; filter f_notice { level (notice .. emerg); }; filter f_info { level (info .. emerg); }; filter f_debug { level (debug .. emerg); }; filter f_kern { facility (kern); }; filter f_user { facility (user); }; filter f_mail { facility (mail); }; filter f_daemon { facility (daemon); }; filter f_auth { facility (auth); }; filter f_syslog { facility (syslog); }; filter f_lpr { facility (lpr); }; filter f_news { facility (news); }; filter f_uucp { facility (uucp); }; filter f_cron { facility (cron); }; filter f_local0 { facility (local0); }; filter f_local1 { facility (local1); }; filter f_local2 { facility (local2); }; filter f_local3 { facility (local3); }; filter f_local4 { facility (local4); }; filter f_local5 { facility (local5); }; filter f_local6 { facility (local6); }; filter f_local7 { facility (local7); }; filter f_kern_debug { filter (f_kern) and filter (f_debug); }; filter f_daemon_notice { filter (f_daemon) and filter (f_notice); }; filter f_mail_crit { filter (f_mail) and filter (f_crit); }; filter f_syslog { filter (f_err) or filter (f_kern_debug) or filter (f_daemon_notice) or filter (f_mail_crit); }; destination d_syslog { file ("/var/log/syslog"); }; destination d_log_server { tcp ("127.0.0.1" port (514) tcp-keep-alive(yes) log_fifo_size(10000)); }; log { source (s_local); filter (f_syslog); destination (d_syslog); }; log { source (s_local); filter (f_syslog); destination (d_log_server); }; On the server I want to get all the messages in a file with the same name as they are in on the client. Do I have to repeat all these filters again on the server or is there a way to find what log statement they matched on the client? Basically, I dont want to be repeating the config everywhere because it will become a maintenance nightmare. John

1 0

clustering syslog-ng
by John.Dickinson＠nominet.org.uk 19 Oct '05

19 Oct '05

Hi, I am trying to setup a clustered syslog server using polyserve, syslog-ng (1.6.8) and stunnel. Log messages are sent using TCP for reliability. I am testing the setup by generating 10 messages per second on a client. The messages are numbered so that I can see if any are missing. Logging is working but when I force the syslog service to fail over to a different node in the cluster I lose messages. From what I have read on these lists I would expect syslog-ng 1.6.8 to lose one message when it finds that the first server has failed but should queue the rest until the service returns on the other node. However I am losing around 200 messages. This part of the client config destination d_log_server { tcp ("127.0.0.1" port (514) tcp-keep-alive(yes) log_fifo_size(10000)); }; log { source (s_local); filter (f_local7_info); destination (d_log_server); }; and the server options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_ext { tcp(ip(127.0.0.1) port(514) tcp-keep-alive(yes)); }; destination d_dev1 { file("/logs/dev1.log"); }; log { source(s_ext); destination(d_dev1); }; This stunnel config is the same as the example on the stunnel web site. Any ideas? I know it is only beta but would I be better off using syslog-ng 1.9? I realise that I could do this without the clustering and just send all the messages to both servers but this way I have no duplication of logs. Thanks John

1 0