- syslog-ng - lists.balabit.hu

patterndb ESTRING delimiter char
by Martin Holste 16 Nov '10

16 Nov '10

I need to find a valid char to use as the ESTRING delimiter that won't appear in a URI. I had been using comma, which usually works, but lots of sites have commas in the URI which messes up the parsing of these logs. Tab is outlawed as per the documentation or I'd have used that. I've seen pipe used in URI's, so that's no better. Keep in mind that what I'm parsing has been URI unescaped. Is there a way to use a null byte or something? Thanks, Martin

2 4

patterndb and syslog from cisco
by Yann Forum 16 Nov '10

16 Nov '10

Hello, I'm writing patterndb.xml files to filter syslog messages from servers and CISCO routers. Currently, CISCO sends syslog with that format: Nov 3 15:36:02 srv01.dom.test 36779: .Nov 3 14:50:30.403: %SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user: user01] [Source: 10.0.0.1] [localport: 22] at 15:50:30 CET Wed Nov 3 2010 Nov 3 15:39:02 srv01.dom.test 36780: .Nov 3 14:53:30.255: %SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user: user01] [Source: 10.0.0.1] [localport: 22] at 15:53:30 CET Wed Nov 3 2010 Nov 3 15:42:01 srv01.dom.test 36781: .Nov 3 14:56:30.378: %SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user: user01] [Source: 10.0.0.1] [localport: 22] at 15:56:30 CET Wed Nov 3 2010 The problem comes from the program name which changes for each message: 36779, 36780, 36781, etc. For this reason, I can't use patterndb mechanism. How may I solve my problem? I think it's not allowed to change the program name with the "rewrite" rule. I have the same problem with switches from Alcatel... Regards, Yann I.

5 5

Configuration tuning for reliability
by Ben Tisdall 16 Nov '10

16 Nov '10

Dear list, Apologies for the long winded post but I'd really appreciate your comments on: 1) A configuration that did not perform adequately 2) My understanding of the relevant tuneables Number of clients: 40 Log sources per client: 35. 34 of these are pure file source driver, system logs use the standard redhat config. Dest: single loghost over tcp/ip, loghost uses mysql driver Due to other pressures I made no attempt to tune parameters that influence reliability before the test and performance was commensurately poor (about 40% of entries discarded at the client, confirmed by packet capture on the loghost). Client global opts: * log_msg_size (24576) * log_fifo_size (1000) * log_fetch_limit (10) * flush_lines(0) Client logging options: * each source does a program-override for loghost filtering, no other processing * flags(flow-control) NOT SET Loghost global opts * log_msg_size(32768) * log_fifo_size(1000) Loghost logging opts: * flags(flow-control) - NOT SET * streams undergoes a rewrite pass a simple filter (on program name), rewrite, parse, then mysql dest - the loghost load has always been negligable. >From what I NOW understand I need to do something like this: Client: log_iw_size >= SOURCES_PER_CLIENT * log_fetch_limit eg 35 * 10 = 350 log_fifo_size >= SOURCES_PER_CLIENT * log_fetch_limit eg 35 * 10 = 350 AND log_fifo_size >= SOURCES_PER_CLIENT * log_iw_size eg 35 * 350 = 12250 So it appears to me that setting log_fifo_size to > 12250 would be correct. Loghost Less idea about this, do I need: log_iw_size >= NUMBER_OF_CLIENTS * log_fetch_limit ( * SOURCES_PER_CLIENT ? ) eg 40 * 10 * 35 = 14000 And log_fifo_size >= log_iw_size ? Is flow control important between the network source and the mysql dests? Further information: * Some of the larger logfiles output ~ 4 lines/sec * Approx 4G aggregate logs generated over 14 hours by the 40 hosts * Reliability is more important than speed - these logs are not analysed in real time. That said, the client should have to spend hours completing the log transfer once its workload has been processed.. Again, apologies for the length of the post and many thanks in advance for any help. Ben Tisdall PhotoBox

4 6

syslog-ng OSE 3.2 Administrator Guide - beta
by Fekete Robert 14 Nov '10

14 Nov '10

Hi, I have released a beta version of the new administrator guide. I have updated it to cover (hopefully) every important change and feature in 3.2beta, but it still needs a review. So if something isn't working as written, is missing, or contains errors, please let me know. You can find a summary of new sections here: http://robert.blogs.balabit.com/2010/11/syslog-ng-open-source-edition-3-2-a… Regards, Robert

2 1

dont-create-tables flag in 3.2 admin documentation
by Andreas Maechler 14 Nov '10

14 Nov '10

Hi all First of all, thank you for another nice syslog-ng documentation. One comment about the new 3.2 guide: Shouldn't the sql() option 'dont-create-tables' be listed as part of the flag() option instead of being listed as a separate option? - Andreas

2 1

Problems using the syslog-protocol flag
by Andreas Maechler 14 Nov '10

14 Nov '10

Hi all Running syslog-ng 3.2beta1 on FreeBSD, I'm trying to use the 'syslog-protocol' flag in a unix-dgram source: source s_local { unix-dgram("/var/run/log" flags(syslog-protocol)); unix-dgram("/var/run/logpriv" perm(0600)); file("/dev/klog" follow-freq(0) program-override("kernel") flags(no-parse)); internal(); }; That option seems to be ignored though. If I force the option by setting it manually in afsocket_sd_init_instance(), afsocket.c, all works well and incoming messages get parsed according to IETF. Am I missing something or is this a bug? Thanks in advance. Andreas

2 1

CSV parser and empty fields
by Bill Anderson 14 Nov '10

14 Nov '10

I'm using a tab separated format from apache for access logs. My last two fields are referrer and user-agent. Obviously sometimes there is no referrer. Unfortunately when there isn't one apache only logs an empty string instead of the more common "-". This isn't a problem in scripts that parse the resulting logfile as they see the resulting empty field when I log $MSG. However, I just started a new log file that uses the csv-parser w/tab as delimiter and when the referrer field is empty, APACHE.USERAGENT (the last field) gets rolled into APACHE.REFERRER, the second to last field. As a result the template for this page (which uses APACHE.REFERRER) isn't reliable. When REFERRER is empty I want it to be empty (or something I can specify, like a default) not he next field in the parser definition. I've look at the manual and don't see anything about handling empty fields. How do I get syslog-ng/csv-parser to log the empty field instead of moving to the next one? Cheers, Bill

4 10

Not able to recive all syslog messages
by add gy 13 Nov '10

13 Nov '10

Hi, I have installed syslog-ng on RHEL server as log host server for 250 router , switches but some time i recive messages some times i dont , when i check it on other syslog server on windows i revice syslog messages with any problem. Request to please review syslog-ng config file and give some suggetion. config file ------------------------------------------------------------------------------------------------------------------------- # configuration file for syslog-ng, customized for remote logging # source s_internal { internal(); }; destination d_syslognglog { file("/var/log/syslog-ng.log"); }; log { source(s_internal); destination(d_syslognglog); }; # Local sources, filters and destinations are commented out # If you want to replace sysklogd simply uncomment the following # parts and disable sysklogd # # Local sources # #source s_local { # unix-dgram("/dev/log"); # file("/proc/kmsg" log_prefix("kernel:")); #}; # # Local filters # #filter f_messages { level(info..emerg); }; #filter f_secure { facility(authpriv); }; #filter f_mail { facility(mail); }; #filter f_cron { facility(cron); }; #filter f_emerg { level(emerg); }; #filter f_spooler { level(crit..emerg) and facility(uucp, news); }; #filter f_local7 { facility(local7); }; # # Local destinations # #destination d_messages { file("/var/log/messages"); }; #destination d_secure { file("/var/log/secure"); }; #destination d_maillog { file("/var/log/maillog"); }; #destination d_cron { file("/var/log/cron"); }; #destination d_console { usertty("root"); }; #destination d_spooler { file("/var/log/spooler"); }; #destination d_bootlog { file("/var/log/boot.log"); }; # # Local logs - order DOES matter ! # #log { source(s_local); filter(f_emerg); destination(d_console); }; #log { source(s_local); filter(f_secure); destination(d_secure); flags(final); }; #log { source(s_local); filter(f_mail); destination(d_maillog); flags(final); }; #log { source(s_local); filter(f_cron); destination(d_cron); flags(final); }; #log { source(s_local); filter(f_spooler); destination(d_spooler); }; #log { source(s_local); filter(f_local7); destination(d_bootlog); }; #log { source(s_local); filter(f_messages); destination(d_messages); }; # Remote logging source s_remote { tcp(ip(0.0.0.0) port(514)); udp(ip(0.0.0.0) port(514)); }; destination d_separatedbyhosts { file("/var/log/syslog-ng/$HOST/messages" owner("root") group("root") perm(0655) dir_perm(0744) create_dirs(yes)); }; log { source(s_remote); destination(d_separatedbyhosts); }; ------------------------------------------------------------------------------------------------------------------------ Regards, Add

4 14

Source udp driver and flags
by Yann I. 10 Nov '10

10 Nov '10

Hello List, I'm using the syslog-ng OSE version 3.1.2. Currently, I have a syslog-ng server which collects events from several devices : routers (Cisco), switches (Alcatel), other devices which send events to the syslog server by using the port 514 with the protocol UDP. For the router and the switch, I need to use the flag "no-parse" because I have to parse the content of the message by using "patterndb.xml". But... for the others devices I don't want to use the flag "no-parse" because the events respect the syslog format. I think it's not possible to configure syslog-ng like this : source s_udp_01 { udp (flags(no-parse); }; source s_udp_02 { udp (); }; log { source (s_udp_01); (...) }; log { source (s_udp_02); (...) }; The workaround could be the modification of udp port destination for the devices but it's not possible for the switch Alcatel. For the router Cisco, I don't know if it's possible or not... Someone would have an idea for that "problem" ? Regards, Yann I.

3 4

clearing log files
by vijay 10 Nov '10

10 Nov '10

hi all, In my system /var/log all syslog files are accumulating.Is there any way to delete files automatically. vijay

4 5

writing to Database
by vijay 10 Nov '10

10 Nov '10

Hi All, Iam writing syslog-ng entries in to database by configuring in syslog-ng.conf like below destination d_oracle { pipe("/dev/ora.pipe" template ("INSERT INTO logs (LL_HOST, LL_facility, LL_priority,LL_level, LL_tag,LL_DATE, LL_program, LL_msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY','$LEVEL', '$TAG',to_date('$YEAR.$MONTH.$DAY $HOUR:$MIN:$SEC', 'yyyy.mm.dd hh24:mi:ss'),'$PROGRAM', substrc('$MSG',1,511));\n COMMIT;\n") template-escape(yes)); }; Iam writting logs from pipe to database.Iam using below script. nohup sqlplus UNAME/PWORD@IPADDRESS/orcl @/dev/ora.pipe > /dev/null &. Sometimes pipe is getting blocked.one reason we are suspecting is DB server restarting.How to handle these situations.Even the DB server restarting the script process is alive but it is not writing to DB. vijay

2 1

Linux Distribuitions using Syslog-NG
by Renato Gregio 10 Nov '10

10 Nov '10

Hi all, someone can tell me what distros are using Syslog-NG as the main system log? Att Renato Gregio

6 9

Re: [syslog-ng] Package for Mandriva distribution
by Guillaume Rousse 09 Nov '10

09 Nov '10

Le 08/11/2010 18:12, Yann I. a écrit : > Hello list ! > > I'm looking for the package "syslog-ng 3.1.2" for Mandriva but I can't > find it. Do you where I can find that package ? It doesn't exist anymore. The latest stable version (2010.1) has syslog-ng 3.1.1, while the development version (cooker) has switched to 3.2.0beta1. If you really want the 3.1.2, you have to rebuild the previous package from mandriva subversion repository (http://svn.mandriva.com) -- BOFH excuse #175: OS swapped to disk

1 0

Package for Mandriva distribution
by Yann I. 09 Nov '10

09 Nov '10

Hello list ! I'm looking for the package "syslog-ng 3.1.2" for Mandriva but I can't find it. Do you where I can find that package ? Regards, Yann I.

2 2

Determining when logging is quiescent
by Ben Tisdall 06 Nov '10

06 Nov '10

Hi all, I'm setting up logging from ec2 instances that will get destroyed when they've finished their work. Most of the log sources use the file driver. Assuming the application has itself finished writing logs, is there any way of determining, from the client end, when syslog-ng has finished reading all of them? Regards, Ben Tisdall Photobox

2 2

login.pdb reworked
by Peter Czanik 05 Nov '10

05 Nov '10

Hello, Attached is a new version of login.pdb (called login2.pdb). It has patterns for many console/terminal/telnet login/logout events. This version should generate one set of name value pairs for each event, and only one. If you use console/login/telnet for logins, plese give it a try and let me know, how it works for you. I found, that there are some slight variations among messages even between different Ubuntu versions, so I'd like to see, how these patterns work on a larger set of Linux distributions, UNIX revisions. Thank you for your help, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

4 6

Can't get program destination to work
by Chuck 05 Nov '10

05 Nov '10

I am running 3.0.4 on a solaris 10 host. I am trying to send critical and emergency events via smtp. I have the following syntax: destination r_smtp { program ("/usr/local/sbin/syslog-ng-mailer.sh $HOST" template("<$PRI>$DATE $HOST $MSG\n.\n"); ); }; It doesn't like this syntax, however this is exactly what the documentation shows. I have also tried the following thinking the documentation might be wrong: destination r_smtp { program ("/usr/local/sbin/syslog-ng-mailer.sh $HOST"); template("<$PRI>$DATE $HOST $MSG\n.\n"); }; destination r_smtp { program ("/usr/local/sbin/syslog-ng-mailer.sh" template("<$PRI>$DATE $HOST $MSG\n.\n"); ); }; destination r_smtp { program ("/usr/local/sbin/syslog-ng-mailer.sh"); template("<$PRI>$DATE $HOST $MSG\n.\n"); }; Has anyone done this or know how to send alerts via smtp? Thanks, CC

4 7

Thoughts on patterndb syntax
by Lars Kellogg-Stedman 05 Nov '10

05 Nov '10

I've been playing with 3.2beta1 recently and getting my feet wet with the patterndb support, which I haven't had a chance to work with before. I have a few thoughts regarding the patterndb rule syntax, mostly targeted at making things a little bit easier to work with. - Rule IDs Is there any particular reason why unique IDs were selected as rule identifiers? They're not particularly meaningful to people, and they're hard to talk about. It's much easier to say, "we're suddently seeing lots of matches for ssh-accept-connection" than it is to say, "we're suddenly seeing lots of matches for 4dd5a329-da83-4876-a431-ddcb59c2858c". With the current version of syslog-ng it looks like it's possible to use arbitrary identifiers in place of UUIDs, and that's what I'm doing for my local rulesets. This even makes classification metadata more useful, because .classifier.rule_id=ssh-accept-connection is immediately meaningful, while a UUID is useless unless I go grepping around the database. - Whitespace Log messages tend to be long, which makes them unwieldy in a number of situations. It would be nice if instead of this: <pattern>...some very long log message that makes my life difficult and my patterns hard to read ...</pattern> I could do this: <pattern collapse_whitespace="yes">...some very long log message that has been conveniently wrapped to that it's easier to edit, email, and otherwise work with.</pattern> Specifically, enabling "collapse_whitespace" would transform any sequence of whitespace to a single " ". As proposed here this would be a completely backwords-compatible change because the default behavior would remain the same. - Reusable patterns We use Cisco firewall modules on our network. When these devices log a connection-related message, the source and destination address look something like this: ircs:3610:67.186.94.126/41004 That's: interface:vlan:ip/port Which becomes: @ESTRING:fwsm.src_if::@@NUMBER:fwsm.src_vlan@:@IPv4:fwsm.src_ip@/@NUMBER:fwsm.src_port@ When this occurs throughout the ruleset, and multiple times within a single message, it really lowers the readability of the rules. I wish there was a way to modularize this so that I could create custom types, something like this: <type name="FWSM_ADDRSPEC"> @ESTRING:iface::@@NUMBER:vlan@:@IPv4:ip@/@NUMBER:fwsm.port@ </type> And then do this: <pattern>Accepted src @FWSM_ADDRSPEC:fwsm.src:@ dst @FWSM_ADDRSPEC:fwsm.dst:@</pattern> And get this: fwsm.src.iface fwsm.src.ip fwsm.src.port Etc. Anyway, that's all for now.

5 12

Statistics -- how do I turn them off?
by Liam Kirsher 03 Nov '10

03 Nov '10

My log file is filling up with statistics messages from syslog-ng (v. 3.1.2) , one message every 10 minutes. I tried turning it off by putting stats_freq(0) in the options clause, and restarted the syslog-ng daemon but that seems to have no effect. There must be something obvious I'm missing. -- Liam Kirsher PGP: http://liam.numenet.com/pgp/

3 3

Re: [syslog-ng] Statistics -- how do I turn them off?
by sasanka.sahu＠wipro.com 02 Nov '10

02 Nov '10

Hi Anis, Thanks for giving me INFO. Actually I want to deploy on solaris 10 OS . This is the release verion, bash-3.00# cat /etc/release Solaris 10 10/09 s10s_u8wos_08a SPARC Copyright 2009 Sun Microsystems, Inc. All Rights Reserved. Use is subject to license terms. Assembled 16 September 2009 Am not getting this package "syslog-NG" for solaris OS. Regds, Sasanka -----Original Message----- From: Anisur Rehman(IT) [mailto:Rehman@bankmuscat.com] Sent: Tuesday, November 02, 2010 4:39 PM To: Sasanka Sekhar Sahu (WI01 - Enterprise Services) Subject: RE: [syslog-ng] Statistics -- how do I turn them off? Dear sasanka I will recommend to use RHEL 5 .5 with MYSQL database with frontend Php-Syslog-ng. I am using this combination and it is working fine. If you like to test this option let me know. However you can use HPUX option there is inbuilt syslog-NG available with HPUX version v3. Thanks Anis 0968-99826412 -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of sasanka.sahu(a)wipro.com Sent: Tuesday, November 02, 2010 2:55 PM To: syslog-ng(a)lists.balabit.hu Subject: Re: [syslog-ng] Statistics -- how do I turn them off? Hi Anybody can tell am using Solaris OS v 10 . Like to install php-syslog-ng GUI tools . where should I get the same as well as provide me some documentation for the same. Sasanka -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Sandor Geller Sent: Tuesday, November 02, 2010 4:12 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Statistics -- how do I turn them off? I'm filtering statistical messages based on the internal source and the following regexp: filter f_internal_statistics { match("^syslog-ng\[[[:digit:]]+\]: STATS") or match ("^syslog-ng\[[[:digit:]]+\]: Log statistics"); }; (for 3.x only the latter match is needed) 600 seconds is the default setting, the timer gets activated only when stats_freq > 0 so stats_freq(0) should turn off the generation of statistical messages completely. Are the logs generated on the same host or are you processing remote logs as well? On Sat, Oct 30, 2010 at 10:17 PM, Liam Kirsher <liamk(a)numenet.com> wrote: > My log file is filling up with statistics messages from syslog-ng (v. > 3.1.2) , one message every 10 minutes. > I tried turning it off by putting stats_freq(0) in the options clause, > and restarted the syslog-ng daemon but that seems to have no effect. > There must be something obvious I'm missing. > > -- > Liam Kirsher > PGP: http://liam.numenet.com/pgp/ > > ________________________________________________________________________ ______ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > > ________________________________________________________________________ ______ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com ________________________________________________________________________ ______ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html Please do not print this email unless it is absolutely necessary. The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com

1 0

Syslog-ng on Solaris 9 problem
by Elgin Lorenz 29 Oct '10

29 Oct '10

Hello experts, today we switched our syslog-ng server from linux (CentOS 5.5) to a Solaris 9 box. All went well, until I noticed that syslog-ng obviously changes some of the log entries. Here are the 2 examples I noticed: Message on the local host: Oct 18 14:19:15 xxx kernel: IN=eth1 OUT= MAC=xxx SRC=xxx DST=xxx LEN=235 TOS=0x00 PREC=0x00 TTL=128 ID=119 PROTO=UDP SPT=138 DPT=138 LEN=215 Message on the syslog-ng server: Oct 18 14:19:15 xxx kernel: kernel: IN=eth1 OUT= MAC=xxx SRC=xxx DST=xxx LEN=235 TOS=0x00 PREC=0x00 TTL=128 ID=119 PROTO=UDP SPT=138 DPT=138 LEN=215 (one "kernel: " added) Message on the local host: Oct 18 14:03:06 xxx last message repeated 2 times Message on the syslog-ng server: Oct 18 14:03:06 xxx last: message repeated 2 times (one ":" added) On Linux I never saw this behaviour. Until now I couldn't find the mistake. What am I doing wrong? Did anyone else see something like this? Thanks in advance for your help. Kind regards, Elgin Lorenz -- Elgin Lorenz BTU Cottbus Universitaetsrechenzentrum Tel. 0355 693573 E-Mail lorenz(a)tu-cottbus.de

3 8

Logging all message metadata
by Lars Kellogg-Stedman 28 Oct '10

28 Oct '10

As I work with the classification engine, I wish there were a way for me to log *all* metadata associated with a log message. That is, I would like to record any data parsed out of the message by the parsing engine, as well as all the default metadata that syslog-ng generates about a message. Is there any way to do this other than writing my own output driver? As far as I can tell, all of the available drivers -- including the sql() driver -- require me to explicitly list which attributes I want to log. While my ultimate target would be sending this data into a database, I would be happy if I were able to dump it all to a file in some sort of structured format that I could parse with my own tools (or maybe even feed back into syslog-ng). Thanks, -- Lars

3 8

Compilation problem in Debian (libsyslog-ng dependency)
by Jean F. Mousinho 28 Oct '10

28 Oct '10

Hello, I'm trying to manually build the package in Debian, but I'm getting an error when compiling: ... dh_shlibdeps: Compatibility levels before 4 are deprecated. dpkg-shlibdeps: warning: couldn't find library libdbparser.so needed by debian/syslog-ng/bin/pdbtool (its RPATH is '//lib://lib/syslog-ng'). Note: libraries are not searched in other binary packages that do not have any shlibs or symbols file. To help dpkg-shlibdeps find private libraries, you might need to set LD_LIBRARY_PATH. dpkg-shlibdeps: failure: no dependency information found for lib/.libs/libsyslog-ng.so.0 (used by debian/syslog-ng/bin/pdbtool). dh_shlibdeps: command returned error code 512 make: *** [binary-arch] Error 1 dpkg-buildpackage: failure: debian/rules binary gave error exit status 2 ... I've noticed that libdbparser.so is actually located in ./modules/dbparser/.libs/libdbparser.so but the problem seems to be that dpkg-shlibdeps doesn't find dependency information for libsyslog-ng? I don't know much how automake+dpkg-buildpackage work to know how to solve this, even tho I've tried to google for it. Thanks for your time. Jean-François Mousinho

3 3

Having hostname and sender's IP address in header
by Worsham, Michael 28 Oct '10

28 Oct '10

We are trying to streamline the syslog-ng v3.x data from the RHEL server-clients to the RHEL DCS/Syslog server while also providing another destination to a local Tripwire Log Center server. I previously had the version 3.0.8 options (as seen below), which recorded the data files correctly on the DCS server with the correct hostname attribute: options { chain_hostnames(no); time_reopen(10); time_reap(360); log_fifo_size(1000); group(adm); perm(0644); dir_perm(0755); use_dns(yes); use_fqdn(yes); create_dirs(yes); keep_hostname(yes); log_msg_size(32768); stats_freq(0); flush_lines(0); }; Destination (directory and log file structure): destination d_general { file ("/var/log/syslog/general/$HOST/$FULLHOST-$MONTH.$DAY.$YEAR.log" template("$DATE $HOST <$FACILITY.$PRIORITY> $MSGHDR $MSG\n") template_escape(no) ); }; Example: /var/log/syslog/general/CH33Test-WebCO/CH33Test-WebCO-10.25.2010.log However, we found today that Tripwire Log Center (which we are now required to use for log aggregation/reporting needs) doesn't have the ability to do hostname translation (i.e. regex the sender's hostname seen in the datastream and give it an IP address so that it can be tracked). So this means the data file that is being sent from the RHEL clients will need to keep the sender's IP address in the actual data stream. So if I do keep_hostname(no) and still enable the facility.priority template on the DCS server, it will show the IP address rather than the hostname of the reporting client: Oct 25 18:51:12 10.153.13.70 <syslog.err> syslog-ng[2820]: EOF occurred while idle; fd='9' Oct 25 18:51:12 10.153.13.70 <syslog.notice> syslog-ng[2820]: Syslog connection broken; fd='9', server='AF_INET(10.153.29.235:514)', time_reopen='10' Oct 25 18:51:22 10.153.13.70 <syslog.notice> syslog-ng[2820]: Syslog connection established; fd='9', server='AF_INET(10.153.29.235:514)', local='AF_INET(0.0.0.0:0)' However, this now causes a problem on the Syslog/DCS server with the local log file storage requirements: Old Version: /var/log/syslog/general/CH33Test-WebCO/CH33Test-WebCO-10.25.2010.log New Version: /var/log/syslog/general/10.153.13.70/10.153.13.70-10.25.2010.log NOTE: We don't have a local DNS that all of the servers can be queried against and that putting in nearly 100+ sender hosts in the /etc/hosts really isn't a viable option. How do I satisfy each requirement: 1) Allow the incoming data stream to be saved in a format based on the incoming sender's hostname for the directory structure (ex: "/var/log/syslog/general/$HOST/$FULLHOST-$MONTH.$DAY.$YEAR.log")? 2) Allow the IP address of the sender to be seen in the actual data file once saved to the DCS server (ex: "Oct 25 18:51:12 10.153.13.70 <syslog.err>...")? 3) Is there an option to have the incoming sender's data saved with both the hostname and IP address in the same header format (ex: "Oct 25 18:51:12 CH33Test-WebCO 10.153.13.70 <syslog.err>...")? -- Michael ________________________________ CONFIDENTIALITY NOTICE: This email and any attachments are intended solely for the use of the named recipient(s). This email may contain confidential and/or proprietary information of Scientific Research Corporation. If you are not a named recipient, you are prohibited from reviewing, copying, using, disclosing or distributing to others the information in this email and attachments. If you believe you have received this email in error, please notify the sender immediately and permanently delete the email, any attachments, and all copies thereof from any drives or storage media and destroy any printouts of the email or attachments. EXPORT COMPLIANCE NOTICE: This email and any attachments may contain technical data subject to U.S export restrictions under the International Traffic in Arms Regulations (ITAR) or the Export Administration Regulations (EAR). Export or transfer of this technical data and/or related information to any foreign person(s) or entity(ies), either within the U.S. or outside of the U.S., may require advance export authorization by the appropriate U.S. Government agency prior to export or transfer. In addition, technical data may not be exported or transferred to certain countries or specified designated nationals identified by U.S. embargo controls without prior export authorization. By accepting this email and any attachments, all recipients confirm that they understand and will comply with all applicable ITAR, EAR and embargo compliance requirements.

4 10

Error opening log file for writing
by Bill Snow 28 Oct '10

28 Oct '10

I have a openSUSE 11.3 system that I installed syslog-ng on through package management. The intention of the simple config is to get it to write log file messages from one program (pdns) to a separate log file outside of /var/log. No matter what I do with the file and directory permissions, I always get something like: syslog-ng[17864]: Error opening file for writing; filename='/opt/home/user/logs/pdns.log', error='Permission denied (13)' in the log file. syslog-ng is running as root, as follows: # ps xo pid,ppid,user,cmd | grep syslog 17863 1 root supervising syslog-ng 17864 17863 root /sbin/syslog-ng The special config lines in /etc/syslog-ng/syslog-ng.conf look like this: filter f_dyson { match('^pdns'); }; destination dyson { file("/opt/home/user/logs/pdns.log" perm(0644)); }; log { source(src); filter(f_dyson); destination(dyson); }; Googling around indicated a possible problem with capability management, but that is turned off in my version: # syslog-ng --version syslog-ng 3.1.1 Installer-Version: 3.1.1 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.1#master#8747d74491eac3fdf5052194f47a68e659633ca9 Compile-Date: Jul 5 2010 13:47:02 Enable-Threads: off Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-Sun-Door: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: on Enable-SSL: off Enable-SQL: off Enable-Linux-Caps: off Enable-Pcre: on I tried explicitly setting the user/group for the destination file in case syslog-ng was trying to write the file with a non-privileged owner in a directory owned by root.... and also tried setting the directory permissions to 777 just in case. Moving the destination to /var/log/pdns.log works. Of course that's not where I want the file. Anybody have an idea of what might be going on here? Thanks, Bill Snow

2 2