February 2010 - syslog-ng - lists.balabit.hu

[Re: handling errors with sql inserts]
by Juhasz Viktor 11 Feb '10

11 Feb '10

1 0

handling errors with sql inserts
by Bob Belnap 10 Feb '10

10 Feb '10

Hello, I am parsing some of my log messages into different database columns using the sql() function, and I have noticed that when there is an insert error into the database no further logs for that host make it to the db. In fact, syslog-ng keeps attempting the insert, and logs the insert error indefinitely. I couldn't find a way to gracefully handle these sorts of errors in the docs. I'd like to be able to specify an action on a database insert failure (log message to a different log and stop trying to insert, for example). But I do not want a single insert failure to stop all subsequent inserts for that host. What are my options here? I'm currently using OSE 3.0.4. Thanks for your help. --Bob

2 2

Re: [syslog-ng] need help with a match filter
by chris packham 10 Feb '10

10 Feb '10

Someone else on the list may correct me but I think you'll find that the destination is evaluated at the time syslog is started (or restarted) not when a log message is processed. I'm pretty sure there was a discussion on the list about something similar a while back (although I couldn't find it in my emails) So short answer is that you can't use a file desination. You may however get some mileage out of writing a custom program destination and do the last bit of processing in your program. >>> Rory Toma <rory(a)ooma.com> 01/25/10 5:12 PM >>> Looking more at the docs, it looks likes I also need a value for 3.0 syntax. So I am trying something like below. However, the value of $foo_hostdir is not being set correctly. It is being set to "foo_hostdir". How do I get the value to be the value of the match? filter foo_filter { match('host("foo_[0-9A-F]\{9\}$")' value("foo_hostdir") type("string") ); }; # foo destinations destination foo_dest { file("/logs/$foo_hostdir/$R_YEAR$R_MONTH$R_DAY/$HOST-$R_YEAR$R_MONTH$R_DAY.log" owner(root) group(root) perm(0444) template("$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC $MSG\n") template_escape(no)); }; On 1/25/10 12:02 PM, Rory Toma wrote: > I'm trying to dynamically create lognames and directories form the host > name that is sent. My hosts' names are of the form > > foo_<mac address> > > I want to create a destination that would be > > /logs/foo_<first 9 digits of mac address>//foo_.log > > Here is what I tried so far, and it does not work, but is close. Can > someone suggest a config that might work? Thx. > > I can do this individually, but having 100+ filters and 100+ log lines > in the config really slows things down. > > #filter for foo > filter foo_filter { match('host("foo_[0-9A-F]\{9\}$")' > flags("store-matches")); }; > > # foo destinations > destination foo_dest { > > file("/logs/$0/$R_YEAR$R_MONTH$R_DAY/$HOST-$R_YEAR$R_MONTH$R_DAY.log" > owner(root) group(root) perm(0444) > template("$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC $MSG\n") > template_escape(no)); > }; > > > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > > ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

3 7

--fd-limit
by Rory Toma 10 Feb '10

10 Feb '10

My understanding (now) is that you need to set --fd-limit *and* up the ulimit. Simply updating ulimit is not sufficient for syslog-ng to be able to handle more than the default 4k open files. Is this correct?

2 1

Apache + syslog-ng w/o logger?
by Brian Donaldson 09 Feb '10

09 Feb '10

My goal is to enable Apache to log large access_log entries (> 1024 chars) to a syslog-ng server. The vast majority of examples I've seen for connecting Apache to syslog-ng employ the ancient /usr/bin/logger -- but that approach chops up messages longer than 1024 chars. The alternative approach is to use a perl/python script, but I'm nervous about the overhead. Anyone know of an updated logger binary? Thanks for any pointers, -Brian

5 4

two sources one udp address
by Alexei Shilin 06 Feb '10

06 Feb '10

Hi! I have source src { unix-stream("/dev/log" max-connections(512)); internal(); file("/proc/kmsg"); udp(port(514)); }; ... source temp { udp(port(514)); }; and then I have to log rules: one for src and one for temp, no flags are set. Will those two sources play nice with each other and will not interfer? E.g. if one reads from that port, will the log message that was read by the first available for read by the second? -- http://www.fastmail.fm - Email service worth paying for. Try it for free

2 1

Character Conversion Question
by James B Horwath 06 Feb '10

06 Feb '10

I am using SNARE to send messages to syslog-ng (2.0.9). There are fields in the messages separated by a TAB (0x9). When syslog-ng writes this out to a file it convert the hexadecimal (0x9) to \011 (literally slash, zero, one, one). This makes the lines hard to read. How can I either convert the TAB to a space or preserve the TAB in the output. Preserving the TAB would be my preference. I did not see anything in the documentation except writing a regex to convert the character. Thanks in advance, Jim ----------------------------------------- This message, and any attachments to it, may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are notified that any use, dissemination, distribution, copying, or communication of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately by return e-mail and delete the message and any attachments. Thank you.

2 1

Help with spoofing hostname
by PAUL WILLIAMSON 06 Feb '10

06 Feb '10

Thanks for the suggestions yesterday. As usual, there are multiple ways to solve the problem, each of them equally easy! Now on to my next issue... We have a product called Symantic SIM (Security Information Manager) that is on the receiving end of some forwarded messages. I have the keep_hostname(yes) option enabled, and when our SIM gets the message, the originating hostname is in the message. The problem is that is seems like the SIM is detecting that the message is coming from my loghost where Syslog-ng is installed, and tagging every message like it's from that instead of the actual host. We've been over the config with their engineers and our security department, and this is what we got back from Symantec today. "The hostname is available in the message, so it looks like that part is working. The problem is this, the message the SSIM sees is: <TIME> <Syslog Server IP> <Message> The SSIM then puts the syslog server IP as the source and destination of the host. Essentially the message needs to be sent to the SSIM as: <TIME> <Originating device Source IP> <Message> To do this, the message will need to be spoofed." So, I have two questions: 1. Can the messages be spoofed? 2. Does anyone else use this product and would be willing to share configs (of either syslog-ng or SSIM). Thanks, Paul ************************************ This email may contain privileged and/or confidential information that is intended solely for the use of the addressee. If you are not the intended recipient or entity, you are strictly prohibited from disclosing, copying, distributing or using any of the information contained in the transmission. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. This communication may contain nonpublic personal information about consumers subject to the restrictions of the Gramm-Leach-Bliley Act and the Sarbanes-Oxley Act. You may not directly or indirectly reuse or disclose such information for any purpose other than to provide the services for which you are receiving the information. There are risks associated with the use of electronic transmission. The sender of this information does not control the method of transmittal or service providers and assumes no duty or obligation for the security, receipt, or third party interception of this transmission. ************************************

2 1

syslog-ng MSGHDR
by Mij 06 Feb '10

06 Feb '10

Dear syslog-ng folks, I am the maintainer of sshguard, see http://www.sshguard.net . Sshguard can be interfaced with syslog-ng. Multiple users of syslog-ng recently reported that switching to 3.x required a configuration change for preserving the original logging format, see https://sourceforge.net/mailarchive/forum.php?thread_name=EE040D72-0185-41E… https://sourceforge.net/mailarchive/forum.php?thread_name=DA2160C1-09A0-475… We reflected the reports by updating the setup docs to contain a block for the 2.x version and one for 3.x , see http://www.sshguard.net/docs/setup/getlogs/syslog-ng/ However, this change is not apparent in your documentation or changelogs, and other users reported that with even more recent versions, the "old format" is again the correct one. Can you clarify what is the intended template for producing entry tags of the classic format "Jan 21 12:54:09 examplehost proftpd[18965]: applmsg" in the different versions? thanks

2 3

Re: [syslog-ng] Apache + syslog-ng w/o logger?
by Brian Donaldson 06 Feb '10

06 Feb '10

Thanks for the suggestions Jim & Lance. I'm running CentOS 5 using the default logger. I did try what Lance suggested: restarting syslog-ng w/ log_msg_size set to 65K in the options section (both the client and server). However, echo'ing a 8192 character string to logger resulted in several fragments. Guess it's on to Jim's suggestion... On Fri, Feb 5, 2010 at 3:00 AM, <syslog-ng-request(a)lists.balabit.hu> wrote: > Send syslog-ng mailing list submissions to > syslog-ng(a)lists.balabit.hu > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.balabit.hu/mailman/listinfo/syslog-ng > or, via email, send a message with subject or body 'help' to > syslog-ng-request(a)lists.balabit.hu > > You can reach the person managing the list at > syslog-ng-owner(a)lists.balabit.hu > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of syslog-ng digest..." > > > Today's Topics: > > 1. Re: config file question (Rory Toma) > 2. Apache + syslog-ng w/o logger? (Brian Donaldson) > 3. Re: Apache + syslog-ng w/o logger? (Jim Hendrick) > 4. Re: Apache + syslog-ng w/o logger? (Lance Laursen) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 04 Feb 2010 16:52:52 -0800 > From: Rory Toma <rory(a)ooma.com> > Subject: Re: [syslog-ng] config file question > To: Zolt?n Pallagi <pzolee(a)balabit.hu> > Cc: Syslog-ng users' and developers' mailing list > <syslog-ng(a)lists.balabit.hu> > Message-ID: <4B6B6BE4.30109(a)ooma.com> > Content-Type: text/plain; charset="iso-8859-1" > > On 2/2/10 4:49 AM, Zolt?n Pallagi wrote: > > Hi Rory, > > > > Will you tell me the current naming of your log files? I don't know > > the real reason for many log files, but as Robert said, perhaps, there > > are another easier solutions to avoid it. > > > We maintain one log file per client per day, in directory buckets of > 2000 clients. >

2 1