- syslog-ng - lists.balabit.hu

syslog-ng - Apache Logging
by Steven Shepherd 26 Jan '11

26 Jan '11

I have Apache logging working via a named pipe, however the log looks like: Jan 25 14:44:13 $HOSTNAME $VIRTUAL_HOST: 000.000.000.000 - - [25/Jan/2011:14:44:13 -0600] "GET /foo.html HTTP/1.0" 200 24040 "-" "Wget/1.10.2 (Red Hat modified)" I want to strip the syslog timestamp and $HOSTNAME. However, if I use "template("$MSGONLY\n")", it strips the timestamp, hostname *and* the $virtual_host (obtained from including "%V" in LogFormat and using 'vcommon' on apache server). Any ideas on how to remove the syslog data but leave the vhost data in place? Cheers!

2 2

Problem with getting syslog-ng statistics
by Aleksei Plotnikov 25 Jan '11

25 Jan '11

Hello everyone, I have a problem with getting statistics from syslog-ng 3.0.5 (under Debian 5.0.4) using it's socket file. In our configuration logs received by syslog-ng are being forwarded to python scripts (which in turn send logs to database). Since sometimes these scripts cannot handle the load, we want to monitor syslog-ng statistics for dropped logs. Currently, I run a script from cron which get statistics from syslog-ng socket file like this "echo STATS | socat - /var/lib/syslog-ng/syslog-ng.ctl" and then parse output. Most of time it works great, but during peak times when number of logs get higher ( ~2000 logs/sec instead of ~1500 logs/sec), this cron script cannot get any output from statistics. Also this message appears in /var/log/syslog: syslog-ng[21677]: Error writing control channel; error='Broken pipe (32)' Is it possible to tune syslog-ng statistics somehow, or maybe there is a better way to get statistics? Thanks, Aleksei ---------------------------- Võimalik, et oled TASUTA 1600€ soojamaareisist vaid CV sisestamise kaugusel! Vaata lähemalt SIIT. http://www.cv.ee

2 2

AUTO: Shanif Lakhani is out of the office (returning 02/14/2011)
by Shanif Lakhani 25 Jan '11

25 Jan '11

I am out of the office until 02/14/2011. Please contact Robert Holoweckyj and/or Lori-Ann Walker for any urgent matters. Thanks Note: This is an automated response to your message "syslog-ng Digest, Vol 69, Issue 44" sent on 1/24/11 20:55:19. This is the only notification you will receive while this person is away.

1 0

[review request]: MongoDB destination
by Gergely Nagy 24 Jan '11

24 Jan '11

I prepared three branches, in preparation for a pull request, but there's still some minor work left to do, so this is more like a "review the structure and plan before the pull request, please" request. All three branches are integration branches, and as such, I will rebase them from time to time. All three branches are folded into one commit over their base, signed-off, and should have proper descriptions. * integration/libmongo-client This one adds a git submodule, which pulls in the libmongo-client library from github to modules/afmongodb/libmongo-client. Also adds infrastructure to make autogen.sh, configure and make enter this directory and Do The Right Thing. At the moment, linking with the supplied libmongo-client is the only option, because the library is not yet ready for system-wide install. It is easy to extend this framework to use the system library if it exists - that is one of the missing features which should go into this branch, but isn't there yet. * integration/afmongodb/base This is the basic, bare-bones driver, without support for dynamic values: it depends on the previous one, and doesn't change anything outside of modules/afmongodb/, except lib/stats.[ch], where it adds SCS_MONGO for stat counter support. The current state of the driver is reasonably stable, there are three known problems that I am aware of: + Segfaults whenever the writer thread emits an msg_debug() or similar itself. This is actually a bug in ivykis, for which Bazsi had a fix for, but I can't find the fix in his git tree yet (only the log message, but without the patch). + Locking issues: it's fairly easy to push the driver into a deadlock, where it will just sit and wait, not doing a thing. I've only seen this happen with flow-control enabled, though. This will have to be solved before I'm confident enough to ask for a pull. + The libmongo-client library is built as a shared library, and is installed as such. It should be static-only, and not installed, only linked to afmongodb. This will be implemented mostly in libmongo-client, and an update of the integration/libmongo-client branch will pull that in; then this branch will be updated accordingly. * integration/afmongodb/dyn-vars Based on integration/afmongodb/base, this adds dynamic_values() support. Minor modifications are done to nvtable too, but it's a very small patch, and very straightforward too. However, as discussed earlier, the goal in the long run is to develop a generic framework for this kind of thing, so this here is more like a temporary solution so that people can experiment with the key feature the mongodb provides over the others. Until the generic framework is ready, this, I believe, is an acceptable compromise. -- |8]

2 2

Error starting syslog-ng
by Fabien Bagard 24 Jan '11

24 Jan '11

Hi list ! I have a problem to start syslog-ng. I got a syntax error on line in bold : ###### # sources # all known message sources source s_all { # message generated by Syslog-NG internal(); # standard Linux log source (this is the default place for the syslog() # function to send logs to) unix-stream("/dev/log"); # messages from the kernel file("/proc/kmsg" log_prefix("kernel: ")); }; source s_unix { tcp( port(514) * tls(* key_file("/etc/syslog-ng/certs/logger.parrot.biz.key") cert_file("/etc/syslog-ng/certs/logger.parrot.biz.crt") ca_dir("/etc/syslog-ng/certs/CA/") peer_verify(optional-untrusted) ) ); }; But I can't figure out what ! Can someone point this error to me ? Thanks -- Fabien Bagard IT Department tel + 33 (0)1 48 03 60 40 -------------------------------------------------------------------------------- Parrot SA 174, Quai de Jemmapes | 75010 Paris - France tel + 33 (0)1 48 03 60 60 | fax + 33 (0)1 48 03 70 08 http://www.parrot.com -------------------------------------------------------------------------------- This e-mail message and any attached document(s) are for the sole use of the intended recipient(s)and may contain confidential and legally privileged information. Any unauthorized review, copy, use and/or disclosure is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original.

4 9

Syslog-ng error
by Shawn Cannon 24 Jan '11

24 Jan '11

I am trying to setup a second syslog-ng server that will be part of a load balanced cluster. When the 2nd server tries to connect to my mssql database, it tries to run the CREATE TABLE commands which fails because the table already exists since the first syslog-ng server already created them. How do I tell the 2nd server not to try the CREATE TABLE stuff so it will start working? Shawn Cannon IT Professional http://www.shawncannon.com Contact Me <http://www.linkedin.com/pub/shawn-cannon/5/84/81a> LinkedIn<http://www.linkedin.com/pub/shawn-cannon/5/84/81a> <http://www.facebook.com/rolltidega72> Facebook<http://www.facebook.com/rolltidega72> <http://twitter.com/rolltidega> Twitter <http://twitter.com/rolltidega> My blog: My thoughts on VMworld… a week later…<http://www.shawncannon.com/?p=19>

5 10

syslog-ng 3.1.3-2 - working examples of remote logging
by Steven Shepherd 21 Jan '11

21 Jan '11

OK, so syslog has come a long way and many configuration changes in the years :) Can someone provide working examples of (3.1.X) client/server syslog-ng.conf set up for remote logging purposes such as /var/log/messages, /var/log/kern.log, /var/log/auth.log...etc? Would want on local machine as well as remote syslog server. Will also want to remotely log apache access/error log via syslog-ng as well. Any help/tips/pointers is greatly appreciated! Cheers!

3 2

Open source version: syslog-ng-3.1.3-1.rhel5.i386.rpm
by julian 20 Jan '11

20 Jan '11

what install options are assumed for syslog-ng-3.1.3-1.rhel5.i386.rpm: server, relay, client? It is acting as relay.

3 2

oracle db + syslog-ng integration
by vijay 19 Jan '11

19 Jan '11

hi, Iam Using syslog-ng 3.0.4.when i used below configuration destination d_sql { sql(type(oracle) username("myusername") password("mypassword") database("LOGS") table("msgs_${R_YEAR}${R_MONTH}${R_DAY}") columns("datetime varchar(16)", "host varchar(32)", "program varchar(32)", "pid varchar(8)", "message varchar2") values("$R_DATE", "$HOST", "$PROGRAM", "$PID", "$MSGONLY") indexes("datetime", "host", "program", "pid", "message")); }; iam facing error while starting service syntax error in /usr/local/etc/syslog-ng.conf at line 322. syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng the line 322 is---------- sql(type(oracle) username("myusername") password("mypassword") vijay.p

2 3

syslog ng open version 3.1.3 does not relay or write to local file
by Bentayeb, Julian A. 19 Jan '11

19 Jan '11

I have syslog-ng-3.1.3-1.rhel5.i386.rpm open version installed and want syslog clients to send to the syslog-ng server; The intention is to have the server send all logs to a final syslog destination and a keep a copy in a file locally. The only thing that is working is I am seeing SYSLOG packest via tcpdump. No messages come to the local file and no messages go to the final destination. I installed the rpm and there was no mode option (server, relay, client) I could chose. Your help is appreciated. Here is the syslog conf I am using: @version:3.0 # # Relay all logs to final destination # options { mark_freq(30); keep_hostname(yes); }; source s_local { unix-stream("/dev/log"); internal(); }; source s_network { syslog(transport(udp)); }; destination d_syslog_udp { syslog("xx.xx.xx.xx" transport("udp") ); }; log { source(s_local); source(s_network); destination(d_syslog_udp); }; # # Keep a copy of logs coming over the network locally in a file for Splunk # source s_network { syslog(transport(udp));}; destination d_file { file("/var/log/messages" ); }; log { source(s_network); destination(d_file); }; Julian Bentayeb Lawrence Livermore National Laboratory Information and Communications Systems 925-424-5702

1 0