- syslog-ng - lists.balabit.hu

race condition in destination driver deinit?
by Michael Hocke 07 Apr '12

07 Apr '12

-----BEGIN PGP SIGNED MESSAGE----- Me again.... Hi guys, I've been running 3.3.3 on Solaris 10 x86 for quite a bit now. I've got two boxes, both running the same OS release and the same release of syslog-ng. One of them (box A) has a destination configured that doesn't really exist so I've been getting Mar 13 14:46:15 flowmon-sys syslog-ng[20118]: I/O error occurred while writing; fd='24', error='Connection refused (146)' which is perfectly fine and I just ignored it. The other Solaris box (box B) does not have this destination configured. It now happens that after a random number of HUP signals syslog-ng on box A crashes with a segmentation fault and the following backtrace: (gdb) bt #0 0xfef2a7a8 in log_dest_driver_release_queue_method (self=0x80d2958, q=0x83e58955, user_data=0x0) at driver.c:80 #1 0xfef2aa5c in log_dest_driver_deinit_method (s=0x80d2958) at driver.c:80 #2 0xfe914a14 in afsocket_dd_deinit (s=0x80d2958) at afsocket.c:109 #3 0xfef29cd6 in log_dest_group_deinit (s=0x8074198) at dgroup.c:59 #4 0xfef24a21 in log_center_deinit (self=0x8090cc0) at center.c:67 #5 0xfef25493 in cfg_deinit (cfg=0x80d2848) at cfg.c:90 #6 0xfef406b2 in main_loop_reload_config_apply () at mainloop.c:364 #7 0xfef40a42 in main_loop_io_worker_sync_call (func=<value optimized out>) at mainloop.c:364 #8 0x08047900 in ?? () I was able to reproduce this over and over again. Sometimes it happens after 11 HUPs, sometimes after 35, but it eventually does crash. Since the only difference between box A and box B is the one additional destination I suspect that this is the cause for this segfault. Maybe the HUP signal came at a time when syslog-ng was trying to send something to the non-existing destination? I'll try to collect some more data. If anybody could give me some direction on where exactly I should look into, I'll be happy to do that. - - Michael -----BEGIN PGP SIGNATURE----- Version: PGP Desktop 10.0.3 (Build 1) Charset: us-ascii wsBVAwUBT1+ZRZbfnpCg64TVAQEiVgf7BmePJa6Va396QRyBLNPpOHzAvi9p3n07 uVvvYOcMHUFwixZoC0BXs+21EfSkMOmROQWupXBTcEbyBMOf0+HXDsuO7mfZDznU AZzLBzhgaDPf0stx55PqOssKTf28QLtxs1gUmJSb2DgmK9WjtMsovAlS83pQOa6O oQIaK3SBIUtrbijdx7vLvS0VUcqhDzJI9TJ7R+A0x3i9dAWKIvDb7wLW4CvB9k19 7bmmXFwpAClGZ6DS1CdIt4eHuDiuIoes2uXNhh5xReZpCh89FHzv8LxWxVnph3ms 6FctjAt4JMQTGnZRHN7QidYAGwjjabqtGjKWvXtRfoBCdUfLfC+wAA== =JjOa -----END PGP SIGNATURE-----

2 1

AUTO: James B Horwath/TheGuardian is out of the office. (returning 04/09/2012)
by James B Horwath 07 Apr '12

07 Apr '12

I am out of the office until 04/09/2012. I am out of the office until Monday April 9, 2012 with limited access to e-mail. If there are Nitro issues please contact Dante Rodino at 212-919-2819 or Sunil Gupta at 610-807-6710. Any other issues please contact Al Limone manager of IT Security Services & Governance. Thanks, Jim Note: This is an automated response to your message "syslog-ng Digest, Vol 84, Issue 10" sent on 4/6/2012 20:49:46. This is the only notification you will receive while this person is away. ----------------------------------------- This message, and any attachments to it, may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are notified that any use, dissemination, distribution, copying, or communication of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately by return e-mail and delete the message and any attachments. Thank you.

1 0

Handling multiple timezones
by Chris Hiestand 07 Apr '12

07 Apr '12

After some googling and reading the admin guide, I think what I want to do is not possible. But I'd like confirmation. My syslog-ng server gets syslog messages from clients with two timezones: UTC and US/Pacific. But the UTC clients (ESXi servers) cannot specify the UTC timezone, so the collector incorrectly assumes the timestamps are local (US/Pacific). I'd really like to convert the UTC timezone to US/Pacific so my entire output is in US/Pacific. But the only place I can specify a source timezone is in the source section - however both client groups send syslogs to the same source so I cannot do that. So potential workarounds are: A. Set all the clients to output in the UTC timezone so that I can set UTC in the common source and then convert to US/Pacific as the default output timezone. B. Setup another IP address to collect syslogs, and set the different timezone on that source (and configure all of those timezones clients to syslog to a different IP). This means you need N IP addresses to collect N different timezones. Is it considered "Best Practice" to have all clients send syslogs in UTC timezone to avoid this complication? Any help is appreciated. Thanks, Chris

2 2

[PATCH] afsocket: Do not die if a destination is not resolvable at startup.
by Gergely Nagy 06 Apr '12

06 Apr '12

If a destination cannot be resolved at init time, do not panic, just bail out, but otherwise pretend we succeeded: after time_reopen, we'll retry anyway! And on that note, teach afsocket_dd_start_connect() to call afsocket_dd_apply_transport() if our fd is -1 (ie, not connected). Reported-by: Peter Palfrader <weasel(a)debian.org> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/afsocket/afinet.c | 2 +- modules/afsocket/afsocket.c | 8 +++++--- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/modules/afsocket/afinet.c b/modules/afsocket/afinet.c index f83ff38..a7d3635 100644 --- a/modules/afsocket/afinet.c +++ b/modules/afsocket/afinet.c @@ -498,7 +498,7 @@ afinet_dd_apply_transport(AFSocketDestDriver *s) if ((self->bind_ip && !resolve_hostname(&self->super.bind_addr, self->bind_ip)) || !resolve_hostname(&self->super.dest_addr, self->super.hostname)) - return FALSE; + return TRUE; afinet_set_port(self->super.dest_addr, self->dest_port ? : default_dest_port, self->super.flags & AFSOCKET_DGRAM ? "udp" : "tcp"); diff --git a/modules/afsocket/afsocket.c b/modules/afsocket/afsocket.c index ae9c5c2..095f5a5 100644 --- a/modules/afsocket/afsocket.c +++ b/modules/afsocket/afsocket.c @@ -1103,6 +1103,11 @@ afsocket_dd_start_connect(AFSocketDestDriver *self) gchar buf1[MAX_SOCKADDR_STRING], buf2[MAX_SOCKADDR_STRING]; main_loop_assert_main_thread(); + + if (self->fd == -1) + if (!afsocket_dd_apply_transport(self)) + return FALSE; + if (!afsocket_open_socket(self->bind_addr, !!(self->flags & AFSOCKET_STREAM), &sock)) { return FALSE; @@ -1169,10 +1174,7 @@ afsocket_dd_init(LogPipe *s) /* these fields must be set up by apply_transport, so let's check if it indeed did */ g_assert(self->transport); - g_assert(self->bind_addr); - g_assert(self->dest_addr); g_assert(self->hostname); - g_assert(self->dest_name); if (cfg) { -- 1.7.9

2 3

[Bug 171] New: Unwanted IP adress Folder
by bugzilla＠bugzilla.balabit.com 04 Apr '12

04 Apr '12

https://bugzilla.balabit.com/show_bug.cgi?id=171 Summary: Unwanted IP adress Folder Product: syslog-ng Version: 3.1.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: julien.groselle(a)gmail.com Type of the Report: bug Estimated Hours: 0.0 Hi, I work for a lange society, and we use syslog-ng for 5 years now. We have a centralized server with storage tank to keep logs securly. Concretely, we have 2 centralized servers syslog-ng in chrooted environement, and 50 clients servers. Since we use TLS transport in place of stunnel workaround, we have many issues : - First of all, many logs aren't writen in $HOST folder but in IPADDRESS folder. So, to be clear, this is an exemple : # ls drwxr-x--- 8 root adm 4,0K 1 mars 00:07 10.0.0.1 drwxr-x--- 53 root adm 4,0K 19 mars 00:35 host1 I assume that host1 have 10.0.0.1 IP address and # tree 192.168.100.79/2012-03/ 10.0.0.1/2012-03/ ├── 02-user-10.0.0.1.log.bz2 ├── 06-user-10.0.0.1.log.bz2 ├── 07-user-10.0.0.1.log.bz2 ├── 08-user-10.0.0.1.log.bz2 ├── 09-user-10.0.0.1.log.bz2 ├── 12-user-10.0.0.1.log.bz2 ├── 13-user-10.0.0.1.log.bz2 ├── 14-user-10.0.0.1.log.bz2 ├── 15-user-10.0.0.1.log.bz2 ├── 16-user-10.0.0.1.log.bz2 └── 19-user-10.0.0.1.log # tree host1/2012-03/ |grep 19- ├── 19-apache.access-host1.log ├── 19-apache.error-host1.log ├── 19-authpriv-host1.log ├── 19-auth-host1.log ├── 19-cron-host1.log ├── 19-daemon-host1.log ├── 19-kern-host1.log ├── 19-mail-host1.log ├── 19-nagios-host1.log ├── 19-puppetd-host1.log ├── 19-syslog-host1.log └── 19-user-host1.log (we have this problem with many servers) In facility "user" for host 10.0.0.1 in fact i have log for snmptrapd... But why ?? We have config for snmpd but not for snmptrapd... So i have tried to define a default facility => failed After i have tried many dns and hostnames options => failed As anyone here have a way to search for me ? If you need more details, i'm your's. Kind regards. -- JG -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

GSoC wants you to code syslog-ng/Zorp in the summer
by Peter Czanik 03 Apr '12

03 Apr '12

Hello, This was included in last weeks newsletter, but as the deadline is three days away, it needs some extra attention: Google Summer of Code ( http://code.google.com/soc/ ) is a nice opportunity for higher education students to spend their summers productively by coding in open source software projects. This time BalaBit participates in GSoC with the help of the openSUSE <http://www.opensuse.org/> project. If you are interested in enhancing syslog-ng <http://www.balabit.com/network-security/syslog-ng/opensource-logging-system…> or Zorp <http://www.balabit.com/network-security/zorp-gpl/overview>, please see our project ideas on the openSUSE ideas page: http://en.opensuse.org/openSUSE:GSOC_ideas#syslog-ng and http://en.opensuse.org/openSUSE:GSOC_ideas#Zorp *Why it is worth attending?* - You can have a lifetime experience - It looks good in your CV - If you need documentation from Google to provide to your school so you can obtain course credit, you can get it -- after the positive final evaluation from your mentor - You can earn 5000 USD for a 3 months student job *When can I apply?* The deadline is 6th April (Friday) 2012, 19:00 UTC. *Who can apply?* - Students, 18 years of age or older on or before April 23, 2012. Google defines a student as an individual enrolled in or accepted into an accredited institution including (but not necessarily limited to) colleges, universities, masters programs, PhD programs and undergraduate programs. You are eligible to apply if you are enrolled in an accredited pre-university educational program provided you meet all of the other eligibility requirements. More details: http://www.google-melange.com/document/show/gsoc_program/google/gsoc2012/fa… *How to apply?* - Please discuss your application first with the project mentor (syslog-ng: Gergely Nagy algernon(a)balabit.hu <mailto:algernon@balabit.hu>, Zorp: Szilard Pfeiffer coroner(a)balabit.hu <mailto:coroner@balabit.hu>) and then apply at Google at http://www.google-melange.com/ <http://www.google-melange.com/>and introduce yourself on the opensuse-project mailing list(http://lists.opensuse.org/opensuse-project/ <https://lists.opensuse.org/opensuse-project/>) - Please also attach a test exercise (depends on the project you apply to) a log of a successful *syslog-ng* 3.3 compilation from source/a log of a successful *Zorp* 3.9.5 compilation from source. An even more interesting thing would be if applicants already tried the Zorp virtual machines in advance, to get familiar with the software. *When I will be informed about further details?* On 23rd April the accepted student proposals announced on the Google Summer of Code 2012 site. For further timeline, please visit <http://www.google-melange.com/document/show/gsoc_program/google/gsoc2012/fa…>http://www.google-melange.com/document/show/gsoc_program/google/gsoc2012/fa… In case of any questions, please visit the Google Summer of Code 2012 Frequently Asked Questions site: http://www.google-melange.com/document/show/gsoc_program/google/gsoc2012/fa… We wish you good luck, and hope to hear from you soon! (this page is the current content of http://czanik.blogs.balabit.com/2012/03/gsoc-wants-you-to-code-syslog-ngzor… , which hold more recent information later on). Bye, -- Peter Czanik (CzP)<czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

Ongoing 3.3.4 memory leaks
by Marvin Nipper 02 Apr '12

02 Apr '12

Hi. Sorry, looking for some help. Still trying to chase some memory leaks that I see on Solaris 10. I'll crank up syslog-ng, and it'll start running at around 7 or 8MB, and interestingly, it will run that way sometimes for several hours, and then (for reasons unknown), I'll take a look at it a bit later, and suddenly it will have jumped to 26MB (or later on, maybe 75MB). I've already applied the (older) logwriter.c fix and the (new) Gergely driver.h/afsocket.c fix, in an attempt to resolve known issues. Certainly the logwriter fix corrected a super-nasty, continuous leak problem, but this other leak problem still lingers. MY PROBLEM is that I'm wanting to use the debug symbols in syslog-ng, so that I can have the Solaris tools specifically show the exact location where these leaks are happening, BUT, whereas a non-debugging version of syslog-ng executes properly, when I configure syslog-ng with -enable-debug, the resulting executable will no longer start. It immediately exits, coughing up what amounts to a syntax error (even though I'm using the exact same command-line invocation that I use on the non-debugged executable): #/etc/init.d/syslog-ng start syslog-ng service starting. mkfifo: File exists mkfifo: File exists Usage: /usr/local/sbin/syslog-ng { start | stop } # Something about enabling debugging is impacting the operational behavior of the resulting binary. Anyway, I can use (for Solaris) libumem and mdb to narrow this down to a particular module, but without the debug symbols, I can't really get to a specific chunk of code, so it would obviously help to get that working. If anyone has any ideas, I certainly would appreciate the input. THANKS Marvin Nipper This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

3 3

On FreeBSd: syslog-ng: I/O error occurred while writing; fd='xx', error='No buffer space available (yy)'
by Traiano Welcome 02 Apr '12

02 Apr '12

Hi List I've been seeing the following in the messages log of my freebsd syslog server for quite some time now: --- Mar 20 12:19:12 syslog2 syslog-ng[35313]: I/O error occurred while writing; fd='12', error='No buffer space available (55)' Mar 20 12:19:12 syslog2 syslog-ng[35313]: Connection broken; time_reopen='60' Mar 20 12:19:12 syslog2 syslog-ng[35313]: I/O error occurred while writing; fd='13', error='No buffer space available (55)' Mar 20 12:19:12 syslog2 syslog-ng[35313]: Connection broken; time_reopen='60' --- These happen at a frequency of about 7 per minute on average. See attached trend graphs for an idea of the volume of traffic we're doing, as well as the memory and cpu utilisation trends on this server during this period. As can be seen from the graphs, load does not seem to be the issue. Occasionally during the week, the system freezes and requires a reboot, I think it's related to the above message, though I'm not sure. My question is: What does this error mean, and how can I resolve it? I have tried to frame this as an operating system kernel resource issue, and experimented with increasing the freebsd kernel sysctls for UDP performance: --- [root@syslog2 <mailto:root@syslog2.ops.mtnbusiness.co.za> /var/log]# sysctl kern.ipc.nmbclusters=102400 kern.ipc.nmbclusters: 25600 -> 102400 [root@syslog2 <mailto:root@syslog2.ops.mtnbusiness.co.za> /var/log]# sysctl kern.ipc.maxsockbuf=201326592 kern.ipc.maxsockbuf: 100663296 -> 201326592 [root@syslog2 <mailto:root@syslog2.ops.mtnbusiness.co.za> /var/log]# sysctl net.inet.udp.recvspace=33554432 net.inet.udp.recvspace: 16777216 -> 33554432 --- This has reduced the frequency of the errors a little, but in general the problem still remains. Syslog version: -- [root@syslog2]# syslog-ng -V syslog-ng 2.0.10 -- FreeBSD version: -- FreeBSD 7.2-STABLE #0 -- Any help would be much appreciated! Traiano

3 4

Building syslog-ng on OpenBSD v5 with SQL support
by Richard Brooks 01 Apr '12

01 Apr '12

I'm having a really hard time trying to find out how to tweak the make file supplied on the ports for syslog-ng v3.1.4, so that I get a binary that supports SQL on OpenBSD. Any tips would be gratefully received. Regards Richard

2 2

Known memory leak issues in 3.3.4?
by Marvin Nipper 01 Apr '12

01 Apr '12

After considerable effort, I managed to get 3.3.4 built (using gcc 4.4.2, along with glib 2.29.92 and the requisite eventlog 0.2.12). The problem is that the resulting syslog-ng component grows continuously (at an alarming rate). Whereas my 3.0.5 syslog-ng process starts out running in just under 3MB, and then stabilizes at about 3.3MB, the 3.3.4 version starts out small, and then at the point where I killed it (about 2 hours after starting it), it had already grown to 277MB. I'm doing nothing fancy here (config shown below). No data transformations, no use of time normalization features, no DB components. In short, I'm not trying to use any of the many new/elaborate features that have been added over the past few years. I'm just "eating log input", writing it to handful of files and pipes, just like I've been doing since the syslog-ng 2.x days. As one of the regular problems that I see mentioned in various threads is "some new fix for a memory leak", I'm just trying to see if there's some known problem in the virgin 3.3.4 code, that might be causing my problems. (I.E. should I wait for 3.3.5, and try again?) FYI, my environment is Solaris 10 x86. Aside from having moved to the newer gcc compiler (in order to even attempt to compile the latest syslog-ng version), the only other major component change was the newer version of glib, which as I recall, was also necessary to resolve compilation issues with the new syslog-ng versions. Anyway... before I beat my head against the wall trying to figure out what to do next, I figured that I'd ask the easy question (about "known bugs") first, so that I don't spend any time chasing a problem that may already be fixed. (I did dig thru the mailing list, and saw at least one mention of a memory leak in 3.3.4, but it seemed to be related to a specific bit of functionality, that I didn't think applied in my situation.) (And just a side-comment, FWIW, I noticed that when I swapped the 3.3.4 version, for the 3.0.5 version, the host system immediately incurred what I estimate to be at least a 20% high CPU load that had been required by the 3.0.5 version. And when I reverted back to 3.0.5 a few hours later, I saw that happen in reverse. I'm not sure why the newest version requires more resources, again, when I'm not asking it to do anything new, but it definitely seems that it does.) Thanks for your time and input. Existing config file shown below. Note that other than the version change, the only other modification (from the file that I was using in 3.0.5) relates to the threading changes, which I made based upon Bazsi's recommendations in an email on 3/18, about how to avoid potential data loss on inbound UDP packets. Again, thanks for any help/input: @version:3.3 options { dir_perm(0755); perm(0640); group(wheel); chain_hostnames(no); keep_hostname(yes); log_fifo_size(41000); threaded(no); dns_cache_size(5000); dns_cache_expire(86400); dns_cache_expire_failed(86400); }; source any_udp { udp(flags(store-legacy-msghdr)); }; source any_tcp { tcp(port(601) max-connections(40) flags("store-legacy-msghdr", "threaded") use_dns(no)); }; destination SEC {pipe("/tmp/sec"); }; destination WIN-PIPE {pipe("/tmp/windows"); }; destination routers_log { file("/var/adm/log/routers.log" create_dirs(yes) flags("threaded")); }; destination ravlin_log { file("/var/adm/log/ravlin.log" create_dirs(yes) flags("threaded")); }; destination windows_log { file("/var/adm/log/windows.log" create_dirs(yes) flags("threaded")); }; destination workstation_log { file("/var/adm/log/workstation.log" create_dirs(yes) flags("threaded")); }; destination catch-all_log { file("/var/adm/log/catch-all.log" create_dirs(yes) flags("threaded")); }; destination test { file("/var/adm/log/test.log" create_dirs(yes) flags("threaded")); }; destination dev_null {}; destination secdevedc01 { udp("10.132.193.245" spoof_source(yes) flags("threaded")); }; destination engedc01 { udp("10.132.192.199" spoof_source(yes) flags("threaded")); }; destination egssiem { udp("10.10.36.157" spoof_source(yes) flags("threaded")); }; log { source(any_udp); destination(SEC); }; log { source(any_udp); destination(egssiem); }; filter f_4 { facility(syslog) and level(info..emerg); }; log { source(any_udp); filter(f_4); destination(WIN-PIPE); }; log { source(any_udp); filter(f_4); destination(windows_log); flags(final); }; filter f_1_acl { facility(local5) and level(debug..emerg) and match("SEC-6-IPACCESSLOG" value("MESSAGE")) ; }; filter f_1 { facility(local5) and level(debug..emerg); }; log { source(any_udp); filter(f_1_acl); destination(routers_log); flags(final); }; log { source(any_udp); filter(f_1); destination(engedc01); }; log { source(any_udp); filter(f_1); destination(routers_log); flags(final); }; filter SonicWallNoise { match("id=firewall" value("MESSAGE")) and filter(SonicWallMsgs); }; filter SonicWallMsgs { match("m=97" value("MESSAGE")) or match("m=98" value("MESSAGE")) or match("m=537" value("MESSAGE")); }; log { source(any_udp); filter(SonicWallNoise); destination(dev_null); flags(final); }; filter f_3 { facility(local0) and level(debug..emerg); }; log { source(any_udp); filter(f_3); destination(ravlin_log); flags(final); }; log { source(any_tcp); destination(workstation_log); flags(final); }; log { source(any_udp); destination(catch-all_log); flags(final); }; Marvin Nipper Director of Security Stream Global Services mailto:marvin.nipper@stream.com ph: (303) 670-2705 PGP Key ID: 0x8EE28551 (DSS/DH) 8C5D 403A D107 0A95 672B B637 BCF1 919A 8EE2 8551 This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

5 10

[Bug 166] New: syslog-ng abort when using pgsql
by bugzilla＠bugzilla.balabit.com 31 Mar '12

31 Mar '12

https://bugzilla.balabit.com/show_bug.cgi?id=166 Summary: syslog-ng abort when using pgsql Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: whille(a)163.com Type of the Report: --- Estimated Hours: 0.0 syslog-ng aborted after running a while, several minutes or within a hour. service syslog-ng will restart process, so i'm using just process to debug: /usr/local/sbin/syslog-ng -d -F > syslogdbg.log 2>&1 last log of syslog-ng itself: Incoming log entry; line='<118>1 2012-03-02T18:54:00+08:00 signal signalingd - - [meta sequenceId="287054"] {"device_ext_ip":"172.18.195.218","to":"407edfb0-cbd5-4b5b-be63-2b3b632b0cc1","app":"d160fca0-3793-11e1-b86c-0800200c9a66","event_id":771,"chunk_type":"conf","chunk_number":1,"event_src_ip":"110.132.142.102","event_dst_ip":"110.132.142.100","devicemac":"222DB98C1600","devicename":"DCS-940","from":"F07D68078020"}\x0a' Rewrite expression evaluation result; value='.SDATA.meta.sequenceId', new_value='4' Filter rule evaluation begins; filter_rule='f_prg' Filter node evaluation result; filter_result='match' Filter rule evaluation result; filter_result='match', filter_rule='f_prg' Filter rule evaluation begins; filter_rule='f_oq0' Filter node evaluation result; filter_result='not-match', filter_type='CMP' Filter rule evaluation result; filter_result='not-match', filter_rule='f_oq0' Filter rule evaluation begins; filter_rule='f_oq1' Filter node evaluation result; filter_result='not-match', filter_type='CMP' Filter rule evaluation result; filter_result='not-match', filter_rule='f_oq1' file logqueue-fifo.c: line 388 (log_queue_fifo_ack_backlog): assertion failed: (s->parallel_push_notify == NULL) syslog-ng -V syslog-ng 3.3.4 Installer-Version: 3.3.4 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.3#master#5e44eb46b0d7b86b62f17698e2b6de875ac8d7c6 Compile-Date: Mar 5 2012 21:18:10 Default-Modules: affile,afprog,afsocket,afuser,basicfuncs,csvparser,dbparser,syslogformat,afsql Available-Modules: afmongodb,dbparser,afsocket-tls,affile,basicfuncs,csvparser,syslogformat,dummy,afsocket,convertfuncs,afprog,afuser,afsql,confgen Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: on Enable-Linux-Caps: off Enable-Pcre: off -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

[PATCH] json-parser: Change the way arrays are formatted
by Gergely Nagy 31 Mar '12

31 Mar '12

Instead of parsing an array into key-value pairs where the key has a suffix of a dot, followed by the array index, make the key resemble the javascript array syntax: array[index]. This makes it more intuitive to reach into arrays, and also makes it easier to discover arrays when formatting back to JSON. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/jsonparser/jsonparser.c | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-) diff --git a/modules/jsonparser/jsonparser.c b/modules/jsonparser/jsonparser.c index b362f49..44406da 100644 --- a/modules/jsonparser/jsonparser.c +++ b/modules/jsonparser/jsonparser.c @@ -1,6 +1,6 @@ /* - * Copyright (c) 2011 BalaBit IT Ltd, Budapest, Hungary - * Copyright (c) 2011 Gergely Nagy <algernon(a)balabit.hu> + * Copyright (c) 2011-2012 BalaBit IT Ltd, Budapest, Hungary + * Copyright (c) 2011-2012 Gergely Nagy <algernon(a)balabit.hu> * * This program is free software; you can redistribute it and/or modify it * under the terms of the GNU General Public License version 2 as published @@ -96,14 +96,13 @@ log_json_parser_process_single (struct json_object *jso, gint i, plen; g_string_assign (sb_string (key), obj_key); - g_string_append_c (sb_string (key), '.'); plen = sb_string (key)->len; for (i = 0; i < json_object_array_length (jso); i++) { g_string_truncate (sb_string (key), plen); - g_string_append_printf (sb_string (key), "%d", i); + g_string_append_printf (sb_string (key), "[%d]", i); log_json_parser_process_single (json_object_array_get_idx (jso, i), prefix, sb_string (key)->str, msg); -- 1.7.9

2 2

syslog-ng large NVTable patch
by Peter Eisenlohr 31 Mar '12

31 Mar '12

Hi, as requested by bazsi in the forums http://www.syslog.org/forum/index.php?topic=1218.0 here's a patch against syslog-ng enabling it to handle log messages larger than the current 256k limit. This is done by extending NVTable's data structures from 16 to 32 bit. The patch also contains simple code for automatic resizing of the receive buffer. 'log_msg_size' is still honoured, though. I have been using this to regularly receive multi-MB XML messages and process them into log files. I developed the patch against 3.3.1, but also tested it with 3.3.3 and on x64 as well. 'make check' runs fine, too -- but still, use at your own risk. Peter

3 6

Use multiple templates for a single file?
by Mikkel Leth Carlsen 31 Mar '12

31 Mar '12

Hi Is it somehow possible to apply multiple templates to a single destination/file? My current setup is something along the lines of: destination dst_foo_1 { file("foo_1.log" template(template_foo_1)); }; destination dst_foo_2 { file("foo_2.log" template(template_foo_2)); }; template template_foo_1 { template("$A $B $C"); }; template template_foo_2 { template("$D $E $F"); }; log { source(src_udp); filter(filter_condition_1); parser(parser_foo); destination(dst_foo_1); flags(final); }; log { source(src_udp); filter(filter_condition_2); parser(parser_foo); destination(dst_foo_2); flags(final); }; However, I would really like to write log into a single file - but with two different templates depending on the matched filter. Is that possible? I don't suppose pointing both destinations to the same file would be wise? /Mikkel

3 2

[PATCH 1/2] cfg-lexer: Split cfg_lexer_include_file() into two functions
by Gergely Nagy 31 Mar '12

31 Mar '12

To be able to keep the code simple in the future, split cfg_lexer_include_file() into two: the intent is that we'll check if the file exists, and if not, try the same thing as a glob, otherwise continue as we did before. This would nest too deep, so split out the current bulk into a separate function. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- lib/cfg-lexer.c | 59 +++++++++++++++++++++++++++++++++++------------------- 1 files changed, 38 insertions(+), 21 deletions(-) diff --git a/lib/cfg-lexer.c b/lib/cfg-lexer.c index 6143a27..cfe229f 100644 --- a/lib/cfg-lexer.c +++ b/lib/cfg-lexer.c @@ -1,6 +1,6 @@ /* - * Copyright (c) 2002-2010 BalaBit IT Ltd, Budapest, Hungary - * Copyright (c) 1998-2010 Balázs Scheidler + * Copyright (c) 2002-2012 BalaBit IT Ltd, Budapest, Hungary + * Copyright (c) 1998-2012 Balázs Scheidler * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -379,32 +379,18 @@ cfg_lexer_start_next_include(CfgLexer *self) return TRUE; } -gboolean -cfg_lexer_include_file(CfgLexer *self, const gchar *filename_) +static gboolean +cfg_lexer_include_file_simple(CfgLexer *self, gchar *filename) { - struct stat st; CfgIncludeLevel *level; - gchar *filename; + struct stat st; - if (self->include_depth >= MAX_INCLUDE_DEPTH - 1) + if (stat(filename, &st) < 0) { - msg_error("Include file depth is too deep, increase MAX_INCLUDE_DEPTH and recompile", - evt_tag_str("filename", filename_), - evt_tag_int("depth", self->include_depth), - NULL); + g_free(filename); return FALSE; } - filename = find_file_in_path(cfg_args_get(self->globals, "include-path"), filename_, G_FILE_TEST_EXISTS); - if (!filename || stat(filename, &st) < 0) - { - msg_error("Include file/directory not found", - evt_tag_str("filename", filename_), - evt_tag_str("include-path", cfg_args_get(self->globals, "include-path")), - evt_tag_errno("error", errno), - NULL); - return FALSE; - } self->include_depth++; level = &self->include_stack[self->include_depth]; level->include_type = CFGI_FILE; @@ -488,10 +474,41 @@ cfg_lexer_include_file(CfgLexer *self, const gchar *filename_) g_slist_foreach(level->file.files, (GFunc) g_free, NULL); g_slist_free(level->file.files); level->file.files = NULL; + return FALSE; } gboolean +cfg_lexer_include_file(CfgLexer *self, const gchar *filename_) +{ + struct stat st; + CfgIncludeLevel *level; + gchar *filename; + + if (self->include_depth >= MAX_INCLUDE_DEPTH - 1) + { + msg_error("Include file depth is too deep, increase MAX_INCLUDE_DEPTH and recompile", + evt_tag_str("filename", filename_), + evt_tag_int("depth", self->include_depth), + NULL); + return FALSE; + } + + filename = find_file_in_path(cfg_args_get(self->globals, "include-path"), filename_, G_FILE_TEST_EXISTS); + if (!filename || stat(filename, &st) < 0) + { + msg_error("Include file/directory not found", + evt_tag_str("filename", filename_), + evt_tag_str("include-path", cfg_args_get(self->globals, "include-path")), + evt_tag_errno("error", errno), + NULL); + return FALSE; + } + else + return cfg_lexer_include_file_simple(self, filename); +} + +gboolean cfg_lexer_include_buffer(CfgLexer *self, const gchar *name, gchar *buffer, gsize length) { CfgIncludeLevel *level; -- 1.7.9

2 3

[PATCH] afmongodb: Restore compatibility with syslog-ng 3.3
by Gergely Nagy 31 Mar '12

31 Mar '12

In syslog-ng 3.3, we had the host() and port() options for the mongodb driver, these were removed in 3.4 in favour of servers(). This patch re-adds this two. Using host() and port() will now prepend the specified combination to the server list. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/afmongodb/afmongodb-grammar.ym | 2 ++ modules/afmongodb/afmongodb-parser.c | 2 ++ modules/afmongodb/afmongodb.c | 25 +++++++++++++++++++++++++ modules/afmongodb/afmongodb.h | 2 ++ 4 files changed, 31 insertions(+), 0 deletions(-) diff --git a/modules/afmongodb/afmongodb-grammar.ym b/modules/afmongodb/afmongodb-grammar.ym index a21e23c..71dd46d 100644 --- a/modules/afmongodb/afmongodb-grammar.ym +++ b/modules/afmongodb/afmongodb-grammar.ym @@ -75,6 +75,8 @@ afmongodb_option | KW_USERNAME '(' string ')' { afmongodb_dd_set_user(last_driver, $3); free($3); } | KW_PASSWORD '(' string ')' { afmongodb_dd_set_password(last_driver, $3); free($3); } | KW_SAFE_MODE '(' yesno ')' { afmongodb_dd_set_safe_mode(last_driver, $3); } + | KW_HOST '(' string ')' { afmongodb_dd_set_host(last_driver, $3); free($3); } + | KW_PORT '(' LL_NUMBER ')' { afmongodb_dd_set_port(last_driver, $3); } | value_pair_option { afmongodb_dd_set_value_pairs(last_driver, $1); } | dest_driver_option ; diff --git a/modules/afmongodb/afmongodb-parser.c b/modules/afmongodb/afmongodb-parser.c index b8c9ead..9d72878 100644 --- a/modules/afmongodb/afmongodb-parser.c +++ b/modules/afmongodb/afmongodb-parser.c @@ -36,6 +36,8 @@ static CfgLexerKeyword afmongodb_keywords[] = { { "username", KW_USERNAME }, { "password", KW_PASSWORD }, { "safe_mode", KW_SAFE_MODE }, + { "host", KW_HOST }, + { "port", KW_PORT }, { NULL } }; diff --git a/modules/afmongodb/afmongodb.c b/modules/afmongodb/afmongodb.c index de78b23..9d75799 100644 --- a/modules/afmongodb/afmongodb.c +++ b/modules/afmongodb/afmongodb.c @@ -112,6 +112,23 @@ afmongodb_dd_set_password(LogDriver *d, const gchar *password) } void +afmongodb_dd_set_host(LogDriver *d, const gchar *host) +{ + MongoDBDestDriver *self = (MongoDBDestDriver *)d; + + g_free(self->host); + self->host = g_strdup(host); +} + +void +afmongodb_dd_set_port(LogDriver *d, gint port) +{ + MongoDBDestDriver *self = (MongoDBDestDriver *)d; + + self->port = port; +} + +void afmongodb_dd_set_servers(LogDriver *d, GList *servers) { MongoDBDestDriver *self = (MongoDBDestDriver *)d; @@ -450,6 +467,14 @@ afmongodb_dd_init(LogPipe *s) value_pairs_add_scope(self->vp, "nv-pairs"); } + if (self->host) + { + gchar *srv = g_strdup_printf ("%s:%d", self->host, + (self->port) ? self->port : 27017); + self->servers = g_list_prepend (self->servers, srv); + g_free (self->host); + } + self->host = NULL; self->port = 27017; if (!mongo_util_parse_addr(g_list_nth_data(self->servers, 0), &self->host, diff --git a/modules/afmongodb/afmongodb.h b/modules/afmongodb/afmongodb.h index 9950d26..e1f4b57 100644 --- a/modules/afmongodb/afmongodb.h +++ b/modules/afmongodb/afmongodb.h @@ -30,6 +30,8 @@ LogDriver *afmongodb_dd_new(void); void afmongodb_dd_set_servers(LogDriver *d, GList *servers); +void afmongodb_dd_set_host(LogDriver *d, const gchar *host); +void afmongodb_dd_set_port(LogDriver *d, gint port); void afmongodb_dd_set_database(LogDriver *d, const gchar *database); void afmongodb_dd_set_collection(LogDriver *d, const gchar *collection); void afmongodb_dd_set_user(LogDriver *d, const gchar *user); -- 1.7.9

2 1

[PATCH] dummy: Remove this module.
by Gergely Nagy 31 Mar '12

31 Mar '12

The dummy module does nothing, yet, is installed by default. It's not a terribly great example of a simple module, either. For these reasons and more, lets bid it a tearful farewell. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- configure.in | 1 - modules/Makefile.am | 2 +- modules/dummy/Makefile.am | 14 ------- modules/dummy/dummy-grammar.ym | 75 --------------------------------------- modules/dummy/dummy-parser.c | 48 ------------------------- modules/dummy/dummy-parser.h | 35 ------------------ modules/dummy/dummy.c | 76 ---------------------------------------- modules/dummy/dummy.h | 37 ------------------- 8 files changed, 1 insertions(+), 287 deletions(-) delete mode 100644 modules/dummy/Makefile.am delete mode 100644 modules/dummy/dummy-grammar.ym delete mode 100644 modules/dummy/dummy-parser.c delete mode 100644 modules/dummy/dummy-parser.h delete mode 100644 modules/dummy/dummy.c delete mode 100644 modules/dummy/dummy.h diff --git a/configure.in b/configure.in index 995c202..d9b50bc 100644 --- a/configure.in +++ b/configure.in @@ -1160,7 +1160,6 @@ AC_OUTPUT(dist.conf lib/Makefile syslog-ng/Makefile modules/Makefile - modules/dummy/Makefile modules/afsocket/Makefile modules/afsql/Makefile modules/afstreams/Makefile diff --git a/modules/Makefile.am b/modules/Makefile.am index bbb3b1a..983f67a 100644 --- a/modules/Makefile.am +++ b/modules/Makefile.am @@ -1 +1 @@ -SUBDIRS = afsocket afsql afstreams affile afprog afuser afmongodb afsmtp csvparser confgen syslogformat pacctformat basicfuncs dbparser tfjson tfuuid jsonparser dummy +SUBDIRS = afsocket afsql afstreams affile afprog afuser afmongodb afsmtp csvparser confgen syslogformat pacctformat basicfuncs dbparser tfjson tfuuid jsonparser diff --git a/modules/dummy/Makefile.am b/modules/dummy/Makefile.am deleted file mode 100644 index 5e90ff6..0000000 --- a/modules/dummy/Makefile.am +++ /dev/null @@ -1,14 +0,0 @@ -moduledir = @moduledir@ -AM_CPPFLAGS = -I$(top_srcdir)/lib -I../../lib -module_LTLIBRARIES = libdummy.la - -export top_srcdir - -libdummy_la_SOURCES = dummy-grammar.y dummy.c dummy.h dummy-parser.c dummy-parser.h -libdummy_la_LIBADD = $(MODULE_DEPS_LIBS) -libdummy_la_LDFLAGS = $(MODULE_LDFLAGS) - -BUILT_SOURCES = dummy-grammar.y dummy-grammar.c dummy-grammar.h -EXTRA_DIST = $(BUILT_SOURCES) dummy-grammar.ym - -include $(top_srcdir)/build/lex-rules.am diff --git a/modules/dummy/dummy-grammar.ym b/modules/dummy/dummy-grammar.ym deleted file mode 100644 index 3179e11..0000000 --- a/modules/dummy/dummy-grammar.ym +++ /dev/null @@ -1,75 +0,0 @@ -/* - * Copyright (c) 2002-2010 BalaBit IT Ltd, Budapest, Hungary - * Copyright (c) 1998-2010 Balázs Scheidler - * - * This program is free software; you can redistribute it and/or modify it - * under the terms of the GNU General Public License version 2 as published - * by the Free Software Foundation, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA - * - * As an additional exemption you are allowed to compile & link against the - * OpenSSL libraries as published by the OpenSSL project. See the file - * COPYING for details. - * - */ - -%code requires { - -#include "dummy-parser.h" - -} - -%code { - -#include "cfg-parser.h" -#include "dummy-grammar.h" -#include "plugin.h" - -extern LogDriver *last_driver; - -} - -%name-prefix "dummy_" -%lex-param {CfgLexer *lexer} -%parse-param {CfgLexer *lexer} -%parse-param {LogDriver **instance} -%parse-param {gpointer arg} - - -/* INCLUDE_DECLS */ - -%token KW_DUMMY -%token KW_DUMMY_OPT - -%% - -start - : LL_CONTEXT_DESTINATION KW_DUMMY - { - last_driver = *instance = dummy_dd_new(); - } - '(' dummy_options ')' { YYACCEPT; } - ; - -dummy_options - : dummy_option dummy_options - | - ; - -dummy_option - : KW_DUMMY_OPT '(' yesno ')' { ((DummyDestDriver *)last_driver)->opt = $3; }; - | dest_driver_option - ; - - -/* INCLUDE_RULES */ - -%% diff --git a/modules/dummy/dummy-parser.c b/modules/dummy/dummy-parser.c deleted file mode 100644 index 2951563..0000000 --- a/modules/dummy/dummy-parser.c +++ /dev/null @@ -1,48 +0,0 @@ -/* - * Copyright (c) 2002-2010 BalaBit IT Ltd, Budapest, Hungary - * Copyright (c) 1998-2010 Balázs Scheidler - * - * This program is free software; you can redistribute it and/or modify it - * under the terms of the GNU General Public License version 2 as published - * by the Free Software Foundation, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA - * - * As an additional exemption you are allowed to compile & link against the - * OpenSSL libraries as published by the OpenSSL project. See the file - * COPYING for details. - * - */ - -#include "dummy.h" -#include "cfg-parser.h" -#include "dummy-grammar.h" - -extern int dummy_debug; -int dummy_parse(CfgLexer *lexer, DummyDestDriver **instance, gpointer arg); - -static CfgLexerKeyword dummy_keywords[] = { - { "dummy", KW_DUMMY }, - { "dummy_opt", KW_DUMMY_OPT }, - { NULL } -}; - -CfgParser dummy_parser = -{ -#if ENABLE_DEBUG - .debug_flag = &dummy_debug, -#endif - .name = "dummy", - .keywords = dummy_keywords, - .parse = (int (*)(CfgLexer *lexer, gpointer *instance, gpointer arg)) dummy_parse, - .cleanup = (void (*)(gpointer)) log_pipe_unref, -}; - -CFG_PARSER_IMPLEMENT_LEXER_BINDING(dummy_, LogDriver **) diff --git a/modules/dummy/dummy-parser.h b/modules/dummy/dummy-parser.h deleted file mode 100644 index 37a9e8f..0000000 --- a/modules/dummy/dummy-parser.h +++ /dev/null @@ -1,35 +0,0 @@ -/* - * Copyright (c) 2002-2010 BalaBit IT Ltd, Budapest, Hungary - * Copyright (c) 1998-2010 Balázs Scheidler - * - * This program is free software; you can redistribute it and/or modify it - * under the terms of the GNU General Public License version 2 as published - * by the Free Software Foundation, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA - * - * As an additional exemption you are allowed to compile & link against the - * OpenSSL libraries as published by the OpenSSL project. See the file - * COPYING for details. - * - */ - -#ifndef DUMMY_PARSER_H_INCLUDED -#define DUMMY_PARSER_H_INCLUDED - -#include "cfg-parser.h" -#include "cfg-lexer.h" -#include "dummy.h" - -extern CfgParser dummy_parser; - -CFG_PARSER_DECLARE_LEXER_BINDING(dummy_, LogDriver **) - -#endif diff --git a/modules/dummy/dummy.c b/modules/dummy/dummy.c deleted file mode 100644 index f27667b..0000000 --- a/modules/dummy/dummy.c +++ /dev/null @@ -1,76 +0,0 @@ -/* - * Copyright (c) 2002-2010 BalaBit IT Ltd, Budapest, Hungary - * Copyright (c) 1998-2010 Balázs Scheidler - * - * This program is free software; you can redistribute it and/or modify it - * under the terms of the GNU General Public License version 2 as published - * by the Free Software Foundation, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA - * - * As an additional exemption you are allowed to compile & link against the - * OpenSSL libraries as published by the OpenSSL project. See the file - * COPYING for details. - * - */ - -#include "dummy.h" -#include "dummy-parser.h" -#include "plugin.h" -#include "messages.h" - -static void -dummy_dd_queue(LogPipe *s, LogMessage *msg, const LogPathOptions *path_options, gpointer user_data) -{ - DummyDestDriver *self = (DummyDestDriver *) s; - msg_notice("Dummy plugin received a message", - evt_tag_str("msg", log_msg_get_value(msg, LM_V_MESSAGE, NULL)), - evt_tag_int("opt", self->opt), - NULL); - - log_dest_driver_queue_method(s, msg, path_options, user_data); -} - -LogDriver * -dummy_dd_new(void) -{ - DummyDestDriver *self = g_new0(DummyDestDriver, 1); - - log_dest_driver_init_instance(&self->super); - self->super.super.super.queue = dummy_dd_queue; - - return &self->super.super; -} - -extern CfgParser dummy_dd_parser; - -static Plugin dummy_plugin = -{ - .type = LL_CONTEXT_DESTINATION, - .name = "dummy", - .parser = &dummy_parser, -}; - -gboolean -dummy_module_init(GlobalConfig *cfg, CfgArgs *args) -{ - plugin_register(cfg, &dummy_plugin, 1); - return TRUE; -} - -const ModuleInfo module_info = -{ - .canonical_name = "dummy", - .version = VERSION, - .description = "The dummy module is a sample destination driver not really doing any kind of useful work.", - .core_revision = SOURCE_REVISION, - .plugins = &dummy_plugin, - .plugins_len = 1, -}; diff --git a/modules/dummy/dummy.h b/modules/dummy/dummy.h deleted file mode 100644 index d99c7bf..0000000 --- a/modules/dummy/dummy.h +++ /dev/null @@ -1,37 +0,0 @@ -/* - * Copyright (c) 2002-2010 BalaBit IT Ltd, Budapest, Hungary - * Copyright (c) 1998-2010 Balázs Scheidler - * - * This program is free software; you can redistribute it and/or modify it - * under the terms of the GNU General Public License version 2 as published - * by the Free Software Foundation, or (at your option) any later version. - * - * This library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public - * License along with this library; if not, write to the Free Software - * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA - * - * As an additional exemption you are allowed to compile & link against the - * OpenSSL libraries as published by the OpenSSL project. See the file - * COPYING for details. - * - */ - -#ifndef DUMMY_H_INCLUDED -#define DUMMY_H_INCLUDED - -#include "driver.h" - -typedef struct -{ - LogDestDriver super; - gint opt; -} DummyDestDriver; - -LogDriver *dummy_dd_new(void); - -#endif -- 1.7.9

2 1

[PATCH 0/1]: afmongodb: queue & wakeup race condition
by Gergely Nagy 31 Mar '12

31 Mar '12

There is a way to deadlock the mongodb destination, and I don't entirely understand the scenario, and thus, I can't properly explain it, either. Perhaps I'd need more coffee... but anyway! Following this mail, I'll send a patch that fixes the issue for me (I separately sent the same patch to the original reporter of the problem, too), but I'm not entirely happy with it. Nevertheless, I still find it better to send it to the list, where it can - perhaps - receive more testing than I was able to give it.

2 2

[PATCH (3.4) 1/2] configure: Fix --with-json=json-c/json-glib handling.
by Gergely Nagy 31 Mar '12

31 Mar '12

The tests that decide whether the user wanted to select json-c or json-glib were a little bit off, which resulted in anything other than --with-json=auto failing. This has now been corrected. Reported-by: Peter Czanik <czanik(a)balabit.hu> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- configure.in | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/configure.in b/configure.in index 995c202..f766888 100644 --- a/configure.in +++ b/configure.in @@ -606,11 +606,11 @@ fi dnl *************************************************************************** dnl json headers/libraries dnl *************************************************************************** -if test "x$with_json" = "xauto" || test "x$with_json" = "json-c"; then - PKG_CHECK_MODULES(JSON_C, json >= $JSON_C_MIN_VERSION,, JSON_C_LIBS="") +if test "x$with_json" = "xauto" || test "x$with_json" = "xjson-c"; then + PKG_CHECK_MODULES(JSON_C, json >= $JSON_C_MIN_VERSION) fi -if test "x$with_json" = "xauto" || test "x$with_json" = "json-glib"; then - PKG_CHECK_MODULES(JSON_GLIB, json-glib-1.0 >= $JSON_GLIB_MIN_VERSION,, JSON_GLIB_LIBS="") +if test "x$with_json" = "xauto" || test "x$with_json" = "xjson-glib"; then + PKG_CHECK_MODULES(JSON_GLIB, json-glib-1.0 >= $JSON_GLIB_MIN_VERSION) fi dnl *************************************************************************** -- 1.7.9

2 4

[PATCH (3.4)] modules: Do not build or install disabled modules
by Gergely Nagy 31 Mar '12

31 Mar '12

The afmongodb and afsmtp modules had a subtle bug in their Makefile.ams: even if they were disabled, the Makefile still built and installed an empty module. This patch corrects that, by moving the module_LTLIBRARIES inside the if block. Reported-by: Peter Czanik <czanik(a)balabit.hu> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/afmongodb/Makefile.am | 5 ++--- modules/afsmtp/Makefile.am | 2 +- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/modules/afmongodb/Makefile.am b/modules/afmongodb/Makefile.am index c0eec33..d022912 100644 --- a/modules/afmongodb/Makefile.am +++ b/modules/afmongodb/Makefile.am @@ -1,14 +1,13 @@ - SUBDIRS = @LIBMONGO_SUBDIRS@ DIST_SUBDIRS = libmongo-client moduledir = @moduledir@ AM_CPPFLAGS = -I$(top_srcdir)/lib -I../../lib -module_LTLIBRARIES = libafmongodb.la export top_srcdir if ENABLE_MONGODB +module_LTLIBRARIES = libafmongodb.la libafmongodb_la_CFLAGS = $(LIBMONGO_CFLAGS) libafmongodb_la_SOURCES = afmongodb-grammar.y afmongodb.c afmongodb.h afmongodb-parser.c afmongodb-parser.h @@ -32,4 +31,4 @@ uninstall: $(MAKE) $(AM_MAKEFLAGS) uninstall-am check: - echo "Make check disabled, since it requires a newer glib" \ No newline at end of file + echo "Make check disabled, since it requires a newer glib" diff --git a/modules/afsmtp/Makefile.am b/modules/afsmtp/Makefile.am index 790aad7..bb5b94d 100644 --- a/modules/afsmtp/Makefile.am +++ b/modules/afsmtp/Makefile.am @@ -1,10 +1,10 @@ moduledir = @moduledir@ AM_CPPFLAGS = -I$(top_srcdir)/lib -I../../lib -module_LTLIBRARIES = libafsmtp.la export top_srcdir if ENABLE_SMTP +module_LTLIBRARIES = libafsmtp.la libafsmtp_la_CFLAGS = $(LIBESMTP_CFLAGS) $(IVYKIS_CFLAGS) $(EVENTLOG_CFLAGS) $(SYSLOG_NG_CFLAGS) libafsmtp_la_SOURCES = afsmtp-grammar.y afsmtp.c afsmtp.h afsmtp-parser.c afsmtp-parser.h -- 1.7.9

2 1

Reprocess disk buffer files
by Aaron Daniel 31 Mar '12

31 Mar '12

In the LTS version of PE, we have some servers that have old disk-buffer files (var/*qf) still on disk, with the contents not sent to a central server. Is there a way to force syslog-ng to re-process those, or do we have to manually process them ourselves? The files contain very old log entries that still need to be processed, and syslog-ng appears to be using a newer *.qf file for it's disk buffer. When it does this, it's completely ignoring the old log entries so we have some gaps in our central processing system. (Sorry if this has been asked before, google's not been my friend on this search...) -- Aaron Daniel

2 1

High volume of syslog message getting dropped
by Samuel Petreski 31 Mar '12

31 Mar '12

I'm running syslog-ng 3.3.4 on FreeBSD 8.2, but am getting choppy logging from syslog-ng. I'm continually (every minute or so) getting the following messages from syslog-ng, Mar 19 17:09:24 system syslog-ng[47183]: Syslog connection accepted; fd='51', client='AF_INET(10.33.11.249:24813)', local='AF_INET(0.0.0.0:1470)' Mar 19 17:09:25 system syslog-ng[47183]: Syslog connection closed; fd='51', client='AF_INET(10.33.11.249:24813)', local='AF_INET(0.0.0.0:1470)' Mar 19 17:09:46 system syslog-ng[47183]: Syslog connection accepted; fd='25', client='AF_INET(10.166.166.234:50240)', local='AF_INET(0.0.0.0:1470)' Mar 19 17:09:47 system syslog-ng[47183]: Syslog connection closed; fd='25', client='AF_INET(10.166.166.234:50240)', local='AF_INET(0.0.0.0:1470)' Most of my hosts sending their logs to syslog-ng are Cisco ASAs and they are configured to send their logs securely. Also these devices send syslog messages at a very high rate (few of the devices generate logs in excess of 10GB/day). My relevant syslog-ng paramenters are as follows, **************** #global options options { chain_hostnames(off); keep_hostname(yes); use_fqdn(off); flush_lines(0); stats_freq(360); threaded(yes); dns_cache(2000); dns_cache_expire(87600); flush_lines(10000000); flush_timeout(6000); }; #destination, source and logging destination d_cisco_devices { file("/data/log/$HOST-$YEAR$MONTH$DAY.log" perm(0400)); }; source tls_clients { tcp(ip(0.0.0.0) port(1470) max-connections(300) log_iw_size(300000000) log_fetch_limit(2000000) tls( key_file("/etc/syslog-ng.key") cert_file("/etc/syslog-ng.crt") peer_verify(optional-untrusted)) ); }; log { source(tls_clients); destination(d_cisco_devices); flags(flow-control); }; **************** I've read through the manual and have tried to increase the flush_line, log_fetch_limit, log_iw_size parameters but have not been able to fine tune its operation. Most of the my recorded syslog messages are for about few seconds, and then they start again the next minute. Any help or pointers anyone can provide would be greatly appreciated. Thanks. --Samuel

2 1

allow setting up unix named sockets for listening
by Patrick Hemmer 30 Mar '12

30 Mar '12

When troubleshooting issues with our production syslog servers, its often not possible to deliberately duplicate discovered issues as it then affects production. Something which would be extremely useful in this case is if we could configure syslog-ng to set up a unix socket as a listener. Then whenever clients connect logs can be sent to the client. Syslog-ng currently has unix-stream and unix-dgram destinations, but they act as clients, only sending to already created sockets. There could be other uses for this, basically allowing clients to connect whenever they want and receive the logs. Kind of like a pipe destination, but it works a bit cleaner since if no client is connected, syslog-ng wouldnt have to buffer and drop messages (and it would allow multiple clients). Fairly low priority, but just something that I thought would be useful. -Patrick

2 1

syslog-ng Insider - March 2012
by Peter Czanik 29 Mar '12

29 Mar '12

Dear syslog-ng users, This is the 12th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news. Your feedback and news tips about the next issue is welcome at documentation(a)balabit.com <mailto:documentation@balabit.com> FEATURED NEWS GSoC wants you to code syslog-ng in the summer ---------------------------------------------- GSoC ( http://code.google.com/soc/ ) is a nice opportunity for higher education students to spend their summers productively by coding in open source software projects. This time BalaBit participates in GSoC with the help of the openSUSE project. If you are interested in enhancing syslog-ng or Zorp, please see our project ideas on the openSUSE ideas page: http://en.opensuse.org/openSUSE:GSOC_ideas#syslog-ng and http://en.opensuse.org/openSUSE:GSOC_ideas#Zorp Alpha1 of syslog-ng 3.4 is released ----------------------------------- The first alpha version of syslog-ng 3.4 is released. Major new features are junctions & channels which add even more flexibility to the syslog-ng configuration. There is now also a json parser, smtp destination and modules are now loaded automatically. For more details on what is new, please check Bazsi's blog: http://bazsi.blogs.balabit.com/2012/03/first-alpha-release-of-syslog-ng-3-4… Instead of using the release, it is recommended to use sources from git, which have some major stability fixes: https://github.com/bazsi/syslog-ng-3.4 If you intend to package syslog-ng 3.4, it's recommended to check the mailing list for patches from Algernon, which make packaging easier. A snapshot of his work is available at: http://packages.madhouse-project.org/syslog-ng/algernon/3.4/syslog-ng-alger… There are already packages for openSUSE and an updated syslog-ng-devel port for FreeBSD. See http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The (r)evolution of name value pairs ------------------------------------ Name value pairs were at the heart of syslog-ng even before PatternDB made it obvious. And now the CEE board and the Lumberjack project also push into this direction: instead of free form text messages, use name value pairs for logging. Recent developments in syslog-ng also serve this purpose: v3.3 can output name value pairs in JSON and v3.4 will be able to parse these logs and turn them into name value pairs again. How project Lumberjack can improve logging: http://bazsi.blogs.balabit.com/2012/02/project-lumberjack-to-improve-linux-… How syslog-ng can be used for CEE (JSON) logs: http://algernon.blogs.balabit.com/2012/02/cee-handling-with-syslog-ng/ How using name value pairs can improve logging: http://czanik.blogs.balabit.com/2012/03/state-of-the-art-logging-syslog-ng-… which is also the basis for our syslog-ng presentation at the LOADays conference: http://loadays.org/ syslog-ng community forum ------------------------- BalaBit has had a very positive experience with the syslog-ng community and we believe that your feedback has played a key role in the success of syslog-ng. We have decided to adopt this community model to our other products so that we can have our customers involved in product design. It could be interesting for you to visit this brand new community site at http://communities.balabit.com <http://communities.balabit.com/>. You are one of the first of our friends to be invited. OTHER SHORT NEWS * An introduction to RLTP, a protocol to make syslog-ng PE even more reliable: http://pzolee.blogs.balabit.com/2012/03/zero-message-loss-with-syslog-ng-pr… * The libumberlog library is a thin, LD_PRELOAD-able layer on top of the legacy syslog() function, that turns those calls into something that emits its message part as a JSON formatted structured log message:http://algernon.github.com/libumberlog/ * ELSA, the high performance web GUI for syslog-ng, received a few interesting updates to make resolving security incidents even more quick and efficient. For the latest examples check: http://ossectools.blogspot.com/ NEW RELEASES: * syslog-ng OSE 3.4 alpha1: http://bazsi.blogs.balabit.com/2012/03/first-alpha-release-of-syslog-ng-3-4… * syslog-ng PE 4F2: http://andrea.blogs.balabit.com/2012/02/balabits-new-syslog-ng-premium-edit… * lumberlog 0.1.0: http://algernon.github.com/libumberlog/ ARCHIVE http://insider.blogs.balabit.com/ -- Peter Czanik (CzP)<czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

IP Address folder
by Julien Groselle 28 Mar '12

28 Mar '12

Hello everybody, I work for a lange society, and we use syslog-ng for 5 years now. We have a centralized server with storage tank to keep logs securly. Concretely, we have 2 centralized servers syslog-ng in chrooted environement, and 50 clients servers. Since we use TLS transport in place of stunnel workaround, we have many issues : - First of all, many logs aren't writen in $HOST folder but in IPADDRESS folder. So, to be clear, this is an exemple : # ls drwxr-x--- 8 root adm 4,0K 1 mars 00:07 10.0.0.1 drwxr-x--- 53 root adm 4,0K 19 mars 00:35 host1 I assume that host1 have 10.0.0.1 IP address and # tree 192.168.100.79/2012-03/ 10.0.0.1/2012-03/ ├── 02-user-10.0.0.1.log.bz2 ├── 06-user-10.0.0.1.log.bz2 ├── 07-user-10.0.0.1.log.bz2 ├── 08-user-10.0.0.1.log.bz2 ├── 09-user-10.0.0.1.log.bz2 ├── 12-user-10.0.0.1.log.bz2 ├── 13-user-10.0.0.1.log.bz2 ├── 14-user-10.0.0.1.log.bz2 ├── 15-user-10.0.0.1.log.bz2 ├── 16-user-10.0.0.1.log.bz2 └── 19-user-10.0.0.1.log # tree host1/2012-03/ |grep 19- ├── 19-apache.access-host1.log ├── 19-apache.error-host1.log ├── 19-authpriv-host1.log ├── 19-auth-host1.log ├── 19-cron-host1.log ├── 19-daemon-host1.log ├── 19-kern-host1.log ├── 19-mail-host1.log ├── 19-nagios-host1.log ├── 19-puppetd-host1.log ├── 19-syslog-host1.log └── 19-user-host1.log (we have this problem with many servers) In facility "user" for host 10.0.0.1 in fact i have log for snmptrapd... But why ?? We have config for snmpd but not for snmptrapd... So i have tried to define a default facility => failed After i have tried many dns and hostnames options => failed As anyone here have a way to search for me ? If you need more details, i'm your's. Kind regards. -- JG

1 1