October 2011 - syslog-ng - lists.balabit.hu

Output to file in /proc
by Valentijn Sessink 13 Dec '13

13 Dec '13

Hello list, As most of the functionality of "sshguard", "fail2ban" and others is in syslog-ng, I'm trying to use syslog-ng to add entries to my Linux kernel based firewall (iptables). I'm almost ready to implement one, but I have a question about undeliverable log lines: what happens if a certain file() destination blocks? Will syslog-ng retry to deliver the same message, or will it lose the message? What I'm doing is using the "ipt_recent" module; with this module, one can check if a certain IP address was seen before. For example, the following rule will block you if you were seen 5 or more times ("hitcount 5") in the "violators" list last minute ("seconds 60): iptables -A INPUT -m recent \ --rcheck --hitcount 5 --seconds 60 --name violators -j DROP The fun part is, that you can add IP adresses by stuffing them into a file in /proc: echo '+127.0.0.1' > /proc/net/xt_recent/violators ... will add 127.0.0.1 to the "violators" with the current time. However: echo 'whatever' > /proc/net/xt_recent/violators ... gives an error: Invalid argument (22) As far as I can see, syslog-ng will not try again to deliver the same message; but is this by design? I.e. can I trust syslog-ng to not "block" because of a single malformed IP address? Because then, an implementation of sshguard could be simply swapped with a simple: destination df_ipt_recent { file ("/proc/net/xt_recent/blocker" template("+${usracct.device}\n") ); }; filter f_ipt_recent { tags("secevt") and match("REJECT" value("secevt.verdict")); }; log { source (s_all); parser(pattern_db); filter(f_addtofirewall);destination(df_addtofirewall);}; ... which would be cool! Best regards, Valentijn

6 26

[Bug 121] New: After service syslog-ng reload, the first log don' t transfer to remote servicer via TLS
by bugzilla＠bugzilla.balabit.com 08 Feb '13

08 Feb '13

https://bugzilla.balabit.com/show_bug.cgi?id=121 Summary: After service syslog-ng reload, the first log don't transfer to remote servicer via TLS Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Linux Status: NEW Severity: critical Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: yong.zheng(a)nsn.com Type of the Report: bug Estimated Hours: 0.0 The version:syslog-ng 3.0.5 options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (yes); use_fqdn (no); create_dirs (no); keep_hostname (yes); stats_freq (3600); }; I will upload the logs to remote syslog-ng server via TLS, I encounter a issue, On client: after reload the configure via below command: #service syslog-ng reload. I write a log: #logger -p local1.info "this is test" but I don't see the log to transfer to server immediately, I wait many minutes, still don't see. I must write the second log to triger that client send the 2 logs to server #logger -p local1.info "this is test 2" the 2 logs have transfered to server immediately successfully, I can see the 2 logs on server. It is 100% reproduced. Is this a known issue, or not ? Thanks. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

[Bug 42] New: capabilities, chown, chmod
by bugzilla＠bugzilla.balabit.com 20 Dec '12

20 Dec '12

https://bugzilla.balabit.com/show_bug.cgi?id=42 Summary: capabilities, chown, chmod Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: zbyniu(a)pld-linux.org Type of the Report: bug Estimated Hours: 0.0 Let's take a look at syslog-ng-3.0.1/src/affile.c lines 60-83 1. CAP_SYS_ADMIN is needed only for /proc/kmsg, it is added w/o check 2. CAP_DAC_READ_SEARCH should be added only if open fail with errno 13 2a. CAP_DAC_OVERRIDE should be added only if open fail with errno 13 and with CAP_DAC_READ_SEARCH set 3. fchown needs CAP_CHOWN unconditionaly 4. fchmod needs CAP_FOWNER if file owner != euid (root here) 5. all caps should be restored summary: - CAP_SYS_ADMIN and CAP_DAC_OVERRIDE are set always even if unnecessary, and permanently - owner, group and perm doesn't work -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

2 10

kernel logging feature requests
by Patrick H. 01 Feb '12

01 Feb '12

While setting up a new server at home I've come across 2 feature requests that would be fairly nice to have (and not that hard to implement I would think). 1) When the 'kernel' flag is set on a file() source (like for reading /proc/kmsg), look for the printk time (eg "[ 1234.567890]") and calculate when the message would have occurred instead of just using when the line was read off the file. Basically check to see the number of seconds the system has been up, subtract the printk time, and then subtract that from current time. 2) I grab all kernel messages with priority of crit or higher and send it to the usertty() destination, but this destination doesnt support templates. It'd be nice to be able to define the template. I mostly just want to change the time format and remove the hostname (since these will only come from localhost on my setup).

4 8

[Bug 142] New: --worker-threads=0 support
by bugzilla＠bugzilla.balabit.com 25 Jan '12

25 Jan '12

https://bugzilla.balabit.com/show_bug.cgi?id=142 Summary: --worker-threads=0 support Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: arekm(a)maven.pl Type of the Report: enhancement Estimated Hours: 0.0 Hello, I was hoping that threading support would solve my problem described in https://bugzilla.balabit.com/show_bug.cgi?id=113. Unfortunately it looks like it is impossible to create configuration for which each target will use one thread by default (as only such configuration is safe in case of some targets blocking on I/O). This feature request is about adding some option like: --worker-threads=0 that would limit number of threads to numer of sources + number of destinations, so each possible blocking I/O would be happening in separate thread. Yes, it would be more than cpu cores but this configuration is about reliability and not performance. (right now I have only one, network logging target that needs to always work, so probably disabling threading globally and enabling threading only for this network target would get me "always working" logging via net, even if some infinite I/O blocking happens. What would be best is to have ability to have such config on by default for all targets) -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 12

patterndb & not describing the entire log message
by Christophe Brocas 22 Dec '11

22 Dec '11

Hello As I said before in other message, I currently work with patterndb to write a pattern file describing Windows log messages related to file system. These messages are sent to Syslog-NG through Snare software. Example : Aug 12 15:08:48 10.0.0.1 MSWinEventLog 1 Security 541 ven. août 12 15:08:46 201 4656 Microsoft-Windows-Security-Auditing D00000001\accout1 N/A Success Audit hostname.d00000001.subdom1.dom1.fr Système de fichiers Un handle vers un objet a été demandé. Sujet : ID de sécurité : S-1-5-21-514083418-1667312537-733111063-1179 Nom du compte : account1 Domaine du compte : D00000001 ID d’ouverture de session : 0x8e90858 Objet : Serveur de l’objet : Security Type d’objet : File Nom de l’objet : E:\xxx\yyyy\filename.doc.doc ID du handle : 0x2b0 Informations sur le processus : ID du processus : 0xeb0 Nom du processus : C:\Windows\System32\inetsrv\w3wp.exe Informations sur la demande d’accès : ID de la transaction : {00000000-0000-0000-0000-000000000000} Accès : DELETE SYNCHRONIZE ReadAttributes Masque d’accès : 0x110080 Privilèges utilisés pour les vérifications d’accès : - Nombre de SID restreints : 0 540 Question : That kind of messages are extremely long and some areas seem empty in my example but may be not in the future. Is there a way with pattern to not describe the entire message but only the relevant part for us (the equivalent of wildcard in regexp) ? Thanks Christophe ***************************************************** "Le contenu de ce courriel et ses éventuelles pièces jointes sont confidentiels. Ils s'adressent exclusivement à la personne destinataire. Si cet envoi ne vous est pas destiné, ou si vous l'avez reçu par erreur, et afin de ne pas violer le secret des correspondances, vous ne devez pas le transmettre à d'autres personnes ni le reproduire. Merci de le renvoyer à l'émetteur et de le détruire. Attention : L'organisme de l'émetteur du message ne pourra être tenu responsable de l'altération du présent courriel. Il appartient au destinataire de vérifier que les messages et pièces jointes reçus ne contiennent pas de virus. Les opinions contenues dans ce courriel et ses éventuelles pièces jointes sont celles de l'émetteur. Elles ne reflètent pas la position de l'organisme sauf s'il en est disposé autrement dans le présent courriel." ******************************************************

4 7

An interesting assert in 3.3.1
by Gergely Nagy 01 Dec '11

01 Dec '11

A while ago I noticed that my syslog-ng 3.3 segfaults every time I boot up my desktop machine at home (but then it gets restarted by the supervisor), and once its booted up, I can't reproduce the problem anymore. I assume it has something to do with the system being in a reasonably early state, or the load being particularly high. While I didn't have time to debug it further, I'd like to record the current state of investigation here: Core was generated by `/usr/sbin/syslog-ng -p /var/run/syslog-ng.pid --enable-core'. Program terminated with signal 6, Aborted. #0 0x00007fd927cc6405 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. in ../nptl/sysdeps/unix/sysv/linux/raise.c (gdb) bt #0 0x00007fd927cc6405 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x00007fd927cc9680 in *__GI_abort () at abort.c:92 #2 0x00007fd928ae6e31 in g_assertion_message (domain=<optimized out>, file=<optimized out>, line=<optimized out>, func=0x7fd929609dc0 "log_writer_work_perform", message=0x7fd920007a60 "assertion failed: ((self->super.flags & PIF_INITIALIZED) != 0)") at /tmp/buildd/glib2.0-2.28.6/./glib/gtestutils.c:1358 #3 0x00007fd928ae73d0 in g_assertion_message_expr (domain=0x0, file=0x7fd929609acb "../../../lib/logwriter.c", line=128, func=0x7fd929609dc0 "log_writer_work_perform", expr=<optimized out>) at /tmp/buildd/glib2.0-2.28.6/./glib/gtestutils.c:1369 #4 0x00007fd9295d3570 in log_writer_work_perform (s=0x1d68080) at ../../../lib/logwriter.c:128 #5 0x00007fd9295d3897 in log_writer_io_flush_output (s=0x1d68080) at ../../../lib/logwriter.c:198 #6 0x00007fd9295ff986 in iv_run_active_list (active=0x7fff3da27700) at ../../../../../lib/ivykis/lib/iv_main.c:226 #7 0x00007fd9295ffb0d in iv_main () at ../../../../../lib/ivykis/lib/iv_main.c:269 #8 0x00007fd9295d8082 in main_loop_run () at ../../../lib/mainloop.c:722 #9 0x0000000000401a22 in main (argc=1, argv=0x7fff3da27878) at ../../../syslog-ng/main.c:260 (gdb) The particular writer is a program destination (sshguard), and at that point, it's probably not running yet. (For the record, I don't have threaded(yes) in my config at the moment). I'll look into this further, as time permits. -- |8]

1 1

[Bug 111] New: RPM build errors: Installed (but unpackaged) file(s) found:
by bugzilla＠bugzilla.balabit.com 14 Nov '11

14 Nov '11

https://bugzilla.balabit.com/show_bug.cgi?id=111 Summary: RPM build errors: Installed (but unpackaged) file(s) found: Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: packaging AssignedTo: folti(a)balabit.hu ReportedBy: owen.mann(a)interactivedata.com Type of the Report: bug Estimated Hours: 0.0 when attempting `rpmbuild -tb syslog-ng_3.2beta1.tar.gz`, the compilation completes, but: [...] Checking for unpackaged file(s): /usr/lib/rpm/check-files /var/tmp/syslog-ng-root error: Installed (but unpackaged) file(s) found: /bin/pdbtool /bin/update-patterndb /etc/syslog-ng/modules.conf /etc/syslog-ng/scl.conf /lib/libsyslog-ng.la /lib/libsyslog-ng.so /lib/libsyslog-ng.so.0 /lib/libsyslog-ng.so.0.0.0 /lib/syslog-ng/libaffile.la /lib/syslog-ng/libaffile.so /lib/syslog-ng/libafprog.la /lib/syslog-ng/libafprog.so /lib/syslog-ng/libafsocket-notls.la /lib/syslog-ng/libafsocket-notls.so /lib/syslog-ng/libafsocket-tls.la /lib/syslog-ng/libafsocket-tls.so /lib/syslog-ng/libafsocket.so /lib/syslog-ng/libafuser.la /lib/syslog-ng/libafuser.so /lib/syslog-ng/libbasicfuncs.la /lib/syslog-ng/libbasicfuncs.so /lib/syslog-ng/libconfgen.la /lib/syslog-ng/libconfgen.so /lib/syslog-ng/libconvertfuncs.la /lib/syslog-ng/libconvertfuncs.so /lib/syslog-ng/libcsvparser.la /lib/syslog-ng/libcsvparser.so /lib/syslog-ng/libdbparser.la /lib/syslog-ng/libdbparser.so /lib/syslog-ng/libdummy.la /lib/syslog-ng/libdummy.so /lib/syslog-ng/libsyslogformat.la /lib/syslog-ng/libsyslogformat.so /sbin/syslog-ng-ctl /share/include/scl/pacct/plugin.conf /share/include/scl/syslogconf/README /share/include/scl/syslogconf/convert-syslogconf.awk /share/include/scl/syslogconf/plugin.conf /share/include/scl/system/generate-system-source.sh /share/include/scl/system/plugin.conf /share/xsd/patterndb-1.xsd /share/xsd/patterndb-2.xsd /share/xsd/patterndb-3.xsd [...] If these files are added to the %files section in syslog-ng.spec, the build completes successfully. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 9

Possible memleak in 3.3 HEAD
by Jakub Jankowski 12 Nov '11

12 Nov '11

Hi, I think I'm having a memleak in 3.3's HEAD. My production system, which receives ~300 messages/s throughout the day (so ~26.4 million messages daily) leaks ~700MB of memory per day. I did some investigation, here are the details: # syslog-ng -V syslog-ng 3.3.0beta2 Installer-Version: 3.3.0beta2 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.3#no_branch#d6eea6a1130f8343b6600976d9f2319f9ed86c88 Compile-Date: Sep 9 2011 16:32:59 Default-Modules: affile,afprog,afsocket,afuser,basicfuncs,csvparser,dbparser,syslogformat,afsql Available-Modules: syslogformat,affile,afmongodb,basicfuncs,csvparser,dbparser,afsql,convertfuncs,afprog,confgen,afsocket,afuser,afsocket-tls,dummy Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: on Enable-Linux-Caps: on Enable-Pcre: off # My build is patched with: - https://github.com/algernon/libmongo-client/commit/93c5cff05feec5d7f421acfd… (so it compiles on rhel5) - https://lists.balabit.hu/pipermail/syslog-ng/2011-April/016375.html (I need this feature) I don't think any of these is the reason, though. My test configuration is basically this: $ cat /etc/syslog-ng/syslog-ng.conf @version: 3.3 options { time_reap (1); log_fifo_size (100000); use_dns (no); use_fqdn (no); create_dirs (yes); keep_hostname (yes); log_msg_size(16384); owner ("root"); group ("root"); perm (0644); dir_owner ("root"); dir_group ("root"); dir_perm (0755); }; source s_www { syslog(ip(0.0.0.0) port(515) flags(no-multi-line) log_iw_size(6000) max-connections(30) tags("src.syslog", "src.port.515", "www", "www.accesslog")); }; filter f_svc_www_accesslog { tags("www.accesslog"); }; parser p_apache_split_vhost_access { csv-parser( columns("APACHE.ACCESS.VHOST", "APACHE.ACCESS.LOG_MESSAGE") flags(greedy) delimiters(" ") ); }; parser dbp_apache_split_vhost_access { db-parser( file("/etc/syslog-ng/pdb1.xml") ); }; destination d_access_split { file("/logs/$S_YEAR/$S_MONTH/$S_DAY/${APACHE.ACCESS.VHOST:-default}-access.log-${S_YEAR}${S_MONTH}${S_DAY}" template("${APACHE.ACCESS.LOG_MESSAGE}\n")); }; log { source(s_www); filter(f_svc_www_accesslog); parser(dbp_apache_split_vhost_access); destination(d_access_split); flags(final); }; $ Those two parsers (csv-parser and db-parser) are doing exactly the same, they split my Apache HTTPd access logs into two parts: first column (which is the vhost name) and the rest of the line. I have them both because I needed to know if that's a leak in any particular parser type. (Apparently not) I'm not very experienced with valgrind, but I did some tests, and what I've found is that the number of "definitely lost" blocks depends on the number of files created by d_access_split destination. So I took 10000 lines of some real httpd logs and did two tests: 1) replace real vhost names with completely random strings, to maximize number of files created by d_access_split 2) replace real vhost names with constant string, so my destination always writes to the same file. And this is what I got, using "G_SLICE=always-malloc valgrind --leak-check=full -v --show-reachable=yes --track-origins=yes --log-file=valgrind.log syslog-ng -F --no-caps -p /var/run/syslog-ng.pid" 1) random files: ==22416== 3,675,952 bytes in 9,989 blocks are definitely lost in loss record 578 of 578 ==22416== at 0x4021EC2: calloc (vg_replace_malloc.c:418) ==22416== by 0x40FF57D: g_malloc0 (in /lib/libglib-2.0.so.0.1200.3) ==22416== by 0x4052899: log_queue_fifo_new (logqueue-fifo.c:438) ==22416== by 0x4046057: log_dest_driver_acquire_queue_method (driver.c:153) ==22416== by 0x47081D1: affile_dw_init (driver.h:178) ==22416== by 0x4707C41: affile_dd_open_writer (logpipe.h:239) ==22416== by 0x405C258: main_loop_call (mainloop.c:145) ==22416== by 0x4707A6B: affile_dd_queue (affile.c:1127) ==22416== by 0x40451C8: log_dest_group_queue (logpipe.h:288) ==22416== by 0x404B9DD: log_multiplexer_queue (logpipe.h:288) ==22416== by 0x404EC16: log_parser_queue (logpipe.h:288) ==22416== by 0x4047473: log_filter_pipe_queue (logpipe.h:288) ==22416== ==22416== LEAK SUMMARY: ==22416== definitely lost: 3,676,156 bytes in 9,992 blocks ==22416== indirectly lost: 6,541 bytes in 56 blocks ==22416== possibly lost: 16,967 bytes in 117 blocks ==22416== still reachable: 76,908 bytes in 3,286 blocks ==22416== suppressed: 0 bytes in 0 blocks 2) single file (constant filename): ==19460== 368 bytes in 1 blocks are definitely lost in loss record 525 of 577 ==19460== at 0x4021EC2: calloc (vg_replace_malloc.c:418) ==19460== by 0x40FF57D: g_malloc0 (in /lib/libglib-2.0.so.0.1200.3) ==19460== by 0x4052899: log_queue_fifo_new (logqueue-fifo.c:438) ==19460== by 0x4046057: log_dest_driver_acquire_queue_method (driver.c:153) ==19460== by 0x47081D1: affile_dw_init (driver.h:178) ==19460== by 0x4707C41: affile_dd_open_writer (logpipe.h:239) ==19460== by 0x405C258: main_loop_call (mainloop.c:145) ==19460== by 0x4707A6B: affile_dd_queue (affile.c:1127) ==19460== by 0x40451C8: log_dest_group_queue (logpipe.h:288) ==19460== by 0x404B9DD: log_multiplexer_queue (logpipe.h:288) ==19460== by 0x404EC16: log_parser_queue (logpipe.h:288) ==19460== by 0x4047473: log_filter_pipe_queue (logpipe.h:288) (...) ==19460== LEAK SUMMARY: ==19460== definitely lost: 572 bytes in 4 blocks ==19460== indirectly lost: 6,541 bytes in 56 blocks ==19460== possibly lost: 12,919 bytes in 106 blocks ==19460== still reachable: 76,908 bytes in 3,286 blocks ==19460== suppressed: 0 bytes in 0 blocks Could anyone proficient in syslog-ng source code take a look on this? Can I do anything else to help you diagnose this? (I can provide complete valgrind output logs if this helps.) Also, I'll try to reproduce this on other versions later this week, unless someone fixes this earlier. :) Cheers, -- Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D

7 43

RFC: value-pairs key rewrite framework, part N+1
by Gergely Nagy 12 Nov '11

12 Nov '11

Hi! After my former mails (see the thread starting at http://thread.gmane.org/gmane.comp.syslog-ng/11355/focus=11421) I'd like to do a recap, and ask for comments, as I came up with a few new ideas. The purpose of the value-pairs key rewrite framework is to make it possible to apply various transformations to the keys we selected with VP. So that we can add or remove prefixes, replace parts of the string, and so on. As of this writing, code to do this exists on one of my branches, but it hasn't been touched for a while, and I planned to update it in the near future. And that's when it dawned on me, that perhaps the syntax isn't all that great. To show why I think that, let's see an example first: value-pairs( scope("everything") rekey( add-prefix(".secevt" "events") add-prefix(".classifier" "syslog-ng") shift(".sdata.*" 1) replace("." "_") ) ); This will add an "events" prefix to each key that starts with ".secevt", so that ".secevt.verdict" becomes "events.secevt.verdict"; similary, ".classifier.class" becomes "syslog-ng.classifier.class"; keys that match '.sdata.*' get shifted to the right, removing the dot. And all remaining dots at the begininng of a key will get replaced by an underscore instead. This kinda makes sense, and I could even massage the syntax into format-json: $(format-json --scope everything --rekey --add-prefix .secevt=events --addprefix .classifier=syslog-ng --shift .sdata.*=1 --replace .=_ --end-rekey) However, this syntax has the downside of transformations being global: I can't choose subset of my keys, and apply a list of transformations on those and only those. Once I made a transformation, any transformations in the list afterwards will see the transformed key. So I can't easily say: "take all the keys starting with '.sdata.', shift them 6 chars, then replace any key names that start with 'win' with 'lose', and finally prefix them with 'whatever'". With the current syntax, that's next to impossible to do sanely. It's also not all that intuitive.. So I came up with a different syntax: wiring rekey into the key() option of value-pairs! That way, we already selected a subset to work on, and the transformations would apply to only those. (This could be combined with the global syntax aswell, though) So it'd look something like this: value-pairs( scope("everything") key(".secevt.*" rekey(add-prefix("events"))) key(".classifier.*" rekey(add-prefix("syslog-ng"))) key(".sdata.*" rekey(shift(1))) key(".*" rekey(replace ("." "_"))) ); This would achieve the exact same effect as the example above, but with a clearer syntax, perhaps. It would also mean that key rewriting can be described at the same place where the key is selected to begin with. The downside of this is that it'd be a bit harder to come up with a syntax for format-json that mimics the config file syntax. So, instead of trying to do that and end up with something horrible, I have another proposal: lets make value-pairs a top-level citizen, so that it joins the ranks of filter{} and rewrite{} and the like! That way, we could turn the following ugly thing: destination d_structured { mongodb( value-pairs( scope("everything") key(".secevt.*" rekey(add-prefix("events"))) key(".classifier.*" rekey(add-prefix("syslog-ng"))) key(".sdata.*" rekey(shift(1))) key(".*" rekey(replace ("." "_"))) ) ); file("/var/log/structured.json" template("$(format-json <repeat the above stuff, but with format-json syntax>)\n")); }; Into this beauty: valuepairs vp_example { scope(everything); key(".secevt.*" rekey(add-prefix("events"))); key(".classifier.*" rekey(add-prefix("syslog-ng"))); key(".sdata.*" rekey(shift(1))); key(".*" rekey(replace ("." "_"))); }; destination d_structured { mongodb(value-pairs(vp_example)); file("/var/log/structured.json" template("$(format-json --with-config vp_example)\n")); }; This would have the nice consequence of not having to keep two parsers in sync: format-json would only have a --with-config option, and nothing else. So, in the end, this whole boils down to two questions: * What do you think about allowing (or even moving) rekey() inside key() options? * What do you think about introducing a top-level valuepairs element, and dropping the format-json argument parsing stuff? -- |8]

2 5