- syslog-ng - lists.balabit.hu

Save ALL logs
by Josu Lazkano 05 Jul '11

05 Jul '11

Hello again, I just installed my new log server and I am configuring to save ALL logs on the log server. On the client side I have rsyslog configured this way: $ cat /etc/rsyslog.conf $ModLoad imuxsock $ModLoad imklog $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat $RepeatedMsgReduction on $KLogPath /proc/kmsg $WorkDirectory /rsyslog/work $ActionQueueType LinkedList $ActionQueueFileName srvrfwd1 $ActionResumeRetryCount -1 $ActionQueueSaveOnShutdown on *.* @@logserver On the server side I have syslog-ng: source logs_externs{ tcp(); }; destination rdf_all { file("/var/log/extern/${HOST}/${YEAR}/${MONTH}/"); }; log { source(logs_externs); destination(rdf_all); }; I want to get all the logs on the log servers and then archive each month. This way it doesn't work, how I must configure it? Thanks for all your help. Kind regards.

2 1

[Bug 127] New: 3.2.4 ose segfaults during startup
by bugzilla＠bugzilla.balabit.com 01 Jul '11

01 Jul '11

https://bugzilla.balabit.com/show_bug.cgi?id=127 Summary: 3.2.4 ose segfaults during startup Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: balazs.nemeth(a)balabit.com Type of the Report: bug Estimated Hours: 0.0 This bugreport has been opened from an email came to info(a)balabit.com. ------------------------ Core was generated by `syslog-ng -d -p /var/run/syslog-ng.pid -e --enable-core -s'. Program terminated with signal 11, Segmentation fault. #0 0x00000031566795d0 in strcmp () from /lib64/libc.so.6 (gdb) bt #0 0x00000031566795d0 in strcmp () from /lib64/libc.so.6 #1 0x00002b71bfe565d5 in cfg_process_flag (handlers=0x2b71c0082230, base=0x228b5f0, flag=0xffffffff <Address 0xffffffff out of bounds>) at cfg-parser.c:249 #2 0x00002b71c04a6ec6 in afsocket_parse (lexer=0x2265440, last_driver=0x7fffe7126608) at afsocket-grammar.y:864 #3 0x00002b71bfe58c67 in cfg_parser_parse (self=0x2b71c06b2760, cfg=<value optimized out>, yylloc=<value optimized out>) at cfg-parser.h:82 #4 plugin_parse_config (self=0x2b71c06b2760, cfg=<value optimized out>, yylloc=<value optimized out>) at plugin.c:124 #5 0x00002b71bfe570ae in main_parse (lexer=0x2265440, dummy=0x7fffe7128a28) at cfg-grammar.y:539 #6 0x00002b71bfe37a01 in cfg_parser_parse (self=0x22649b0, lexer=0x2265440, parser=0x2b71c00829e0, result=0x2269640) at cfg-parser.h:82 #7 cfg_run_parser (self=0x22649b0, lexer=0x2265440, parser=0x2b71c00829e0, result=0x2269640) at cfg.c:376 #8 0x00002b71bfe37b03 in cfg_read_config (self=0x22649b0, fname=0x402bb7 "/etc/syslog-ng/syslog-ng.conf", syntax_only=<value optimized out>, preprocess_into=0x0) at cfg.c:398 #9 0x000000000040220c in initial_init () #10 0x000000000040280c in main () (gdb) quit This causes it: source 32001_udp { udp( port(32001) so_rcvbuf(12000000) flags(no_multi_line) ); }; ------------------ ps.: With the given config line snippet it is easy to reproduce. I have cc'd the original author of the email. I hope he doesn't mind it. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

[Bug 126] New: [systemd] Add installation of systemd unit file via build-sys
by bugzilla＠bugzilla.balabit.com 01 Jul '11

01 Jul '11

https://bugzilla.balabit.com/show_bug.cgi?id=126 Summary: [systemd] Add installation of systemd unit file via build-sys Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: d(a)falconindy.com Type of the Report: enhancement Estimated Hours: 0.0 I'd like to see syslog-ng offer installation of the systemd unit file via the build system, similar to how other services (e.g., udev or rsyslog) provide this, with the --with-systemdsystemunitdir configure option. autotools is unfortunately not my forte, so the attached patch may not be the most eloquent of solutions (borrowing bits from udev), but it does the job, and does not appear to bother configurations which avoid --enable-systemd or the new --with flag. I felt it was prudent to separate the --enable-systemd flag from this new flag as there's certainly a case for using the --enable flag without the --with flag. I've also provided a one line amendment to the service file itself, which removes the ExecReload. This is discouraged by upstream, as systemd is unable to properly track the behavior of the daemon due to the effects of an asynchronous trigger. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 10

Transparent TCP forward: what am I doing wrong?
by JP Vossen 30 Jun '11

30 Jun '11

I need to be able to accept TCP/514 and transparently forward it to UDP/514 on the same box. I have a config that almost works, except I'm getting an "extra" time stamp and hostname on the TCP messages. I'm using Netcat to send: echo "<182>loop_test[$$]: LOOP testing 514 UDP, one ping only..." | nc -u -w1 192.168.1.151 514 echo "<182>loop_test[$$]: LOOP testing 514 TCP, one ping only..." | nc 192.168.1.151 514 I'm getting: <182>loop_test[11061]: LOOP testing 514 UDP, one ping only... <182>Jun 30 01:48:36 192.168.1.10 loop_test[11061]: LOOP testing 514 TCP, one ping only... I want: <182>loop_test[11061]: LOOP testing 514 UDP, one ping only... <182>loop_test[11061]: LOOP testing 514 TCP, one ping only... If I send the messages into syslog-ng or rsyslog, I get the correct data with no "extra" fields in the log file. But unfortunately, I need to send messages into another proprietary syslog listener, and in that one I'm getting the extra fields. I'd blame the whole mess on that, except that when I do a packet dump I do see the 2 extra fields in there. So I *think* that syslog-ng and rsyslog are smart enough to handle them and my other thing isn't. I'm already using keep_timestamp(no) and keep_hostname(no) but they have no effect I've been able to detect. I've tried commented out, set to yes and set to no, but no matter what I get the 2 extra fields. I've also tried syslog-ng 3.0.8 and 3.2.4, same effect. This is the relevant part of the syslog-ng config I'm using: ####### BEGIN: Forward TCP syslog source s_tcpincoming { tcp( ip(0.0.0.0) port(514) max_connections(1000) keep_timestamp(no) keep_hostname(no) ); }; destination d_fe_tcp { udp( "127.0.0.2" port(514) spoof_source(yes) ); }; log { source(s_tcpincoming); # With no "filter" we get everything, which is what we want destination(d_fe_tcp); }; ####### END: Forward TCP syslog As I said, my goal is to receive TCP/514 and **transparently** forward logs with no changes, as if they came in via UDP, to the localhost via UDP/514. In other words, I'm using syslog-ng as a shim to feed syslog over TCP to a listener which only listens on UDP. Why doesn't it work? What totally obvious thing am I missing? Am I doing anything else dumb? TIA, JP ----------------------------|:::======|------------------------------- JP Vossen, CISSP |:::======| http://bashcookbook.com/ My Account, My Opinions |=========| http://www.jpsdomain.org/ ----------------------------|=========|------------------------------- "Microsoft Tax" = the additional hardware & yearly fees for the add-on software required to protect Windows from its own poorly designed and implemented self, while the overhead incidentally flattens Moore's Law.

4 4

Auto-reply
by ago＠lsc.hu 30 Jun '11

30 Jun '11

Tisztelt Hölgyem/Uram! 2011.07.3-ig szabadságon tartózkodom. Ajánlatkérés vagy egyéb üzleti levelezés esetén írjon Tóth László kollégámnak, az info(a)lsc.hu címre. Támogatási kérdés esetén használja support felületünket a megadott felhasználónév-jelszó párossal. Üdvözlettel, Deim Ágoston

1 0

Auto-reply
by ago＠lsc.hu 29 Jun '11

29 Jun '11

Tisztelt Hölgyem/Uram! 2011.07.3-ig szabadságon tartózkodom. Ajánlatkérés vagy egyéb üzleti levelezés esetén írjon Tóth László kollégámnak, az info(a)lsc.hu címre. Támogatási kérdés esetén használja support felületünket a megadott felhasználónév-jelszó párossal. Üdvözlettel, Deim Ágoston

1 0

SYSLOG-NG and AIX packages.
by dbarry＠princesscruises.com 28 Jun '11

28 Jun '11

Greetings all: (Although it looks like this list is mostly populated by devs, the descriptor describes it as a user mailiong list as well, so I hope this message is OK here. If not, my apologies, and hopefully I can be re-directed.) I've had minor experience with personal experimentation with syslog-ng in a simple Redhat enviroment, where all I had to do was download the package. However, I'm trying to encourage my work environment to move to syslog-ng in our production environment. My work enivo consists of some AIX boxes, versions 5.3L, 6.x, and now 7.1. I believe our AIX team would be comfortable deploying standard packages, however, from what I can see in the evaluation section, it looks like the latest AIX packages I see is syslog-ng-2.0.10-1.aix5.2.ppc I see packages for other distros, but I think that is all for AIX.. >From what I know, our AIX team doesn't typically build and complile, the mostly install and run pre-packaged items on them, such as webservers, databases, etc. In my poking around, I see that there is active development with AIX in mind, but if I am understanding correctly, there are no current AIX packaged distros --- they have to be built by the end user. Is this correct? Can we purchase compiled packages and binaries for our environment? Or is compiling a build/package something that is trivial? (I suspect not). For now, these boxes would only really be "clients", if that makes any difference at all. And if I am wrong, and should be looking elsewhere for more recent AIX binaries/packages I would really appreciate a point in the right direction. I'd really be grateful for some guidance here. At this point, I'd be remiss not to thank all the folks behind the project. From my personal usage, its clear what a phenomenal project this is. Thanks much, d. The information contained in this email and any attachment may be confidential and/or legally privileged and has been sent for the sole use of the intended recipient. If you are not an intended recipient, you are not authorized to review, use, disclose or copy any of its contents. If you have received this email in error please reply to the sender and destroy all copies of the message. Thank you.

2 1

TCP vs. UDP
by Josu Lazkano 27 Jun '11

27 Jun '11

Hello again, I am reading on internet about syslog-ng. Next week I will start to change all my syslog configuration. I have 50 servers with rsyslog on client side and syslog-ng on the server side. The logs goes with UDP on the LAN. Is it better TCP? I am on a secure LAN so I don't going to apply SSL yet. Is there any important advantage using TCP? Any recommendation¿? Thanks for all your help and best regards.

2 2

Re: [syslog-ng] TCP vs. UDP
by Deim Ágoston 27 Jun '11

27 Jun '11

hi, TCP provides reliable transport because of TCP. Beside that there's no harm if you use TLS and there's no such thing that secure network if the clients connect to the net or there's public services opened to the internet. If you want to ensure that nobody impersonate the log clients or the log server use TLS. regards, Ago > Hello again, I am reading on internet about syslog-ng. Next week I will > start to change all my syslog configuration. I have 50 servers with > rsyslog on client side and syslog-ng on the server side. The logs goes > with UDP on the LAN. > > Is it better TCP? I am on a secure LAN so I don't going to apply SSL yet. > > Is there any important advantage using TCP? Any recommendationż? > > Thanks for all your help and best regards. > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > >

1 0

[Bug 124] New: [regression] broken $FULLHOST after upgrade to 3.2.4
by bugzilla＠bugzilla.balabit.com 27 Jun '11

27 Jun '11

https://bugzilla.balabit.com/show_bug.cgi?id=124 Summary: [regression] broken $FULLHOST after upgrade to 3.2.4 Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: enrico.scholz(a)informatik.tu-chemnitz.de Type of the Report: --- Estimated Hours: 0.0 syslog-ng 3.2.4 sets the chained hostname from local sources always to 'hostname/hostname'. Previous version (2.x and 3.1.4) used 'localhost@hostname' for this resp. '<sourcename>@hostname'. This breaks existing setups because * destination filenames which contain $FULLHOST can not be used anymore because the '/' prevents their creation; e.g. my remote logserver has | destination d_iptables { file("/var/log/remote/IPTABLES/log.$FULLHOST" sync(10)); }; which created e.g. 'log.localhost@walther'. Now, it fails with | Error opening file for writing; filename='/var/log/remote/IPTABLES/log.walther/walther', error='No such file or directory (2)' * originating sources can not be identified anymore; e.g. I use syslog-ng to log events within chroot environments. E.g. I have | source kerberos { unix-dgram ("/vservers/kerberos/dev/log" keep-alive(no)); }; | source ldap { unix-dgram ("/vservers/ldap/dev/log" keep-alive(no)); }; Previously, messages could be identified because they were reported to come from 'kerberos@<host>' or 'ldap@<host>'. Now, the source is reported as '<host>/<host>' for both environments. 'strace' shows that the syslog client creates '<host>/<host>' strings. The syslog-ng client is configured as ----- @version:3.2 ################################################################### #### #### DO NOT EDIT THIS FILE! #### This file is managed by cfengine; any manual changes will be overridden #### ################################################################### options { flush_lines (0); time_reopen (10); log_fifo_size (16384); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); chain_hostnames (yes); }; source localhost { unix-stream ("/dev/log"); file("/proc/kmsg" program_override("kernel")); internal(); }; destination stunnel { tcp("127.0.0.1" port(520)); tcp("127.0.0.1" port(521)); }; ## Common block for kernel-logging filter f_kern { facility(kern); }; filter f_min_emerg { level(emerg); }; filter f_min_alert { level(alert..emerg); }; filter f_min_crit { level(crit..emerg); }; filter f_min_err { level(err..emerg); }; filter f_min_warn { level(warn..emerg); }; filter f_kern_err { filter(f_kern) and filter(f_min_warn); }; filter f_kern_crit { filter(f_kern) and filter(f_min_crit); }; destination d_kern { file("/var/log/kernel" fsync(yes) perm(0644)); }; destination d_cons { file("/dev/console"); }; log { filter(f_kern_err); source(localhost); destination(d_kern); }; log { filter(f_kern_crit); source(localhost); destination(d_cons); }; ## End kernel-logging block} log { destination(stunnel); flags(catch-all); }; ----- I am using the syslog-ng package from Fedora EPEL6 (http://koji.fedoraproject.org/koji/buildinfo?buildID=245187) -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2