- syslog-ng - lists.balabit.hu

GSoC: Proposal Submitted
by Rajul Srivastava 17 Mar '14

17 Mar '14

Hello, I have submitted my proposal regarding ' Type Hinting from the start' for GSoC 2014. It is at the link https://www.google-melange.com/gsoc/proposal/review/student/google/gsoc2014… I shall request you all to review it and make some suggestions and comments on how I can improve it. I am particularly worried about the timeline, if I have not given ample time to tasks. Any suggestion or criticism is more that welcome. Regards, Rajul -- Rajul Final year Undergraduate Student, Department of Chemistry, Indian Institute of Technology Kharagpur

2 3

Background/foreground problem
by Hardcastle, Iain (CBC - Hamilton) 17 Mar '14

17 Mar '14

Hi all, Odd one I've come across, cant find any clues on the web so figured I'd post here. Trying to get syslog-ng integrated into the Observium NMS package. Not sure whether the problem lies with syslog-ng, PHP or the sucking of data into Observium. Let me explain: With syslog-ng started in foreground mode 'syslog-ng -Fevd' everything works as expected. I see syslog packets coming in via tcpdump, I see the log flash up on the terminal and it correctly routes into the right part of the Observium front end. With syslog-ng started as a daemon (from boot - the desired config setup - or with 'service syslog-ng start'), nothing gets to Observium. I can still see the syslog packets in tcpdump obviously but I'm blind as to what happens to the messages after that. One thing I have noticed is that when running syslog-ng in foreground mode, I see "Starting destination program; cmdline='/opt/observium/syslog.php'" in the startup messages and a corresponding process in the 'ps -lax' output "php /opt/observium/syslog.php" With syslog-ng started as a daemon, I do not see this php process in the ps output.. Bit stuck as to where to look next. Has anyone seen this behavior before? Centos6.5 and standard packages and configs. System works great otherwise. Syslog-ng config file in its entirety below: options { chain_hostnames(0); flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (1); }; source s_net { udp(); }; destination d_observium { program("/opt/observium/syslog.php" template ("$HOST||$FACILITY||$PRIORITY||$LEVEL||$TAG||$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC||$MS G||$PROGRAM\n") template-escape(yes)); }; log { source(s_net); destination(d_observium); }; Any ideas? iain Deloitte refers to one or more of Deloitte Touche Tohmatsu Limited, a UK private company limited by guarantee, and its network of member firms, each of which is a legally separate and independent entity. Please see http://www.deloitte.com/about for a detailed description of the legal structure of Deloitte Touche Tohmatsu Limited and its member firms.

2 2

odd behavior writing "binary" logfiles
by jrhendri＠roadrunner.com 17 Mar '14

17 Mar '14

I have a really odd problem that I hope someone can assist with. My install is inserting a number of zeroes into the logfile between the date and the rest of the message. Here are some details: System is RHEL: $ uname -a Linux vxpip-eeisl001 2.6.32-358.el6.x86_64 #1 SMP Tue Jan 29 11:47:41 EST 2013 x86_64 x86_64 x86_64 GNU/Linux $ Syslog-ng was built on the box yesterday with these sources: eventlog-0.2.12+20120504+1700 syslog-ng-3.5.3 $ /usr/local/sbin/syslog-ng --version syslog-ng 3.5.3 Installer-Version: 3.5.3 Revision: ssh+git://algernon@git.balabit/var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.5#master#ccb05a22408ba4c837d998b2538854d994f845a5 Compile-Date: Mar 12 2014 11:37:32 Available-Modules: afmongodb,afstomp,syslogformat,affile,basicfuncs,csvparser,confgen,system-source,afamqp,linux-kmsg-format,afprog,afuser,afsocket,dbparser,cryptofuncs,afsocket-notls Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off Enable-Pcre: off This is the default config file that came with it: ############################################################################# # Default syslog-ng.conf file which collects all local logs into a # single file called /var/log/messages. # @version: 3.5 @include "scl.conf" source s_local { system(); internal(); }; source s_network { udp(); }; destination d_local { file("/var/log/messages"); }; log { source(s_local); # uncomment this line to open port 514 to receive messages #source(s_network); destination(d_local); }; Here is the file: $ sudo file /var/log/messages /var/log/messages: data $ sudo cat /var/log/messages Mar 13 10:43:00 syslog-ng[18451]: syslog-ng starting up; version='3.5.3' Mar 13 10:43:01 CROND[18454]: (root) CMD (/util/avail/get_avail.sh 1>/util/avail/logs/get_avail.out 2>&1) Mar 13 10:43:16 sudo: a0142566 : TTY=pts/2 ; PWD=/home/a0142566 ; USER=root ; COMMAND=/usr/bin/file /var/log/messages And this is a capture from vi in hex mode on that file. 0000000: 4d61 7220 3133 2031 303a 3433 3a30 3020 Mar 13 10:43:00 0000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0000050: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0000060: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0000070: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0000080: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0000090: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0000100: 0000 0000 0000 0000 0000 0000 0000 0020 ............... 0000110: 7379 736c 6f67 2d6e 675b 3138 3435 315d syslog-ng[18451] 0000120: 3a20 7379 736c 6f67 2d6e 6720 7374 6172 : syslog-ng star 0000130: 7469 6e67 2075 703b 2076 6572 7369 6f6e ting up; version 0000140: 3d27 332e 352e 3327 0a4d 6172 2031 3320 ='3.5.3'.Mar 13 0000150: 3130 3a34 333a 3031 2000 0000 0000 0000 10:43:01 ....... 0000160: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0000170: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0000180: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0000190: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00001a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00001b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00001c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00001d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00001e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00001f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0000200: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0000210: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0000220: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0000230: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0000240: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0000250: 0000 0000 0000 0000 2043 524f 4e44 5b31 ........ CROND[1 0000260: 3834 3534 5d3a 2028 726f 6f74 2920 434d 8454]: (root) CM 0000270: 4420 282f 7574 696c 2f61 7661 696c 2f67 D (/util/avail/g 0000280: 6574 5f61 7661 696c 2e73 6820 313e 2f75 et_avail.sh 1>/u 0000290: 7469 6c2f 6176 6169 6c2f 6c6f 6773 2f67 til/avail/logs/g 00002a0: 6574 5f61 7661 696c 2e6f 7574 2032 3e26 et_avail.out 2>& 00002b0: 3129 0a4d 6172 2031 3320 3130 3a34 333a 1). Any ideas? Thanks in advance folks!! Jim

3 4

Re: [syslog-ng] odd behavior writing "binary" logfiles - followup data
by Jim Hendrick 14 Mar '14

14 Mar '14

Will do. Probably tomorrow at the earliest. Jim Sent from my Verizon Wireless 4G LTE Smartphone -------- Original message -------- From: Balazs Scheidler <bazsi77(a)gmail.com> Date:03/14/2014 5:33 PM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Subject: Re: [syslog-ng] odd behavior writing "binary" logfiles - followup data During startup syslog-ng queries the hostname using gethostname. Can you strace syslog-ng and look for this call to see what it returns? We are also resolving this using the dns which is also interesting. Thanks On Mar 13, 2014 7:08 PM, <jrhendri(a)roadrunner.com> wrote: OK - after much poking around (including testing this on a different host with this morning's latest source) here's what I found I believe that syslog-ng is (for some reason) unable to come up with a correct hostname for the local system. I would really like to know why this is happening

1 0

error while loading shared libraries
by Pete Perreault 14 Mar '14

14 Mar '14

Hi- I had syslog-ng 3.5.3 operational without an issue. I wanted to implement the smtp driver so installed libesmtp and then re-compiled both enventlog and syslog-ng. The only difference was my configure arguments for syslog-ng, which was configure --with-libesmtp=/usr/local/lib. Everything looked good afterwards. syslog-ng Open Source Edition 3.5.3 configured Compiler options: compiler : gcc -std=gnu99 compiler options : -g -O2 -Wall -pthread -pthread -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/usr/local/include/eventlog -I$(top_srcdir)/lib/ivykis/src/include -I$(top_builddir)/lib/ivykis/src/include -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 linker flags : prefix : /usr/local linking mode : dynamic embedded crypto : no __thread keyword : yes Submodules: ivykis : internal libmongo-client : internal librabbitmq : internal Features: Debug symbols : no GCC profiling : no Memtrace : no IPV6 support : yes spoof-source support : no tcp-wrapper support : no Linux capability support : no PCRE support : yes Env wrapper support : no systemd support : no (unit dir: none) Modules: Module search path : /usr/local/lib/syslog-ng Sun STREAMS support (module): no SSL support (module) : yes SQL support (module) : no PACCT module (EXPERIMENTAL) : no MongoDB destination (module): yes JSON support (module) : no SMTP support (module) : yes AMQP destination (module) : yes STOMP destination (module) : yes GEOIP support (module) : no Redis support (module) : no Yet, now when I attempt to start syslog-ng I receive the following error. /usr/local/sbin/syslog-ng: error while loading shared libraries: libsyslog-ng-3.5.3.so: cannot open shared object file: No such file or directory However, the file looks good also. ls -l /usr/local/lib/libsyslog-ng-3.5.3.so -rwxr-xr-x 1 root root 2227624 Mar 12 08:47 /usr/local/lib/ libsyslog-ng-3.5.3.so any thoughts as to what might be wrong? Thanks,

2 1

Google Summer of Code proposals
by Tusa Viktor 14 Mar '14

14 Mar '14

Dear Google Summer of Code Applicants! Please note that you have only one week left for submitting your proposals. Please register in www.google-melange.com as a student, upload the necessary document, and you can start writing your proposal. Some of you already knows, that we will give you a small task which will test your eligibility for the project. We will give these tasks after your proposal submission, but not before March 17 (Monday). If you need a few tips about how to write a good proposal, here is a blog post about it: http://www.di.ens.fr/~baghdadi/TXT_blog/5_advices_to_get_your_proposal_acce… Happy Coding! Best Regards, Viktor Tusa

1 0

AUTO: James B Horwath is out of the office (returning 03/18/2014)
by James B Horwath 14 Mar '14

14 Mar '14

I am out of the office until 03/18/2014. I will be out of the office until 03/18/2013 with limited access to e-mail. If you have an urgent issue please contact Sunil Gupta at (610)-807-6710 or Dylan Yoder at (610)-807-7230. Regards Jim Horwath IT Security Services & Governance, Corporate BTS and IT Security/Risk Tel 610-807-8795 Mobile 610-533-6972 3900 Burgess Place, Bethlehem, Pa. 18017 Jim_Horwath(a)glic.com Note: This is an automated response to your message "syslog-ng Digest, Vol 107, Issue 29" sent on 3/14/2014 6:40:24. This is the only notification you will receive while this person is away. ----------------------------------------- This message, and any attachments to it, may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are notified that any use, dissemination, distribution, copying, or communication of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately by return e-mail and delete the message and any attachments. Thank you.

1 0

syslog-ng news: GSoC, incubator, conferences
by Peter Czanik 14 Mar '14

14 Mar '14

Hi, Yesterday the syslog-ng newsletter went out with broken links and for some users it ended up in the spam folder for gmail users (and possibly others). I'm still debugging the cause of these problems. Until a good solution is found, here is a link to the on-line version: https://insider.blogs.balabit.com/2014/03/gsoc-incubator-conferences/ Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

Config visualization - concrete task
by Sandeep K Chaudhary 14 Mar '14

14 Mar '14

Hi guys, I have installed syslog-ng on my system, and have been looking at this[1] repository a little. I think I can start doing something concrete now. Can someone please give me the task so that I can start working on it? I hope that [1] is the correct repo to look at. [1] https://github.com/balabit/syslog-ng-3.5 Thanks, Sandeep.

3 3

insider 2014-03: GSoC; Incubator; Conferences;
by Peter Czanik 13 Mar '14

13 Mar '14

Dear syslog-ng users, This is the 32th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news. FEATURED NEWS Google Summer of Code (GsoC) ---------------------------- This year the syslog-ng project was accepted to GSoC as a mentoring organization. We vould like to thank to the Debian and openSUSE projects for vouching for us and openSUSE to let us participate previous years under the openSUSE umbrella. We don't know yet the number of students we can mentor, but we already have a nice number of project ideas and students who expressed interest. It's not yet too late to browse the ideas page _https://github.com/balabit/syslog-ng/wiki/GSoC2014-Idea-&-Project-list_ and write a project proposal to the syslog-ng mailing list. Contact information is available at _https://github.com/balabit/syslog-ng/wiki/GSoC2014_ syslog-ng incubator updates --------------------------- The syslog-ng incubator is a collection of tools and modules which are not (yet) part of the official repository. Version 0.2.1 was released recently with many bugfixes, advanced LUA related features, support for FreeBSD and a python script based ElasticSearch destination. The syslog-ng incubator is available in many ways. First of all, the source is available in git at _https://github.com/balabit/syslog-ng-incubator_and as a release tgz at _http://packages.madhouse-project.org/syslog-ng/incubator/syslog-ng-incubat… There are also packages for openSUSE and Fedora with riemann support available in a subpackage. To use it, visit one of the below web pages and add the necessary repositories, which also have syslog-ng 3.5 and the riemann C client available. Fedora: _http://copr.fedoraproject.org/coprs/czanik/syslog-ng35/_ openSUSE: _https://build.opensuse.org/project/show/home:czanik:syslog-ng35_ A FreeBSD port is also work in progress, once ready, it will be committed to sysutils/syslog-ng-incubator. Riemann support is not included. syslog-ng at conferences ------------------------ The next confirmed events are: * Open Source Data Center Conference, 8-10 April, Berlin, Germany: _http://www.netways.de/osdc/_, Giving a talk titled "Monitoring with syslog-ng, Riemann and Kibana" There will be two developers present, so if anyone attends, and has Incubator, Lua, Riemann, etc. related questions, we'll be happy to answer. * Linux Open Admin Days, 5-6 April, Antwerp, Belgium: _http://loadays.org/_, Giving a talk titled “Babel Fish for DevOps: syslog-ng”. Those, who visited FOSDEM will be familiar with the basis of the presentation, but this time with a lot more details and configuration examples thanks to the more available time. * Infosecurity Europe, 29 April - 1 May, London, United Kingdom: _http://www.infosec.co.uk/_, Giving a talk titled "Finding method in the madness: the challenges of the automatic classification of log messages". The talk will be given by Balazs Scheidler (Bazsi) who you will also be able to find at the BalaBit booth at the event. * LinuxTag, 8-10 May, Berlin, Germany: _http://linuxtag.org <http://linuxtag.org/>_, Giving a talk titled "Finding method in the madness: the challenges of the automatic classification of log messages". BalaBit will also be sponsoring the event so you'll be able to find us and get T-shirts and talk with our engineers. The talk will be a revised version of the talk given a week before at Infosecurity Europe. NEW RELEASES * syslog-ng incubator 0.2.1: _https://github.com/balabit/syslog-ng-incubator/blob/master/NEWS_ * syslog-ng OSE 3.5.4: _https://lists.balabit.hu/pipermail/syslog-ng-announce/2014-March/000182.html_ * syslog-ng OSE 3.5.4.1: _https://lists.balabit.hu/pipermail/syslog-ng-announce/2014-March/000183.html_ * syslog-ng PE 4.0.7: _https://lists.balabit.hu/pipermail/syslog-ng-announce/2014-March/000181.html_ Your feedback and news tips about the next issue is welcome at _documentation(a)balabit.com <mailto:documentation@balabit.com>_To read this newsletter on-line, visit: _http://insider.blogs.balabit.com/_ -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0