[syslog-ng] odd behavior writing "binary" logfiles - followup data
Jim Hendrick
jrhendri at roadrunner.com
Mon Mar 17 12:34:01 CET 2014
I will take some time today and re-run this with various debugging flags.
(just realized I should have... )
On 03/15/2014 09:37 PM, jrhendri at roadrunner.com wrote:
> Odd - I do not see any gethost calls at all in the strace
>
> this is from "sudo strace /usr/local/sbin/syslog-ng -f /usr/local/etc/syslog-ng.conf > syslog-strace 2>&1"
>
>
> I include the config.status and config.log also to see if that may help.
>
> Jim
>
>
> ---- Balazs Scheidler <bazsi77 at gmail.com> wrote:
>> During startup syslog-ng queries the hostname using gethostname. Can you
>> strace syslog-ng and look for this call to see what it returns?
>>
>> We are also resolving this using the dns which is also interesting.
>>
>> Thanks
>> On Mar 13, 2014 7:08 PM, <jrhendri at roadrunner.com> wrote:
>>
>>> OK - after much poking around (including testing this on a different host
>>> with this morning's latest source) here's what I found
>>>
>>> I believe that syslog-ng is (for some reason) unable to come up with a
>>> correct hostname for the local system.
>>> I would really like to know why this is happening
>>> I would also really like to know if any of the source options I tried
>>> should have worked (I was kind of desperate with some of the combinations...)
>>>
>>> That said - here's my summary:
>>>
>>> Examining the "binary" files, It was always writing 256 bytes of "0" where
>>> ${HOST} should have been.
>>>
>>> It *never* happened with a network source (tried with Kiwi sysloggen using
>>> RCF and non RCF messages)
>>> It *always* happened with any local source including all the ones
>>> commented out (I tested each separately)
>>>
>>> source s_local {
>>> unix-dgram("/dev/log"); # standard Linux log source (this is the default
>>> place for the syslog() function to send logs to)
>>> #!# unix-stream("/dev/log"); # standard Linux log source (this is the
>>> default place for the syslog() function to send logs to)
>>> #!# system();
>>> #!# internal();
>>> };
>>>
>>> Once I identified this behavior, I tried a number of options /
>>> combinations to work around this including (each separately)
>>> source s_local {
>>> system();
>>> #!# system( keep_hostname(no) host_override("localhost") );
>>> #!# system( keep_hostname(yes) host_override("localhost") );
>>> #!# system( flags(no-parse) host_override("localhost") );
>>> #!# system( flags(no-hostname) host_override("localhost") );
>>> #!# system( flags(no-parse) );
>>> #!# system( flags(no-hostname) );
>>> #!# internal( );
>>> };
>>>
>>> And nothing worked.
>>>
>>> Finally I kinda punted... I am now doing something really kludgy as a work
>>> around:
>>> source s_local {
>>> system();
>>> internal( );
>>> };
>>>
>>> source s_network {
>>> udp();
>>> };
>>>
>>> destination d_local_template {
>>>
>>> file("/data/syslog-ng/$YEAR/$MONTH/$DAY/localhost/localhost.$FACILITY.$PRIORITY.$DATE"
>>> template("${ISODATE} localhost ${PROGRAM} ${MESSAGE}\n") );
>>> };
>>>
>>>
>>> destination d_network {
>>>
>>> file("/data/syslog-ng/$YEAR/$MONTH/$DAY/$HOST_FROM/$HOST_FROM.$FACILITY.$PRIORITY.$DATE");
>>> };
>>>
>>> log {
>>> source(s_local);
>>> destination(d_local_template);
>>> };
>>>
>>> log {
>>> source(s_network);
>>> destination(d_network);
>>> };
>>>
>>>
>>> ---- jrhendri at roadrunner.com wrote:
>>>> I have a really odd problem that I hope someone can assist with.
>>>>
>>>> My install is inserting a number of zeroes into the logfile between the
>>> date and the rest of the message.
>>>> Here are some details:
>>>>
>>>> System is RHEL:
>>>>
>>>> $ uname -a
>>>> Linux vxpip-eeisl001 2.6.32-358.el6.x86_64 #1 SMP Tue Jan 29 11:47:41
>>> EST 2013 x86_64 x86_64 x86_64 GNU/Linux
>>>> $
>>>>
>>>>
>>>> Syslog-ng was built on the box yesterday with these sources:
>>>> eventlog-0.2.12+20120504+1700
>>>> syslog-ng-3.5.3
>>>>
>>>>
>>>> $ /usr/local/sbin/syslog-ng --version
>>>> syslog-ng 3.5.3
>>>> Installer-Version: 3.5.3
>>>> Revision: ssh+git://algernon@git.balabit
>>> /var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.5#master#ccb05a22408ba4c837d998b2538854d994f845a5
>>>> Compile-Date: Mar 12 2014 11:37:32
>>>> Available-Modules:
>>> afmongodb,afstomp,syslogformat,affile,basicfuncs,csvparser,confgen,system-source,afamqp,linux-kmsg-format,afprog,afuser,afsocket,dbparser,cryptofuncs,afsocket-notls
>>>> Enable-Debug: off
>>>> Enable-GProf: off
>>>> Enable-Memtrace: off
>>>> Enable-IPv6: on
>>>> Enable-Spoof-Source: off
>>>> Enable-TCP-Wrapper: off
>>>> Enable-Linux-Caps: off
>>>> Enable-Pcre: off
>>>>
>>>>
>>>>
>>>> This is the default config file that came with it:
>>>>
>>>>
>>>>
>>> #############################################################################
>>>> # Default syslog-ng.conf file which collects all local logs into a
>>>> # single file called /var/log/messages.
>>>> #
>>>>
>>>> @version: 3.5
>>>> @include "scl.conf"
>>>>
>>>> source s_local {
>>>> system();
>>>> internal();
>>>> };
>>>>
>>>> source s_network {
>>>> udp();
>>>> };
>>>>
>>>> destination d_local {
>>>> file("/var/log/messages");
>>>> };
>>>>
>>>> log {
>>>> source(s_local);
>>>>
>>>> # uncomment this line to open port 514 to receive messages
>>>> #source(s_network);
>>>> destination(d_local);
>>>> };
>>>>
>>>>
>>>>
>>>> Here is the file:
>>>>
>>>>
>>>> $ sudo file /var/log/messages
>>>> /var/log/messages: data
>>>> $ sudo cat /var/log/messages
>>>> Mar 13 10:43:00 syslog-ng[18451]: syslog-ng starting up; version='3.5.3'
>>>> Mar 13 10:43:01 CROND[18454]: (root) CMD (/util/avail/get_avail.sh
>>> 1>/util/avail/logs/get_avail.out 2>&1)
>>>> Mar 13 10:43:16 sudo: a0142566 : TTY=pts/2 ; PWD=/home/a0142566 ;
>>> USER=root ; COMMAND=/usr/bin/file /var/log/messages
>>>> And this is a capture from vi in hex mode on that file.
>>>>
>>>>
>>>> 0000000: 4d61 7220 3133 2031 303a 3433 3a30 3020 Mar 13 10:43:00
>>>> 0000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 0000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 0000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 0000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 0000050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 0000060: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 0000070: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 0000080: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 0000090: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 00000a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 00000b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 00000c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 00000d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 00000e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 00000f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 0000100: 0000 0000 0000 0000 0000 0000 0000 0020 ...............
>>>> 0000110: 7379 736c 6f67 2d6e 675b 3138 3435 315d syslog-ng[18451]
>>>> 0000120: 3a20 7379 736c 6f67 2d6e 6720 7374 6172 : syslog-ng star
>>>> 0000130: 7469 6e67 2075 703b 2076 6572 7369 6f6e ting up; version
>>>> 0000140: 3d27 332e 352e 3327 0a4d 6172 2031 3320 ='3.5.3'.Mar 13
>>>> 0000150: 3130 3a34 333a 3031 2000 0000 0000 0000 10:43:01 .......
>>>> 0000160: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 0000170: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 0000180: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 0000190: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 00001a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 00001b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 00001c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 00001d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 00001e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 00001f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 0000200: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 0000210: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 0000220: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 0000230: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 0000240: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>>>> 0000250: 0000 0000 0000 0000 2043 524f 4e44 5b31 ........ CROND[1
>>>> 0000260: 3834 3534 5d3a 2028 726f 6f74 2920 434d 8454]: (root) CM
>>>> 0000270: 4420 282f 7574 696c 2f61 7661 696c 2f67 D (/util/avail/g
>>>> 0000280: 6574 5f61 7661 696c 2e73 6820 313e 2f75 et_avail.sh 1>/u
>>>> 0000290: 7469 6c2f 6176 6169 6c2f 6c6f 6773 2f67 til/avail/logs/g
>>>> 00002a0: 6574 5f61 7661 696c 2e6f 7574 2032 3e26 et_avail.out 2>&
>>>> 00002b0: 3129 0a4d 6172 2031 3320 3130 3a34 333a 1).
>>>>
>>>>
>>>> Any ideas?
>>>> Thanks in advance folks!!
>>>> Jim
>>>>
>>>>
>>>>
>>> ______________________________________________________________________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation:
>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>
>>>
>>> ______________________________________________________________________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation:
>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>>>
>>>
>>>
>>> ______________________________________________________________________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20140317/9065a854/attachment.htm
More information about the syslog-ng
mailing list