[syslog-ng] need help debugging some network received logs that aren't writing to files

[Chris Moody]

Hmm... that's a thought.   The troublesome device is an IOS system.  
I'llgive 'er a gander to see if there are any other options.  I don't 
recall there being any that controlled the 'hostname' header field though.

-Chris

On 2/18/14 5:10 PM, Scot Needy wrote:
> We had a parsing problem on our ASA where the log contained an extra date so the Host looked like “Feb”.
>
> There was a syslog option in the ASA not to send the date in the header.
>
> On Feb 18, 2014, at 4:59 PM, Chris Moody <chris at node-nine.com> wrote:
>
>> yes - there are tons of spool files being created successfully. As any
>> new network device starts logging we see a new log-spool get created for
>> it's source-ip.
>>
>> Tons of free disk space - almost a Tb of free room.  Loads of
>> processor/mem overhead.  Nothing glaring in syslog-ng's logs (like
>> unable to write or whatnot)
>>
>> Just debugging a host-device that we're not seeing logs accounted for.
>>
>> -Chris
>>
>> On 2/18/14 3:51 PM, Austin Jorden wrote:
>>> Hi Chris,
>>>
>>> Are there *any* folders/files being created at all?
>>>
>>> There's one thing I noticed that isn't specified... which is the
>>> "createdirs = Yes" option. It appears (well, I assume) that you're
>>> wanting it to create a separate text file for each $HOST, not a separate
>>> directory named $HOST...
>>>
>>> - Austin
>>>
>>> On 2/18/2014 2:12 PM, Chris Moody wrote:
>>>> Hello.
>>>>
>>>> First off, thanks a __TON__ for syslog-ng.  I've sworn by this awesome
>>>> code for years now.  I've built all sorts of logging infrastructure with
>>>> it.
>>>>
>>>> I seem to have hit on something though that's got me scratching my head
>>>> and lacking for explanation.  Perhaps I've just been staring at it and
>>>> debugging it too long and am missing something obvious.
>>>>
>>>> I've got an installation with a couple thousand network devices logging
>>>> successfully to output spools on our log aggretor.  This is rockin' and
>>>> works beautifully.  I've got things configured whereby each network
>>>> source logs to it's own individual spool file with the source-ip as the
>>>> spool name.
>>>>
>>>> I'm running into a case though where I have a Cisco switch sending logs
>>>> to my log aggregator but the log-server isn't writing the output to the
>>>> device's spool file.  It is working however for many many more devices
>>>> just like this switch.
>>>>
>>>> I've confirmed via tcpdump that this log traffic does actually hit the
>>>> box, but it never gets recorded into the log spool for that network device.
>>>>
>>>> Since the host is -super- busy receiving logs from other gear
>>>> enterprise-wide, I have to treat it very gingerly, so can't enable too
>>>> much debugging...but I'm really confused why the logs wouldn't show up
>>>> in the log spool..
>>>>
>>>> Here's some bits of the config that are relevant:
>>>> =====
>>>> options {
>>>>            keep_hostname(yes);
>>>>            use_dns(no);
>>>>            use_fqdn(no);
>>>>            stats_freq(600);
>>>>            stats_level(2);
>>>>            # Allow large messages
>>>>            log_msg_size(65536);
>>>> };
>>>>
>>>> # =====================
>>>> # UDP Packet Source
>>>> source s_udp {
>>>>            udp();
>>>> };
>>>>
>>>> # =====================
>>>> # TCP Packet Source
>>>> source s_tcp {
>>>>             tcp(ip(aaa.bbb.ccc.ddd) port(514) max-connections(50000));
>>>> };
>>>>
>>>> # =====================
>>>> destination net_perhost {
>>>>            file("/data/log/per-host/$HOST"
>>>>            owner(root)
>>>>            group(nwadmin)
>>>>            perm(0775)
>>>>            );
>>>> };
>>>>
>>>> # =====================
>>>> log {
>>>>            source(s_tcp);
>>>>            source(s_udp);
>>>>            destination(net_perhost);
>>>> };
>>>> =====
>>>>
>>>> I've checked around for perhaps a different spool name, thinking perhaps
>>>> the data was getting recognized as something other than it's source-ip,
>>>> but haven't seen anything.
>>>>
>>>> Any thoughts?
>>>>
>>>> Cheers,
>>>> -Chris
>>>> ______________________________________________________________________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>
>>> ______________________________________________________________________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>