[syslog-ng] need help debugging some network received logs that aren't writing to files

[Scot Needy]

It wasn’t adding the data to the hostname just adding extra header data that broke the RFC  format. 


On Feb 18, 2014, at 5:14 PM, Chris Moody <chris at node-nine.com> wrote:

> Hmm... that's a thought.   The troublesome device is an IOS system.  
> I'llgive 'er a gander to see if there are any other options.  I don't 
> recall there being any that controlled the 'hostname' header field though.
> 
> -Chris
> 
> On 2/18/14 5:10 PM, Scot Needy wrote:
>> We had a parsing problem on our ASA where the log contained an extra date so the Host looked like “Feb”.
>> 
>> There was a syslog option in the ASA not to send the date in the header.
>> 
>> On Feb 18, 2014, at 4:59 PM, Chris Moody <chris at node-nine.com> wrote:
>> 
>>> yes - there are tons of spool files being created successfully. As any
>>> new network device starts logging we see a new log-spool get created for
>>> it's source-ip.
>>> 
>>> Tons of free disk space - almost a Tb of free room.  Loads of
>>> processor/mem overhead.  Nothing glaring in syslog-ng's logs (like
>>> unable to write or whatnot)
>>> 
>>> Just debugging a host-device that we're not seeing logs accounted for.
>>> 
>>> -Chris
>>> 
>>> On 2/18/14 3:51 PM, Austin Jorden wrote:
>>>> Hi Chris,
>>>> 
>>>> Are there *any* folders/files being created at all?
>>>> 
>>>> There's one thing I noticed that isn't specified... which is the
>>>> "createdirs = Yes" option. It appears (well, I assume) that you're
>>>> wanting it to create a separate text file for each $HOST, not a separate
>>>> directory named $HOST...
>>>> 
>>>> - Austin
>>>> 
>>>> On 2/18/2014 2:12 PM, Chris Moody wrote:
>>>>> Hello.
>>>>> 
>>>>> First off, thanks a __TON__ for syslog-ng.  I've sworn by this awesome
>>>>> code for years now.  I've built all sorts of logging infrastructure with
>>>>> it.
>>>>> 
>>>>> I seem to have hit on something though that's got me scratching my head
>>>>> and lacking for explanation.  Perhaps I've just been staring at it and
>>>>> debugging it too long and am missing something obvious.
>>>>> 
>>>>> I've got an installation with a couple thousand network devices logging
>>>>> successfully to output spools on our log aggretor.  This is rockin' and
>>>>> works beautifully.  I've got things configured whereby each network
>>>>> source logs to it's own individual spool file with the source-ip as the
>>>>> spool name.
>>>>> 
>>>>> I'm running into a case though where I have a Cisco switch sending logs
>>>>> to my log aggregator but the log-server isn't writing the output to the
>>>>> device's spool file.  It is working however for many many more devices
>>>>> just like this switch.
>>>>> 
>>>>> I've confirmed via tcpdump that this log traffic does actually hit the
>>>>> box, but it never gets recorded into the log spool for that network device.
>>>>> 
>>>>> Since the host is -super- busy receiving logs from other gear
>>>>> enterprise-wide, I have to treat it very gingerly, so can't enable too
>>>>> much debugging...but I'm really confused why the logs wouldn't show up
>>>>> in the log spool..
>>>>> 
>>>>> Here's some bits of the config that are relevant:
>>>>> =====
>>>>> options {
>>>>>           keep_hostname(yes);
>>>>>           use_dns(no);
>>>>>           use_fqdn(no);
>>>>>           stats_freq(600);
>>>>>           stats_level(2);
>>>>>           # Allow large messages
>>>>>           log_msg_size(65536);
>>>>> };
>>>>> 
>>>>> # =====================
>>>>> # UDP Packet Source
>>>>> source s_udp {
>>>>>           udp();
>>>>> };
>>>>> 
>>>>> # =====================
>>>>> # TCP Packet Source
>>>>> source s_tcp {
>>>>>            tcp(ip(aaa.bbb.ccc.ddd) port(514) max-connections(50000));
>>>>> };
>>>>> 
>>>>> # =====================
>>>>> destination net_perhost {
>>>>>           file("/data/log/per-host/$HOST"
>>>>>           owner(root)
>>>>>           group(nwadmin)
>>>>>           perm(0775)
>>>>>           );
>>>>> };
>>>>> 
>>>>> # =====================
>>>>> log {
>>>>>           source(s_tcp);
>>>>>           source(s_udp);
>>>>>           destination(net_perhost);
>>>>> };
>>>>> =====
>>>>> 
>>>>> I've checked around for perhaps a different spool name, thinking perhaps
>>>>> the data was getting recognized as something other than it's source-ip,
>>>>> but haven't seen anything.
>>>>> 
>>>>> Any thoughts?
>>>>> 
>>>>> Cheers,
>>>>> -Chris
>>>>> ______________________________________________________________________________
>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>> 
>>>> ______________________________________________________________________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>> 
>>> ______________________________________________________________________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>> 
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>> 
> 
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>