[syslog-ng] CSV parser and empty fields

[Bill Anderson]

> 
> On Fri, Nov 05, 2010 at 03:49:05PM -0600, Bill Anderson wrote:
>> I'm using a tab separated format from apache for access logs. My last two fields are referrer and user-agent. Obviously sometimes there is no referrer. Unfortunately when there isn't one apache only logs an empty string instead of the more common "-". This isn't a problem in scripts that parse the resulting logfile as they see the resulting empty field when I log $MSG. 
>> 
>> However, I just started a new log file that uses the csv-parser w/tab as delimiter and when the referrer field is empty, APACHE.USERAGENT (the last field) gets rolled into APACHE.REFERRER, the second to last field. As a result the template for this page (which uses APACHE.REFERRER) isn't reliable. When REFERRER is empty I want it to be empty (or something I can specify, like a default) not he next field in the parser definition.
>> 
>> I've look at the manual and don't see anything about handling empty fields. How do I get syslog-ng/csv-parser to log the empty field instead of moving to the next one?
...

On Nov 5, 2010, at 4:55 PM, Matthew Hall wrote:

> I think you need to configure some of your flags to the parser.
> 
> Did you try something like these directions here:
> 
> https://www.icts.uiowa.edu/confluence/display/ICTSit/Using+syslog-ng+to+collect+remote+Apache+web+server+logs
> 
> Matthew.


Thanks for your reply, Mathew. Perhaps I wasn't clear enough. The syslog-ng produced logfile that logs $MSG is *just fine*. The tabs are there, and anything that parses it and expects the fields gets them just fine. The problem arises when the template only needs to log fields from the csv parser and a preceding field is empty. If there are any flags on that page that affect how the csv-parser handles empty fields, I'd appreciate them being pointed out, as I didn't see any.


Cheers,
Bill