[syslog-ng] CSV parser and empty fields

[Balazs Scheidler]

On Fri, 2010-11-05 at 15:49 -0600, Bill Anderson wrote:
> I'm using a tab separated format from apache for access logs. My last two fields are referrer and user-agent. Obviously sometimes there is no referrer. Unfortunately when there isn't one apache only logs an empty string instead of the more common "-". This isn't a problem in scripts that parse the resulting logfile as they see the resulting empty field when I log $MSG. 
> 
> However, I just started a new log file that uses the csv-parser w/tab as delimiter and when the referrer field is empty, APACHE.USERAGENT (the last field) gets rolled into APACHE.REFERRER, the second to last field. As a result the template for this page (which uses APACHE.REFERRER) isn't reliable. When REFERRER is empty I want it to be empty (or something I can specify, like a default) not he next field in the parser definition.
> 
> I've look at the manual and don't see anything about handling empty fields. How do I get syslog-ng/csv-parser to log the empty field instead of moving to the next one?

hmm.. csv-parser should handle empty values just fine, provided the
separators are correct.

E.g. in case your referrer field is empty it is expecting:

tab tab user-agent

is that the case?

I've also added unit test cases to cover empty values with \t separated
values and it did work.

here's the patch:

commit f2801031604150c9ad6d1bfe842b61ec79131e1e
Author: Balazs Scheidler <bazsi at balabit.hu>
Date:   Sun Nov 14 10:52:59 2010 +0100

    test_csvparser: added testcase to cover empty values




-- 
Bazsi