[syslog-ng] help with SDATA

[Chris Johnson]

Thanks for the reply. I am following your recommendation to create my own pair from the SDATA. With that in mind, here is my current configuration

#####Sources#####
source s_network {
        udp(tags("firewall"));
};

#####filter#####
filter f_session_close {message("RT_FLOW_SESSION_CLOSE"); };

#####Destinations#####
destination d_mongodb {
        mongodb(
                value-pairs(
                        pair("reason","${.SDATA.junos at 2636.1.1.1.2.36.reason}")
                        scope("base")
                )
        );
};

#####Log#####
log {
        source(s_network);
        filter(f_session_close);
        destination(d_mongodb);
};


But what I am seeing in mongo is the following which does not have the pair defined in the destination statement:
{
	"DATE" : "Feb 10 09:48:25",
	"FACILITY" : "user",
	"HOST" : "X.X.X.X",
	"MESSAGE" : "2012-02-10T09:48:41.668  RT_FLOW - RT_FLOW_SESSION_CLOSE [junos at 2636.1.1.1.2.36 reason=\"unset\" source-address=\"X.X.X.x\" source-port=\"123\" destination-address=\"X.X.X.X\" destination-port=\"123\" service-name=\"junos-ntp\" nat-source-address=\"X.X.X.X\" nat-source-port=\"13773\" nat-destination-address=\"X.X.X.X\" nat-destination-port=\"123\" src-nat-rule-name=\"source-nat-rule\" dst-nat-rule-name=\"None\" protocol-id=\"17\" policy-name=\"trust-to-untrust\" source-zone-name=\"trust\" destination-zone-name=\"untrust\" session-id-32=\"30391\" packets-from-client=\"1\" bytes-from-client=\"76\" packets-from-server=\"1\" bytes-from-server=\"76\" elapsed-time=\"60\" application=\"N/A\" nested-application=\"N/A\" username=\"N/A\" role=\"N/A\" packet-incoming-interface=\"vlan.0\"]",
	"PRIORITY" : "info",
	"PROGRAM" : "1",
	"_id" : ObjectId("4f355869aa8f27451a000001")
}

If there is any additional information I can provide, please let me know, I will be happy to do so.
Thanks again for all of your help
Chris


On Feb 10, 2012, at 5:11 AM, Gergely Nagy wrote:

> Chris Johnson <masterof0 at gmail.com> writes:
> 
>> Based on the following structured syslog, I am trying to extract the reason from the SDATA portion of the log.
>> 
>> 2012-02-05T16:24:45.368  RT_FLOW - RT_FLOW_SESSION_CLOSE [junos at 2636.1.1.1.2.36 reason=\”unset\” ]
>> 
>> Problem I think I am running into is the value-pair that I am trying to extract includes the dots so the parser thinks its nested information
>> 
>> destination d_mongodb {
>>        mongodb(
>>                value-pairs(
>> 			scope("everything")
>>                        key(".SDATA.junos at 2636.1.1.1.2.36.reason")
>>                )
>>        );
>> };
>> 
>> 
>> Thoughts? Any help is greatly appreciated.
> 
> I just tried this with syslog-ng 3.4, without the key() (because
> scope("everything") already includes the key), and the reason ended up
> like this:
> 
>> db.messages.find({}, {_SDATA: 1, _id: 0})
> { "_SDATA" : { "junos at 2636" : { "1" : { "1" : { "1" : { "2" : { "36" : { "reason" : "unset" } } } } } } } }
> 
> This does look awkward, indeed. If you just want the reason, with a
> better key name, then the best you can do at the moment is to use
> something like the following:
> 
>  value-pairs (
>    pair("SDATA.junos at 2636-1-1-1-2-36.reason",
>         "${.SDATA.junos at 2636.1.1.1.2.36.reason}")
>    scope("base")
>  );
> 
> The same should work in 3.3, too. If it doesn't, then I screwed
> something up terribly.
> 
> Nevertheless, an easy way to replace recurring patterns in a value-pair
> key name would be neat, and would fit into the key rewrite feature
> already in 3.4... I'll think about this a little more.
> 
> -- 
> |8]
> 
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>