[syslog-ng] Program() in destination driver not working for Macros defined in CSV-PARSER

[Evan Rempel]

I don't have anything quite as elaborate as what you have, but here is 
an example I have running in my production environment


template t_rule_id { template("${.classifier.rule_id}\n"); 
template_escape(no); };

destination d_msgid_profiler {
         program("/opt/flare/bin/msgid_profiler" template(t_rule_id) );
};

I use a patterndb message parser to end up with the macro 
.classifier.rule_id and then send that to my program.

Basically what you have done.

One of the things that I do to troubleshoot things like this is to make 
a destination such as

template t_json { template("$(format-json --scope everything )\n"); 
template_escape(no); };
destination d_syslog_json    { file("/var/log/syslog-json.log" 
owner("root") group("syslogs") perm(0640) template(t_json)); };

and then add this destination right beside the one that you are having 
problems with.

log {
     source(your_source);
     destination(d_mesg);
     destination(d_syslog_json);
};

and then you can look at all of the macros that are present in the 
/var/log/syslog-json.log file and verify
what you think is happening.

To look at the json object in an easy to use manner, see 
http://jsonprettyprint.com/

Evan.



On 03/09/2014 04:50 PM, Justin B wrote:
> Can you share me a sample of the shell script that I can write to read 
> the message lines that are being passed and extract those values into 
> a different variables and output them into output log files??
>
> On Fri, Mar 7, 2014 at 4:10 PM, Balazs Scheidler <bazsi77 at gmail.com 
> <mailto:bazsi77 at gmail.com>> wrote:
>
>     You can on stdin but not as arguments.
>
>     On Mar 7, 2014 9:24 PM, "Justin B" <justinkala at gmail.com
>     <mailto:justinkala at gmail.com>> wrote:
>
>         Yes Empty Fields.
>         I am using this log path
>         log { source (remote); filter (f_messages); parser
>         (p_apache);destination (r_messages);   };
>         Are you saying the I cannot pass the Macro values to a shell
>         script through Program on destination driver??
>
>         On Fri, Mar 7, 2014 at 2:47 PM, Evan Rempel <erempel at uvic.ca
>         <mailto:erempel at uvic.ca>> wrote:
>
>             My recollection is that macros are not expanded for
>             program names. Whant you get is the environment variables from
>             the shell that is used to start your program, so in most
>             cases this will be empty.
>
>             I think this is dangerous and did mantion it on the list
>             previously.
>
>
>             On 03/06/2014 10:32 PM, Balazs Scheidler wrote:
>             > What do you get in your script? Empty fields?
>             >
>             > The program destination has to be on a direct log path
>             subsequent to the parser.
>             >
>             > On Mar 4, 2014 7:27 PM, "Justin B" <justinkala at gmail.com
>             <mailto:justinkala at gmail.com> <mailto:justinkala at gmail.com
>             <mailto:justinkala at gmail.com>>> wrote:
>             >
>             >
>             >     Hello
>             >     On My Apache logs I applied csv_parser() and defined
>             the Macros.
>             >       parser p_apache {
>             >     csv-parser(columns("apache.ETSTAMP", "apache.TYPE",
>             "apache.EHOSTNAME","apache.ESOURCE", "apache.EOUTCOME",
>             "apache.EMSG","apache.EUSERID")
>             >     delimiters("|")  );
>             >     };
>             >     I want to launch a script whenever the UDP messages
>             are in.So I defined the
>             >       destination d_mesg (program("/tmp/test.sh"
>             template("|${apache.ETSTAMP}|${apache.TYPE}|${apache.EHOSTNAME}|${apache.ESOURCE}|${apache.EOUTCOME}|${apache.EMSG}|${apache.EUSERID}\n"));
>             };
>             >     script is working fine with other destination
>             drivers. Please help
>             >     --
>             >     Kale
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20140309/6944e9cc/attachment-0001.htm