-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Thank a lot for the help. I tested with syslog-ng 3.4.0 alpha3 and its working but I use 3.3 :-( So I will use regex and move to json parser when 3.4 will be stable :) Thanks again ! Sébastien On 10/05/2012 12:00 PM, syslog-ng-request@lists.balabit.hu wrote:

Sebastien Pasche <braoru@gmail.com> writes:

...

...

I will present to you what I want to do and what I actually have.

I would like to extract a field from a json log arriving in this source :

source s_collector_tcp_json { tcp(ip(0.0.0.0) port(514) flags(no-multi-line) flags(no-parse)); };

And replacing the Program field I use in my destination [...] from the field @type of this json log :

{ "@source": "tcp://127.0.0.1:9999/client/127.0.0.1:57530", "@type": "tomcat_logstash_raw_json", "@tags": [ "tomcat_site" ], "@fields": { "priority": "INFO", "logger_name": "com.zzz.user.UserData", "thread": "TP-Processor7", "class": "org.apache.jsp.WEB_002dINF.jsp.user.ViewInvoiceDetail_jsp", "file": "ViewInvoiceDetail_jsp.java:162", "method": "_jspService", "prop_userIp": "192.168.215.50", "prop_userId": "1440704" }, "@source_host": "127.0.0.1:57530", "@source_path": "com.leshop.user.UserData", "@message": "order : {WAREHOUSE_TYPE=drive, OID=5693367, ORDER_DATE=2012-10-03 08:49:17.41, SHIPPING_FRESH=0.0, FROZEN_DEPOSIT=0.0, WAREHOUSE_ID=5, DUE_AMOUNT=0.0, TOTAL_CREDITS=0.0, ADDRESS_NUMBER=, DELIV_HELPFUL_INDICATION=, DELIVERY_MODE=20:00, DELIVERY_DATE=2012-10-03 00:00:00.0, TOTAL=134.75, ACTION_TOTAL=0.0, ORDER_NUMBER=abc-014085706-xyz, TRACK_TRACE=, RETAILER_GROUP=0, ZIP=, ORDER_STATE=3, PAYMENT_TYPE=7, DELIV_DOORCODE=, FROZEN_FEES=0.0, ENV_CO2=0.0, NAME= , ENV_CO2_RETAIL=0.0, HIDE_BVR=false, ADDRESS=, TOTAL_CREDIT=0.0, MODIFICATION_STATE=1, REMINDER_LEVEL=0, SUBTOTAL=134.75, GRAND_TOTAL=134.75, BVR_REFERENCE=, CITY=, DELIV_PHONE=, SHIPPING_FIXED=0.0}", "@timestamp": "2012-10-03T06:49:23.373000Z" } [...]

Assuming that the JSON arrives on a single line, something along these lines should do the trick:

parser p_tomcat_json { json-parser(prefix("json.")); };

rewrite rw_tomcat_site_logstash_json_program_name { set("${json.type}", value("$PROGRAM")); };

And then chain it together:

log { source(s_collector_tcp_json); parser(p_tomcat_json); rewrite(rw_tomcat_site_logstash_json_program_name); destination(d_file_normal_r); };

Hope that helps!

-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iQIcBAEBAgAGBQJQbrE/AAoJEE3IBph3MKVPidkQALooeGZRm2FEIGPN7uqJVrRc 2yeQ0L68Nny0WLnR2mZU4nOy7EHMKLPufrYINAp1FyNM7U4xDOkLTxoVhkPtejis hKce2Z/djJX1cABlPl0qf3ClBsSnH7oe745BvSdXUnBXFGc7mBV7E10GF4tb4Ce/ ElUUEx2UyIfCByOcETOJkvZrY1WSky1MqJZLI5dX2BzjKsdBML5Bmi/pj1VVRCaa WUC6lkSnGUdoyAdVCF7hHzNwQbm1txTUjsdo1oUvdaly4ASDkHWuHhRXsNw9gugA 3TRNugj/69oeOzxZkt/eVukpcK2JbeJ6UGVWpYe8Vo3x5slHXT5tObOcIni4eCYU /DnGFs7UT1nGmQ8HYTkytgnDUa3dpAcVU60xdyLsjkx7WPUmLjKJcKD//cXdJXH/ uzI3WvEEqnPqb3oRJfNqrR0Ikn1tF0jOHpJxeW3lQIYMaadahdTUQ1dhccycUWrB ISEoy1tFSUJepOHcZf+RGRTopxJVxvpK7JondR3oV0BMDf00Vgu8SLmuWl/ok3aZ KFAZ019kMYX1Bq0sCAtOyJwWzIXUFEtMz8rEkc1V0fbAI032WY6IzUecWuLGM783 uZ/C2bDBR2Xgbs/oDMhZImKJzjouN8IGuNK9ZaRMH0Qx0dx4wjc8QORD9hB2kmmF lEjQIEQ8D8ADAavqwURp =pUIX -----END PGP SIGNATURE-----