[syslog-ng] Adding specific information from Syslog-ng to Redis

[Jim Hendrick]

Another option where you can assign name-value pairs yourself:
Have you looked at patterndb ? Here are a couple snippets that are working
well for me:

parser p_proxy {
  db-parser(file("/usr/local/etc/patterndb.d/proxy.xml"));
};

destination d_redis {
  redis (
    host("localhost")
    command("LPUSH", "logstash", "$(format-json
type=proxyproxy_time=${PROXY.TIME} proxy_time_taken=${PROXY.TIME_TAKEN}
proxy_c_ip=${PROXY.C_IP} proxy_sc_status=${PROXY.SC_STATUS}
proxy_s_action=${PROXY.S_ACTION} proxy_sc_bytes=int64(${PROXY.SC_BYTES})
proxy_cs_bytes=int64(${PROXY.CS_BYTES}) proxy_cs_method=${PROXY.CS_METHOD}
proxy_cs_uri_scheme=${PROXY.CS_URI_SCHEME} proxy_cs_host=${PROXY.CS_HOST}
proxy_cs_uri_port=${PROXY.CS_URI_PORT}
proxy_cs_uri_path=${PROXY.CS_URI_PATH}
proxy_cs_uri_equery=${PROXY.CS_URI_EQUERY}
 proxy_cs_username=${PROXY.CS_USERNAME}
proxy_cs_auth_group=${PROXY.CS_AUTH__GROUP}
proxy_s_supplier_name=${PROXY.S_SUPPLIER_NAME}
proxy_content_type=${PROXY.CONTENT_TYPE} proxy_referrer=${PROXY.REFERRER}
proxy_user_agent=${PROXY.USER_AGENT}
proxy_filter_result=${PROXY.FILTER_RESULT}
proxy_cs_categories=${PROXY.CS_CATEGORIES}
proxy_x_virus_id=${PROXY.X_VIRUS_ID} proxy_s_ip=${PROXY.S_IP}
proxy_any=${PROXY.ANYREST})\n")
  );
};
log {
  source(s_network);
  parser(p_proxy);
  destination(d_redis);
};


Hope this helps.

Jim

On Sun, Dec 25, 2016 at 9:27 AM, Scheidler, Balázs <
balazs.scheidler at balabit.com> wrote:

> I would embed json formatted strings as redis list elements.
>
> You can format that using $(format-json)
>
>
> On Dec 24, 2016 1:17 AM, <johnsc301 at gmail.com> wrote:
>
>> I am trying to send information from Syslog-ng to Redis. In
>> /etc/syslog-ng/syslog-ng.conf I added this: destination d_redis { redis(
>> host("127.0.0.1") port(6379) command("RPUSH", "sensor_name",
>> "${sensor_name}")); };
>>
>>
>>
>> I am trying to create a list of variables, hopefully one being timestamp.
>> Here is an example of my syslog that I am trying to pull:
>> http://pastebin.com/Hx5vW4VA
>>
>> Here's is syslog-ng.conf, for reference: http://pastebin.com/2VQFBNmK
>>
>> Those are logs being sent from Snort to Syslog-ng through Snort. I want
>> to connect to Redis.
>>
>> I saw that the command parameters are: comma-separated list of strings
>> ("<redis-command>", "<first-command-parameter>",
>> "<second-command-parameter>", "<third-command-parameter>") from:
>> https://www.balabit.com/documents/syslog-ng-ose-latest
>> -guides/en/syslog-ng-ose-guide-admin/html/reference-
>> destination-redis.html
>>
>>  I'm assuming I'd say RPUSH <something>... However, I am unsure of how
>> to find the correct parameters.
>>
>> Specifically, for now, I want to create a list (RPUSH) of timestamps, IP
>> addresses (to and from), and event type (ICMP, for example).
>>
>> I did find this list of parameters:
>>
>> https://www.balabit.com/documents/syslog-ng-ose-latest-
>> guides/en/syslog-ng-ose-guide-admin/html/syslog-ng-parameter-index.html
>>
>> A good first try, I'd like to make a list of timestamps. How can I set
>> the d_redis(command()) within syslog-ng.conf to do this?
>>
>>
>>
>>
>>
>> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
>> Windows 10
>>
>>
>>
>> ____________________________________________________________
>> __________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=
>> syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
>>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20161225/d69f46c3/attachment-0001.html>