[syslog-ng] Parse client IP out of Proxy Protocol Line in TCP syslog->ELB->syslog-ng

[Balazs Scheidler]

I gave a simpler solution at the end of.the thread.
On Oct 20, 2015 16:27, "Nadine Miller" <nadine.miller at defpoint.com> wrote:

> Thanks Tibor and Laszlo, I will explore both of these options.
>
> =Nadine=
>
> --
> Nadine Miller
> Principal Security Engineer
> email: nadine.miller at defpoint.com
> phone: (408) 667-9004 (Eastern)
>
> On Oct 16, 2015, at 4:04 AM, Tibor Benke <ihrwein at gmail.com> wrote:
>
> Maybe the csv-parser would also work with space as a separator character.
>
> 2015-10-16 9:53 GMT+02:00 Budai, László <laszlo.budai at balabit.com>:
> Hi,
>
> a more specific example that I've created (just a POC):
>
> @version: 3.7
> @include "scl.conf"
>
> #PROXY_STRING + single space + INET_PROTOCOL + single space + CLIENT_IP +
> single space + PROXY_IP + single space + CLIENT_PORT + single space +
> PROXY_PORT + "\r\n"
> python {
> import socket
> import json
> def aws_elb_proxy_protocol2json(logmsg, arg):
>   out = {}
>   token_separator = ' '
>   tokens = arg.split(token_separator)
>   if not tokens or len(tokens) < 6:
>     return json.dumps({"aws_elb_proxy_protocol2json.error": "split
> failure", "message": arg})
>
>   out["proxy"] = tokens[0]
>   out["inet_protocol"] = tokens[1]
>   out["client_ip"] = tokens[2]
>   out["proxy_ip"] = tokens[3]
>   out["client_port"] = tokens[4]
>   out["proxy_port"] = tokens[5]
>
>   return json.dumps(out)
> };
>
> block parser aws_elb2json (
>   template("${MSG}")
>   rec-sep(' ')
>   field-sep(' ')
> ) {
>   json-parser(template("$(python aws_elb_proxy_protocol2json
> `template`)"));
> };
>
> source s_aws_elb {
>   file("/tmp/aws-elb.log" flags(no-parse));
> };
>
> destination d_client_port_odd_json {
>     file("/tmp/aws_elb_client_port_odd_json.log" template("$(format-json
> -s nv-pairs)\n"));
> };
>
> destination d_client_port_even {
>     #file("/tmp/aws_elb_client_port_even.log" template("$(format-json -s
> nv-pairs)\n"));
>     file("/tmp/aws_elb_client_port_even.log");
> };
>
> filter f_client_port_odd { match("\d*[13579]$" value("client_port")); };
> filter f_client_port_even { match("\d*[02468]$" value("client_port")); };
>
> log {
>     source(s_aws_elb);
>     parser { aws_elb2json(); };
>
>     filter(f_client_port_odd);
>     destination(d_client_port_odd_json);
> };
>
> log {
>     source(s_aws_elb);
>     parser { aws_elb2json(); };
>
>     filter(f_client_port_even);
>     destination(d_client_port_even);
> };
>
>
> regards,
> L.
>
> On Thu, Oct 15, 2015 at 7:28 PM, Nadine Miller <nadine.miller at defpoint.com
> > wrote:
> I've searched through the archives and spent some time trying to find
> possible answers on the web, but haven't found a definitive answer.
>
> I'm in a situation where I need to parse syslog streams being
> forwarded through an AWS ELB. The normal configuration of the ELB
> resets the source IP to be the ELB's IP address. Logs are coming from
> multiple AWS VPCs, and we've already discovered duplicate hostnames
> across different VPCs, which has mingled logs from different hosts
> into one receiving log file.
>
> The ELB has another mode, referred to as "Proxy Protocol" which adds a
> single line to the TCP stream in the form:
>
> PROXY_STRING + single space + INET_PROTOCOL + single space + CLIENT_IP
> + single space + PROXY_IP + single space + CLIENT_PORT + single space
> + PROXY_PORT + "\r\n"
>
> Example:
>
> PROXY TCP4 198.51.100.22 203.0.113.7 35646 80\r\n
>
> Is it possible to use this proxy line in syslog-ng to properly
> segregate the log messages? If so, what would be the best method to
> use? I've done a lot of filtering/templating with normal UDP syslog
> and syslog-ng, but this is the first time I've had to consider
> something crazy like this.
>
> Currently there is no option at this time to change configurations at
> endpoints sending the  syslog messages, nor can we remove the ELB.
>
> For reference:
>
> http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/enable-proxy-protocol.html
>
> Thanks in advance--
> =N=
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20151021/f594c3cc/attachment.htm