[syslog-ng] Parse client IP out of Proxy Protocol Line in TCP syslog->ELB->syslog-ng

[Nadine Miller]

Thanks Tibor and Laszlo, I will explore both of these options.

=Nadine=

--
Nadine Miller
Principal Security Engineer
email: nadine.miller at defpoint.com
phone: (408) 667-9004 (Eastern)

On Oct 16, 2015, at 4:04 AM, Tibor Benke <ihrwein at gmail.com> wrote:

Maybe the csv-parser would also work with space as a separator character.

2015-10-16 9:53 GMT+02:00 Budai, László <laszlo.budai at balabit.com>:
Hi,

a more specific example that I've created (just a POC):

@version: 3.7
@include "scl.conf"

#PROXY_STRING + single space + INET_PROTOCOL + single space + CLIENT_IP +
single space + PROXY_IP + single space + CLIENT_PORT + single space +
PROXY_PORT + "\r\n"
python {
import socket
import json
def aws_elb_proxy_protocol2json(logmsg, arg):
  out = {}
  token_separator = ' '
  tokens = arg.split(token_separator)
  if not tokens or len(tokens) < 6:
    return json.dumps({"aws_elb_proxy_protocol2json.error": "split
failure", "message": arg})

  out["proxy"] = tokens[0]
  out["inet_protocol"] = tokens[1]
  out["client_ip"] = tokens[2]
  out["proxy_ip"] = tokens[3]
  out["client_port"] = tokens[4]
  out["proxy_port"] = tokens[5]

  return json.dumps(out)
};

block parser aws_elb2json (
  template("${MSG}")
  rec-sep(' ')
  field-sep(' ')
) {
  json-parser(template("$(python aws_elb_proxy_protocol2json `template`)"));
};

source s_aws_elb {
  file("/tmp/aws-elb.log" flags(no-parse));
};

destination d_client_port_odd_json {
    file("/tmp/aws_elb_client_port_odd_json.log" template("$(format-json -s
nv-pairs)\n"));
};

destination d_client_port_even {
    #file("/tmp/aws_elb_client_port_even.log" template("$(format-json -s
nv-pairs)\n"));
    file("/tmp/aws_elb_client_port_even.log");
};

filter f_client_port_odd { match("\d*[13579]$" value("client_port")); };
filter f_client_port_even { match("\d*[02468]$" value("client_port")); };

log {
    source(s_aws_elb);
    parser { aws_elb2json(); };

    filter(f_client_port_odd);
    destination(d_client_port_odd_json);
};

log {
    source(s_aws_elb);
    parser { aws_elb2json(); };

    filter(f_client_port_even);
    destination(d_client_port_even);
};


regards,
L.

On Thu, Oct 15, 2015 at 7:28 PM, Nadine Miller <nadine.miller at defpoint.com
> wrote:
I've searched through the archives and spent some time trying to find
possible answers on the web, but haven't found a definitive answer.

I'm in a situation where I need to parse syslog streams being
forwarded through an AWS ELB. The normal configuration of the ELB
resets the source IP to be the ELB's IP address. Logs are coming from
multiple AWS VPCs, and we've already discovered duplicate hostnames
across different VPCs, which has mingled logs from different hosts
into one receiving log file.

The ELB has another mode, referred to as "Proxy Protocol" which adds a
single line to the TCP stream in the form:

PROXY_STRING + single space + INET_PROTOCOL + single space + CLIENT_IP
+ single space + PROXY_IP + single space + CLIENT_PORT + single space
+ PROXY_PORT + "\r\n"

Example:

PROXY TCP4 198.51.100.22 203.0.113.7 35646 80\r\n

Is it possible to use this proxy line in syslog-ng to properly
segregate the log messages? If so, what would be the best method to
use? I've done a lot of filtering/templating with normal UDP syslog
and syslog-ng, but this is the first time I've had to consider
something crazy like this.

Currently there is no option at this time to change configurations at
endpoints sending the  syslog messages, nor can we remove the ELB.

For reference:
http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/enable-proxy-protocol.html

Thanks in advance--
=N=
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation:
http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq



______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation:
http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq



______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation:
http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20151020/a808d707/attachment.htm