[syslog-ng] Parse client IP out of Proxy Protocol Line in TCP syslog->ELB->syslog-ng

[Tusa Viktor]

The backend server itself would be the syslog-ng. ELBs are not just a
simple HTTP proxies, they can act as a TCP forwarder/proxy too. It is a
common logging pattern in AWS to use ELBs to load balance log streams
between multiple log collection servers. To be able to keep the original
source IP and port, ELB uses the Proxy Protocol v1 (
http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt) which injects a
human readable proxy header at the beginning of the TCP stream. So if the
log stream is '\n' separated, the first log message would contain the proxy
header, but no other log message would contain any reference on the
original source ip.

So this problem cannot be solved without keeping the state throughout a
connection. Because ELB will open a new connection to the backend on every
incoming connection from the clients, syslog-ng will see different
connections for different clients. The source IP would be the ELB IP, but
the port would be varying.

With patterndb, you can parse out the proxy header line, and the store the
original IP in a log message field. Then you can write a python
rewrite/filter/template statement (don't know which would be the best)
which can store the original IP of the connection under the key
(source_ip,source_port) in a hash table or a Redis DB. With every log
message in a connection, you can look up the original IP of the connection
by using the key (source_ip, source_port), and add it to the log message.

Regards,
Viktor



On Sun, Oct 18, 2015 at 11:44 AM, Balazs Scheidler <bazsi77 at gmail.com>
wrote:

> Hi,
>
> While thinking about the subject (BTW, could you try the parser I sent?),
> the scenario you were describing doesn't add up.
>
> This line is sent to the backend server, not to a log server, how would
> you send this format to syslog-ng?
>
> Thanks
> On Oct 15, 2015 19:28, "Nadine Miller" <nadine.miller at defpoint.com> wrote:
>
>> I've searched through the archives and spent some time trying to find
>> possible answers on the web, but haven't found a definitive answer.
>>
>> I'm in a situation where I need to parse syslog streams being
>> forwarded through an AWS ELB. The normal configuration of the ELB
>> resets the source IP to be the ELB's IP address. Logs are coming from
>> multiple AWS VPCs, and we've already discovered duplicate hostnames
>> across different VPCs, which has mingled logs from different hosts
>> into one receiving log file.
>>
>> The ELB has another mode, referred to as "Proxy Protocol" which adds a
>> single line to the TCP stream in the form:
>>
>> PROXY_STRING + single space + INET_PROTOCOL + single space + CLIENT_IP
>> + single space + PROXY_IP + single space + CLIENT_PORT + single space
>> + PROXY_PORT + "\r\n"
>>
>> Example:
>>
>> PROXY TCP4 198.51.100.22 203.0.113.7 35646 80\r\n
>>
>> Is it possible to use this proxy line in syslog-ng to properly
>> segregate the log messages? If so, what would be the best method to
>> use? I've done a lot of filtering/templating with normal UDP syslog
>> and syslog-ng, but this is the first time I've had to consider
>> something crazy like this.
>>
>> Currently there is no option at this time to change configurations at
>> endpoints sending the  syslog messages, nor can we remove the ELB.
>>
>> For reference:
>>
>> http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/enable-proxy-protocol.html
>>
>> Thanks in advance--
>> =N=
>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation:
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20151018/d6d696e9/attachment.htm