The backend server itself would be the syslog-ng. ELBs are not just a simple HTTP proxies, they can act as a TCP forwarder/proxy too. It is a common logging pattern in AWS to use ELBs to load balance log streams between multiple log collection servers. To be able to keep the original source IP and port, ELB uses the Proxy Protocol v1 (http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt) which injects a human readable proxy header at the beginning of the TCP stream. So if the log stream is '\n' separated, the first log message would contain the proxy header, but no other log message would contain any reference on the original source ip.

So this problem cannot be solved without keeping the state throughout a connection. Because ELB will open a new connection to the backend on every incoming connection from the clients, syslog-ng will see different connections for different clients. The source IP would be the ELB IP, but the port would be varying.

With patterndb, you can parse out the proxy header line, and the store the original IP in a log message field. Then you can write a python rewrite/filter/template statement (don't know which would be the best) which can store the original IP of the connection under the key (source_ip,source_port) in a hash table or a Redis DB. With every log message in a connection, you can look up the original IP of the connection by using the key (source_ip, source_port), and add it to the log message.

Regards,

Viktor

On Sun, Oct 18, 2015 at 11:44 AM, Balazs Scheidler <bazsi77@gmail.com> wrote:

Hi,

While thinking about the subject (BTW, could you try the parser I sent?), the scenario you were describing doesn't add up.

This line is sent to the backend server, not to a log server, how would you send this format to syslog-ng?

Thanks

On Oct 15, 2015 19:28, "Nadine Miller" <nadine.miller@defpoint.com> wrote:

I've searched through the archives and spent some time trying to find
possible answers on the web, but haven't found a definitive answer.

I'm in a situation where I need to parse syslog streams being
forwarded through an AWS ELB. The normal configuration of the ELB
resets the source IP to be the ELB's IP address. Logs are coming from
multiple AWS VPCs, and we've already discovered duplicate hostnames
across different VPCs, which has mingled logs from different hosts
into one receiving log file.

The ELB has another mode, referred to as "Proxy Protocol" which adds a
single line to the TCP stream in the form:

PROXY_STRING + single space + INET_PROTOCOL + single space + CLIENT_IP
+ single space + PROXY_IP + single space + CLIENT_PORT + single space
+ PROXY_PORT + "\r\n"

Example:

PROXY TCP4 198.51.100.22 203.0.113.7 35646 80\r\n

Is it possible to use this proxy line in syslog-ng to properly
segregate the log messages? If so, what would be the best method to
use? I've done a lot of filtering/templating with normal UDP syslog
and syslog-ng, but this is the first time I've had to consider
something crazy like this.

Currently there is no option at this time to change configurations at
endpoints sending the  syslog messages, nor can we remove the ELB.

For reference:
http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/enable-proxy-protocol.html

Thanks in advance--
=N=
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq