<div dir="ltr">I see. But it&#39;s also a nice use-case for pattern matching support with the `match` keyword :)</div><div class="gmail_extra"><br><div class="gmail_quote">2015-10-16 10:07 GMT+02:00 Budai, László <span dir="ltr">&lt;<a href="mailto:laszlo.budai@balabit.com" target="_blank">laszlo.budai@balabit.com</a>&gt;</span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">You are right, but this is something what I wanted to try for some other reasons :-)<span class="HOEnZb"><font color="#888888"><div><div><br></div><div>L.</div></div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Oct 16, 2015 at 10:04 AM, Tibor Benke <span dir="ltr">&lt;<a href="mailto:ihrwein@gmail.com" target="_blank">ihrwein@gmail.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Maybe the csv-parser would also work with space as a separator character.</div><div class="gmail_extra"><br><div class="gmail_quote">2015-10-16 9:53 GMT+02:00 Budai, László <span dir="ltr">&lt;<a href="mailto:laszlo.budai@balabit.com" target="_blank">laszlo.budai@balabit.com</a>&gt;</span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>a more specific example that I&#39;ve created (just a POC):</div><div><br></div><div><div>@version: 3.7</div><div>@include &quot;scl.conf&quot;</div><div><br></div><div>#PROXY_STRING + single space + INET_PROTOCOL + single space + CLIENT_IP + single space + PROXY_IP + single space + CLIENT_PORT + single space + PROXY_PORT + &quot;\r\n&quot;</div><div>python {</div><div>import socket</div><div>import json</div><div>def aws_elb_proxy_protocol2json(logmsg, arg):</div><div>  out = {}</div><div>  token_separator = &#39; &#39;</div><div>  tokens = arg.split(token_separator)</div><div>  if not tokens or len(tokens) &lt; 6:</div><div>    return json.dumps({&quot;aws_elb_proxy_protocol2json.error&quot;: &quot;split failure&quot;, &quot;message&quot;: arg})</div><div><br></div><div>  out[&quot;proxy&quot;] = tokens[0]</div><div>  out[&quot;inet_protocol&quot;] = tokens[1]</div><div>  out[&quot;client_ip&quot;] = tokens[2]</div><div>  out[&quot;proxy_ip&quot;] = tokens[3]</div><div>  out[&quot;client_port&quot;] = tokens[4]</div><div>  out[&quot;proxy_port&quot;] = tokens[5]</div><div><br></div><div>  return json.dumps(out)</div><div>};</div><div><br></div><div>block parser aws_elb2json (</div><div>  template(&quot;${MSG}&quot;)</div><div>  rec-sep(&#39; &#39;)</div><div>  field-sep(&#39; &#39;)</div><div>) {</div><div>  json-parser(template(&quot;$(python aws_elb_proxy_protocol2json `template`)&quot;));</div><div>};</div><div> </div><div>source s_aws_elb {</div><div>  file(&quot;/tmp/aws-elb.log&quot; flags(no-parse));</div><div>};</div><div> </div><div>destination d_client_port_odd_json {</div><div>    file(&quot;/tmp/aws_elb_client_port_odd_json.log&quot; template(&quot;$(format-json -s nv-pairs)\n&quot;));</div><div>};</div><div><br></div><div>destination d_client_port_even {</div><div>    #file(&quot;/tmp/aws_elb_client_port_even.log&quot; template(&quot;$(format-json -s nv-pairs)\n&quot;));</div><div>    file(&quot;/tmp/aws_elb_client_port_even.log&quot;);</div><div>};</div><div> </div><div>filter f_client_port_odd { match(&quot;\d*[13579]$&quot; value(&quot;client_port&quot;)); };</div><div>filter f_client_port_even { match(&quot;\d*[02468]$&quot; value(&quot;client_port&quot;)); };</div><div><br></div><div>log {</div><div>    source(s_aws_elb);</div><div>    parser { aws_elb2json(); };</div><div><br></div><div>    filter(f_client_port_odd);</div><div>    destination(d_client_port_odd_json);</div><div>};</div><div><br></div><div>log {</div><div>    source(s_aws_elb);</div><div>    parser { aws_elb2json(); };</div><div><br></div><div>    filter(f_client_port_even);</div><div>    destination(d_client_port_even);</div><div>};</div></div><div><br></div><div><br></div><div>regards,</div><div>L.</div></div><div class="gmail_extra"><br><div class="gmail_quote"><span>On Thu, Oct 15, 2015 at 7:28 PM, Nadine Miller <span dir="ltr">&lt;<a href="mailto:nadine.miller@defpoint.com" target="_blank">nadine.miller@defpoint.com</a>&gt;</span> wrote:<br></span><div><div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I&#39;ve searched through the archives and spent some time trying to find<br>
possible answers on the web, but haven&#39;t found a definitive answer.<br>
<br>
I&#39;m in a situation where I need to parse syslog streams being<br>
forwarded through an AWS ELB. The normal configuration of the ELB<br>
resets the source IP to be the ELB&#39;s IP address. Logs are coming from<br>
multiple AWS VPCs, and we&#39;ve already discovered duplicate hostnames<br>
across different VPCs, which has mingled logs from different hosts<br>
into one receiving log file.<br>
<br>
The ELB has another mode, referred to as &quot;Proxy Protocol&quot; which adds a<br>
single line to the TCP stream in the form:<br>
<br>
PROXY_STRING + single space + INET_PROTOCOL + single space + CLIENT_IP<br>
+ single space + PROXY_IP + single space + CLIENT_PORT + single space<br>
+ PROXY_PORT + &quot;\r\n&quot;<br>
<br>
Example:<br>
<br>
PROXY TCP4 198.51.100.22 203.0.113.7 35646 80\r\n<br>
<br>
Is it possible to use this proxy line in syslog-ng to properly<br>
segregate the log messages? If so, what would be the best method to<br>
use? I&#39;ve done a lot of filtering/templating with normal UDP syslog<br>
and syslog-ng, but this is the first time I&#39;ve had to consider<br>
something crazy like this.<br>
<br>
Currently there is no option at this time to change configurations at<br>
endpoints sending the  syslog messages, nor can we remove the ELB.<br>
<br>
For reference:<br>
<a href="http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/enable-proxy-protocol.html" rel="noreferrer" target="_blank">http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/enable-proxy-protocol.html</a><br>
<br>
Thanks in advance--<br>
=N=<br>
______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
</blockquote></div></div></div><br></div>
<br>______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
<br></blockquote></div><br></div>
<br>______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
<br></blockquote></div><br></div>
</div></div><br>______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
<br></blockquote></div><br></div>