<div>Not being able to send to multiple sources can definitely be a bug in the version you are using.</div><div>I had to go to a later version that the one packaged with centos7 to get multiple destinations working.</div><div><br></div><div>We use Elasticsearch too, we take events and pass them through json-format inside of an amqp configuration using syslog-ng's amqp plugin to a rabbit cluster.</div><div><br></div><div>Then we pick up the events from rabbit using workers that do any normalization and enrichment needed before passing to Elasticsearch.</div><div>All of our data is then ingested by ES already structured.</div><div><br></div><div>Alicia</div><div><br></div><div><br><div class="gmail_quote"><div dir="ltr">On Wed, Feb 6, 2019, 12:55 PM Garridan <<a href="mailto:garridan@gmail.com">garridan@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div>It's logs from Cisco ASA firewalls. It doesn't seem to get hung up on any particular message it just starts when I add a device that matches a different filter. A representative config is below...</div><div><br></div><div>So if I write using devices that match filter1, it works fine. When I increase the load and start adding in the devices that match filter2, the service starts restarting and eventually the SQL message is thrown.<br></div><div><br></div><div>Again, I'm a noob who's still learning and not a dev, just a firewall guy. <br></div><div><br></div><div>Thanks for any help!<br></div><div><br></div><div>source s_source1 {<br> network(<br> ip("192.168.100.1")<br> transport("udp")<br> port(514)<br> );<br>};<br><br>destination d_device1 {<br> sql(type(mssql)<br> host("dbserver")<br> port("1433")<br> username("syslogng")<br> password("syslogng")<br> database("syslogng")<br> table("device1")<br> columns("Date varchar(10)", "Time varchar(8)", "Priority varchar(30)", "Hostname varchar(255)", "Text varchar(4096)")<br> values("${R_MONTH}-${R_DAY}-${R_YEAR}", "${R_HOUR}:${R_MIN}:${R_SEC}", "${LEVEL_NUM}", "${HOST}", "${MSGHDR} ${MESSAGE}"));<br>};<br><br>destination d_device2 {<br> sql(type(mssql)<br> host("dbserver")<br> port("1433")<br> username("syslogng")<br> password("syslogng")<br> database("syslogng")<br> table("device2")<br> columns("Date varchar(10)", "Time varchar(8)", "Priority varchar(30)", "Hostname varchar(255)", "Text varchar(4096)")<br> values("${R_MONTH}-${R_DAY}-${R_YEAR}", "${R_HOUR}:${R_MIN}:${R_SEC}", "${LEVEL_NUM}", "${HOST}", "${MSGHDR} ${MESSAGE}"));<br>};<br><br>filter f_device1 {<br> host("192.168.1.1") or host("192.168.1.2");<br>};<br><br>filter f_device2 {<br> host("192.168.2.1") or host("192.168.2.2");<br>};<br><br>log {<br> source(s_source1);<br> filter(f_device1);<br> destination(d_device1);<br> };<br><br>log {<br> source(s_source1);<br> filter(f_device2);<br> destination(d_device2);<br> };<br><br></div></div></div><div dir="ltr"><div dir="ltr"><div dir="auto"><br></div><div dir="auto"><br><div dir="ltr">On Feb 6, 2019, at 1:01 PM, Alicia Smith <<a href="mailto:asmith@mozilla.com" target="_blank">asmith@mozilla.com</a>> wrote:<br><br></div><blockquote type="cite"><div dir="ltr">You'll want to tune your config according to the resources available and the throughput it requires.<div><br></div><div>I can follow up with a link on how to do that.</div><div><br></div><div>Are you using json format from syslog-ng? </div><div>Can you provide an example event that it's getting hung up on?</div><div><br></div><div>Alicia</div><div><br><br><div class="gmail_quote"><div dir="ltr">On Wed, Feb 6, 2019, 11:42 AM Garridan <<a href="mailto:garridan@gmail.com" target="_blank">garridan@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hello! I'm a new syslog-ng user, so please be gentle with me. :)</div><div><br></div><div>I'm attempting to log to an MS-SQL database and would like to send to different tables in the same DB based on the source IP - for example device A to its own table, device B to its own table, and so on.<br></div><div><br></div><div> I thought I would simply need to create the same destination but define different table names in each, it works, however under load the syslog-ng service starts restarting over and over and eventually MS-SQL errors and alerts that the login packet is structurally invalid.<br></div><div><br></div><div>Is it possible to send to different table names in this manner or is there another way to do it?</div><div><br></div><div>Thanks!</div><div><br></div></div>
______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
</blockquote></div></div>
</div></blockquote><blockquote type="cite"><div dir="ltr"><span>______________________________________________________________________________</span><br><span>Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a></span><br><span>Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a></span><br><span>FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a></span><br><span></span><br></div></blockquote></div></div></div>
______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
</blockquote></div></div>