Martin, how do I setup the config file to log to a CSV file so I can try the BCP method? I assume since BCP is a Windows program then I need to use SAMBA in Linux to create a share that BCP can read the log files from? On Thu, Mar 10, 2011 at 10:23 AM, Martin Holste <mcholste@gmail.com> wrote:
Yep, you're dropping most of your messages to SQL. Your dropped stats should equal 0 if you're not dropping any messages. The stats are listed by source and destination so you can see if a certain source can't read the messages fast enough and a certain destination can't write fast enough. I bet if you setup a simple file destination you'll see that it doesn't drop any at all while SQL continues to drop most messages. That's because writing to SQL is much more expensive than writing to file. One solution to this is to write to a file in CSV format and use MS-SQL bcp to load the files written out in batches. If you name the output files from syslog-ng by the minute of the timestamp, then you can have a simple script pick up the previous minute's log file and bcp it into MS-SQL.
On Wed, Mar 9, 2011 at 4:22 PM, Shawn Cannon <shawn@shawncannon.com> wrote:
Can someone tell me what the numbers mean for processed and dropped? Am I logging everything to SQL or am I dropping some?
Mar 9 17:12:03 syslog01 syslog-ng[1738]: Log statistics; processed='source(s_gms)=2279331', dropped='dst.sql(d_mssql#0,freetds,rawsql.adbcefg.net ,1785,Syslog)=2213605', stored='dst.sql(d_mssql#0,freetds,rawsql.abcdefg.net,1785,Syslog)=0',
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html