[syslog-ng] Reading Log file

[Shawn Cannon]

Martin, how do I setup the config file to log to a CSV file so I can try the
BCP method?  I assume since BCP is a Windows program then I need to use
SAMBA in Linux to create a share that BCP can read the log files from?

On Thu, Mar 10, 2011 at 10:23 AM, Martin Holste <mcholste at gmail.com> wrote:

> Yep, you're dropping most of your messages to SQL.  Your dropped stats
> should equal 0 if you're not dropping any messages.  The stats are
> listed by source and destination so you can see if a certain source
> can't read the messages fast enough and a certain destination can't
> write fast enough.  I bet if you setup a simple file destination
> you'll see that it doesn't drop any at all while SQL continues to drop
> most messages.  That's because writing to SQL is much more expensive
> than writing to file.  One solution to this is to write to a file in
> CSV format and use MS-SQL bcp to load the files written out in
> batches.  If you name the output files from syslog-ng by the minute of
> the timestamp, then you can have a simple script pick up the previous
> minute's log file and bcp it into MS-SQL.
>
> On Wed, Mar 9, 2011 at 4:22 PM, Shawn Cannon <shawn at shawncannon.com>
> wrote:
> > Can someone tell me what the numbers mean for processed and dropped?  Am
> I
> > logging everything to SQL or am I dropping some?
> >
> > Mar  9 17:12:03 syslog01 syslog-ng[1738]: Log statistics;
> > processed='source(s_gms)=2279331',
> > dropped='dst.sql(d_mssql#0,freetds,rawsql.adbcefg.net
> ,1785,Syslog)=2213605',
> > stored='dst.sql(d_mssql#0,freetds,rawsql.abcdefg.net,1785,Syslog)=0',
> >
> >
> ______________________________________________________________________________
> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Documentation:
> > http://www.balabit.com/support/documentation/?product=syslog-ng
> > FAQ: http://www.campin.net/syslog-ng/faq.html
> >
> >
> >
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20110314/a6ee27f3/attachment.htm