[syslog-ng] Encoding provlem when writing to postgres

[Balazs Scheidler]

On Wed, 2008-11-05 at 08:48 -0600, Joseph Pietras wrote:
> Is 3.0 open source available?  I could not find it on the download page,
> Thanks
> Joseph

Yes, as nightly snapshots, with the version number "2.9", but the only
reason it has 2.9 as version number as it was not finalized yet. But you
are right, this adds to the confusion, I change it to 3.0 and the first
official release will be 3.0.1.

The nightly snapshots can be found at: 

http://www.balabit.com/downloads/files/syslog-ng/sources/3.0/src-snapshot

> 
> 
> -----Original Message-----
> From: syslog-ng-bounces at lists.balabit.hu [mailto:syslog-ng-bounces at lists.balabit.hu] On Behalf Of ml at bortal.de
> Sent: Wednesday, November 05, 2008 8:36 AM
> To: Syslog-ng users' and developers' mailing list
> Cc: Balazs Scheidler
> Subject: Re: [syslog-ng] Encoding provlem when writing to postgres
> 
> Hey Balazs,
> 
> i think ai am almost there yet ;)
> (see below...)
> 
> Balazs Scheidler wrote:
> >> Am 15.10.2008 um  Uhr haben Sie geschrieben:
> >>
> >>> On Wed, 2008-10-15 at 09:53 +0200, ml at bortal.de wrote:
> >>>
> >>>> Hello List,
> >>>>
> >>>> How can i fix this? I am running postgresql-8.1 with Debian 4.0.
> >>>>
> >>> Probably the syslog client sends non-utf8 characters in its message.
> >>> syslog-ng 2.0 and 2.1 do not really care about the message contents.
> >>>
> >>> 3.0 has support for various encodings and utf8 validation like this:
> >>>
> >>> This assumes that the input is latin1:
> >>>
> >>> source s_net { udp(encoding("iso-8859-1")); };
> >>>
> >>> And this one enforces valid utf8 messages
> >>>
> >>> source s_net { udp(flags(validate-utf8)); };
> >>>
> >>> I was planning to add another flag, but this is not yet implemented:
> >>>
> >>> source s_net { udp(flags(force-utf8)); };
> >>>
> >>> Which would enforce valid utf8 sequences by changing the input.
> >>>
> >>> As an alternative don't use utf8 as the character encoding of your
> >>> database, use latin1, that'll permit any kind of data in the database.
> >>>
> >>>
> syslog-ng -F -v -f /etc/syslog-ng.conf -e
> Configuration file has no version number, assuming syslog-ng 2.1 format.
> Please add @version: maj.min to the beginning of the file;
> WARNING: global: the default value of chain_hostnames is changing to
> 'no' in version 3.0, please update your configuration accordingly;
> Your configuration file uses an obsoleted keyword, please update your
> configuration; keyword='sync', change='flush_lines'
> Your configuration file uses an obsoleted keyword, please update your
> configuration; keyword='stats', change='stats_freq'
> WARNING: input: sources do not remove new-line characters from messages
> by default in version 3.0, please add 'no-multi-line' flag to your
> configuration if you want to retain this functionality;
> WARNING: file source: default value of follow_freq in file sources is
> changing in 3.0 to '1' for all files except /proc/kmsg;
> syntax error in /etc/syslog-ng.conf at line 14.
> 
> 
> Line 14 and 15 look like this:
> -------------------------------------------------------------
> source s_net { udp(encoding("iso-8859-1")); };
> source s_net { udp(flags(validate-utf8)); };
> 
> I also tried to add: @version: 2.9 to the config file, but this did not
> work.
> 
> 
> Any idea?
> 
> Thanks,
> Mario
> 
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
> 
> 
> 
> 
> This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error please notify the originator of the message. This footer also confirms that this
> e-mail message has been scanned for the presence of computer viruses.
> 
>  Any views expressed in this message are those of the individual sender, except where the sender specifies and with authority, states them to be the views of Iowa Student Loan.
> 
> 
> 
-- 
Bazsi