[syslog-ng] Kafka-c issues
Balazs Scheidler
bazsi77 at gmail.com
Fri Jul 15 14:39:18 UTC 2022
Sorry, mixed up 18.04 with xenial. Still, it does not make a difference,
Our dbld/build.manifest file has the packaging options for various
platforms and it says:
# on ubuntu, we start using Python3 at focal onwards.
# libcriterion-dev is available starting with 21.04
ubuntu-xenial python2,nocriterion,nokafka,nojava,nomqtt
ubuntu-bionic python2,nocriterion,nokafka,nomqtt
ubuntu-focal python3,nocriterion,nomqtt
ubuntu-impish python3
ubuntu-jammy python3
So it'd compile kafka-c with focal onwards. If you have an updated
librdkafka compiled locally, you could remove "nokafka" from the bionic
line, before compiling.
All this assumes that you compile syslog-ng via dbld. If you don't, make
sure that your configure invocation does not disable kafka (e.g. with
--disable-kafka option. by default kafka is auto-detected, so if the dev
package is installed, we would compile with it). All this should be
confirmed by checking the end of the configure output which displays the
modules that are enabled.
On Fri, Jul 15, 2022 at 4:19 PM Sanjay Patel <skp at smarthive.io> wrote:
> I have bionic; not sure if that makes a difference?
>
>
>
> *From:* syslog-ng <syslog-ng-bounces at lists.balabit.hu> *On Behalf Of *Balazs
> Scheidler
> *Sent:* Friday, July 15, 2022 3:08 AM
> *To:* Syslog-ng users' and developers' mailing list <
> syslog-ng at lists.balabit.hu>
> *Subject:* Re: [syslog-ng] Kafka-c issues
>
>
>
> You probably don't have the kafka-c driver installed, this can happen as
> ubuntu 18.04 does not have the required librdkafka libraries. If I remember
> correctly, we don't even supply Xenial packages, but I might be wrong
> there. How did you acquire 3.37.1 for Ubuntu Xenial?
>
>
>
> Our ubuntu-focal packages do have kafka-c, if you check out the build
> logs, it's included:
> https://github.com/syslog-ng/syslog-ng/runs/6776527081?check_suite_focus=true
>
>
>
> ```
>
> Kafka support (module) : yes
>
> ```
>
>
>
> On Fri, Jul 15, 2022 at 8:29 AM Sanjay Patel <skp at smarthive.io> wrote:
>
> Attila,
>
> Thanks for the response. Here is the response with your requested change.
>
>
>
> Error parsing config, syntax error, unexpected LL_IDENTIFIER, expecting
> '}' in block destination kafka() at
> /usr/share/syslog-ng/include/scl/kafka/kafka.conf:24:4:5-4:12:
>
> 1
>
> 2 #Start Block block destination kafka() at
> /usr/share/syslog-ng/include/scl/kafka/kafka.conf:24
>
> 3
>
> 4-----> kafka-c(bootstrap_servers("127.0.0.1:9092")
> topic("{incoming}") );
>
> 4-----> ^^^^^^^
>
> 5
>
> 6 #End Block block destination kafka() at
> /usr/share/syslog-ng/include/scl/kafka/kafka.conf:24
>
> 7
>
>
>
> Included from /etc/syslog-ng/syslog-ng.conf:6:3-9:4:
>
> 1 @version: 3.37
>
> 2 @include "scl.conf"
>
> 3 @define kafka-implementation kafka-c
>
> 4
>
> 5 destination d_kafka {
>
> 6-----> kafka(
>
> 6-----> ^^^^^^^
>
> 7-----> bootstrap-servers("127.0.0.1:9092")
>
> 8-----> topic("{incoming}")
>
> 9-----> );
>
> 10 };
>
>
>
>
>
>
>
> *From:* syslog-ng <syslog-ng-bounces at lists.balabit.hu> *On Behalf Of *Attila
> Szakacs (aszakacs)
> *Sent:* Friday, July 15, 2022 2:25 AM
> *To:* Syslog-ng users' and developers' mailing list <
> syslog-ng at lists.balabit.hu>
> *Subject:* Re: [syslog-ng] Kafka-c issues
>
>
>
> Hi Sanjay,
>
>
>
> Can you try switching the define and the include, like:
>
>
>
> @version: 3.37
>
> @include "scl.conf"
>
> @define kafka-implementation kafka-c
>
>
>
> ...
>
>
>
> Cheers,
>
> Attila
>
>
> ------------------------------
>
> *From:* syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of
> Sanjay Patel <skp at smarthive.io>
> *Sent:* Friday, July 15, 2022 8:07 AM
> *To:* Syslog-ng users' and developers' mailing list <
> syslog-ng at lists.balabit.hu>
> *Subject:* Re: [syslog-ng] Kafka-c issues
>
>
>
> *CAUTION:* This email originated from outside of the organization. Do not
> follow guidance, click links, or open attachments unless you recognize the
> sender and know the content is safe.
>
>
>
> Here is what happens when I include it. It tires to use kafka-java
>
>
>
> Error parsing config, syntax error, unexpected LL_IDENTIFIER, expecting
> '}' in block destination kafka() at
> /usr/share/syslog-ng/include/scl/kafka/kafka.conf:24:4:5-4:15:
>
> 1
>
> 2 #Start Block block destination kafka() at
> /usr/share/syslog-ng/include/scl/kafka/kafka.conf:24
>
> 3
>
> 4-----> kafka-java(bootstrap_servers("127.0.0.1:9092")
> topic("{incoming}") );
>
> 4-----> ^^^^^^^^^^
>
> 5
>
> 6 #End Block block destination kafka() at
> /usr/share/syslog-ng/include/scl/kafka/kafka.conf:24
>
> 7
>
>
>
> Included from /etc/syslog-ng/syslog-ng.conf:6:3-9:4:
>
> 1 @version: 3.37
>
> 2 @define kafka-implementation kafka-c
>
> 3 @include "scl.conf"
>
> 4
>
> 5 destination d_kafka {
>
> 6-----> kafka(
>
> 6-----> ^^^^^^^
>
> 7-----> bootstrap-servers("127.0.0.1:9092")
>
> 8-----> topic("{incoming}")
>
> 9-----> );
>
> 10 };
>
>
>
> *From:* syslog-ng <syslog-ng-bounces at lists.balabit.hu> *On Behalf Of *Peter
> Kokai (pkokai)
> *Sent:* Friday, July 15, 2022 2:05 AM
> *To:* Syslog-ng users' and developers' mailing list <
> syslog-ng at lists.balabit.hu>
> *Subject:* Re: [syslog-ng] Kafka-c issues
>
>
>
> hello,
>
>
>
> the @include is required for kafka, but you commented out with #
>
>
>
> --
>
> kokan
>
>
>
> Get Outlook for Android
> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FAAb9ysg&data=05%7C01%7Cattila.szakacs%40balabit.com%7Caaff869a335a430ee9c708da66285200%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637934620622603978%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2Ly%2Fj%2F9kJUQjcC%2Bi3yAjXC6qyevj50Wv5Vx1pUQ9ESU%3D&reserved=0>
> ------------------------------
>
> *From:* syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of
> Sanjay Patel <skp at smarthive.io>
> *Sent:* Friday, July 15, 2022 7:30:21 AM
> *To:* syslog-ng at lists.balabit.hu <syslog-ng at lists.balabit.hu>
> *Subject:* [syslog-ng] Kafka-c issues
>
>
>
> *CAUTION:* This email originated from outside of the organization. Do not
> follow guidance, click links, or open attachments unless you recognize the
> sender and know the content is safe.
>
>
>
> I am trying to setup Syslog-NG t send output to Kakka. I am running ubuntu
> 18.04, Syslog-NG version 3.37.
>
>
>
> This is my syslog-ng.conf ( I have #scl.conf because if I don’t it tries
> to install kafka-java)
>
>
>
>
>
> @version: 3.37
>
> @define kafka-implementation kafka-c
>
> #@include "scl.conf"
>
>
>
> destination d_kafka {
>
> kafka(
>
> bootstrap-servers("127.0.0.1:9092")
>
> topic("{incoming}")
>
> );
>
> };
>
>
>
>
>
> This is the error when I try and run sudo syslog-ng
>
>
>
> Error parsing config, syntax error, unexpected LL_IDENTIFIER, expecting
> '}' in /etc/syslog-ng/syslog-ng.conf:6:3-6:8:
>
> 1 @version: 3.37
>
> 2 @define kafka-implementation kafka-c
>
> 3 #@include "scl.conf"
>
> 4
>
> 5 destination d_kafka {
>
> 6-----> kafka(
>
> 6-----> ^^^^^
>
> 7 bootstrap-servers("127.0.0.1:9092")
>
> 8 topic("{incoming}")
>
> 9 );
>
> 10 };
>
>
>
> Any help will be appreciated.
>
>
>
> SKP
>
>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
> --
>
> Bazsi
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
--
Bazsi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20220715/95869821/attachment-0001.htm>
More information about the syslog-ng
mailing list