Hi , I am new in syslog-ng ,while installing syslog-ng ,i am facing lots dependency problem ,Can you share with me a document for installation syslog-ng on client side and host side. OS:- centos version :- 6.7 My requirement are mentioned below:- 1. I want to store logs of all clients in one server that will be my syslog-ng server (central server). 2. client servers are apache webservers ,applications servers ,where logs are writing , these are custom directory. 3. I have lot of custom logs on apache webserver and application severs. 4. I want to store these logs for long time on central server about 2 year. Query:- 1.Can we send these logs on syslog-ng real time? 2. Will log format be same as on client ? *Thanks & Regards :-* *VINOD SINGH SAUD* *(M):- 09718663552* *(W):-09997645597* *(E) :- vinod.samant.123@gmail.com <vinod.samant.123@gmail.com>* On Mon, May 7, 2018 at 6:05 PM, Delon Lee Di Lun <lee.delon2005@gmail.com> wrote:

Hi.

Thank you for the clarification.

Yours Sincerely, Delon

On Mon, 7 May 2018 at 16:41 Budai, László <laszlo.budai@balabit.com> wrote:

...

Hi,

sorry for the confusion. The orig.tar.gz is a Debian source which means that it contains a debian directory and ready to build a deb package with standard Debian packaging tools.

If you want to build from source just download the https://github.com/balabit/syslog-ng/releases/download/ syslog-ng-3.15.1/syslog-ng-3.15.1.tar.gz .

regards, Laszlo Budai

On Sun, May 6, 2018 at 6:38 AM, Delon Lee Di Lun <lee.delon2005@gmail.com

...

wrote:

...

Hi,

I am trying to download the source code of syslog-ng 3.15.1 from the link below:

https://github.com/balabit/syslog-ng/releases

May I know what is the difference between the file with "orig" and the one without?

Yours Sincerely, Delon

____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Hi!

...

...

...

...

"vinod" == vinod samant <vinod.samant.123@gmail.com> writes:

vinod> Hi , vinod> I am new in syslog-ng ,while installing syslog-ng ,i am facing lots vinod> dependency problem ,Can you share with me a document for installation vinod> syslog-ng on client side and host side. Are you installing from source, or from a binary package? If the latter, I'd suggest taking a look at our 3rd party binaries page: https://syslog-ng.com/3rd-party-binaries Although, I'm not sure if there are any for CentOS6... The latest I can find is syslog-ng 3.13: https://syslog-ng.com/blog/latest-syslog-ng-available-rhel-6-centos-6/ This suggests that you may need to compile from source if you want to have the latest version. If you need help with that, we can certainly help, but will need some more details, such as the exact problems you are running into. Anyway, installing it on the client and server side should be the same. The difference would be in configuration. vinod> 1. I want to store logs of all clients in one server that will be my vinod> syslog-ng server (central server). vinod> 2. client servers are apache webservers ,applications servers ,where logs vinod> are writing , these are custom directory. vinod> 3. I have lot of custom logs on apache webserver and application severs. vinod> 4. I want to store these logs for long time on central server about 2 year. vinod> Query:- vinod> 1.Can we send these logs on syslog-ng real time? By all intents and purposes, yes. A few things (such as network) may introduce slight delays, but otherwise yes. vinod> 2. Will log format be same as on client ? That depends entirely on how you configure your clients and the central server. It is possible to send logs over as-is, without any parsing at all, and make them appear the same on the server too. -- |8]

Hi!

...

...

...

...

"vinod" == vinod samant <vinod.samant.123@gmail.com> writes:

vinod> 1.first one is using command yum install syslog-ng , and it has been vinod> installed ,if this way is wright then what will be the client and server vinod> side configuration,Suppose The client should be configured to read the logs, the server to accept them and put them where you want 'em. See below for two simple examples. vinod> apache writing custom log on vinod> /usr/local/apache/logs/xyz_access_2018-05-07.log ,Client IP:- 10.10.64.1 vinod> ,server IP:- 10.10.64.100. It looks like you're using files with dates in them, so you'll need a wildcard source. Something along these lines may serve as a starting point: ------------------------- * ------------------------- # client config @version: 3.15 source s_apache { wildcard-file( base-dir("/usr/local/apache/logs"); filename-pattern("*_access_*.log"); flags(no-parse); ); }; destination d_central { network("10.0.0.1" port(1234)); }; log { source(s_apache); destination(d_central); }; ------------------------- * ------------------------- # server config @version: 3.15 source s_network { network(port(1234)); }; destination d_all { file("/var/log/all.log" template("${MSG}\n"); }; log { source(s_network); destination(d_all); }; ------------------------- * ------------------------- If you want to have the same filename on the server side, that becomes a bit less trivial, but still doable. You'll have to transfer the filename too, and extract it on the server side. This should be doable, because the ${FILE_NAME} macro on the client contains the file a log line was read from, you can put this into the message sent to the server, where it can be extracted and used to construct the file the message gets saved to. The following thread might be of use if you want to go down this path: https://lists.balabit.hu/pipermail/syslog-ng/2015-March/021906.html I also recommend reading - or at least browsing the relevant parts of - the syslog-ng administrator's guide. It has a lot of helpful information about the configuration file syntax, options, and whatnot: https://syslog-ng.com/documents/html/syslog-ng-ose-3.14-guides/en/syslog-ng-... vinod> 2. Second i have downloaded tar.gz file form github and trying to install vinod> ,But i am facing lots of dependency problem. vinod> Can you explain difference between both way installation which i am triyng vinod> ? If you install from a binary package, you won't have to compile anything. But you are limited to the version of syslog-ng your distribution ships with (unless you use a third-party repository, which you don't appear to be using). When compiling from source, you'll need plenty of development tools - I'd recommend checking out the docker-based building solution Laszlo Budai mentioned in his reply: https://github.com/balabit/syslog-ng/blob/master/dbld/images/centos6.dockerf... This should make it a *lot* easier to compile from source, as it has all the dependencies already installed. There are some notes on how to use the Dockerfile here: https://github.com/balabit/syslog-ng/tree/master/dbld The main difference between source and binary package is like the difference between a recipe and a finished dish: if you have the recipe (source), the ingredients and tools (dependencies, compiler, etc), then you can cook the dish (binary package). Or you can order (download) the finshed food (binary package), where someone else did the cooking for you. :) -- |8]

...

...

...

...

"Delon" == Delon Lee Di Lun <lee.delon2005@gmail.com> writes:

Delon> I am also have some issue compiling some of the modules. E.g. --enable-json Delon> I have installed the necessary json rpm packages. Do I have to compile the Delon> libraries from source? I can see the lib in /usr/lib64 is there something Delon> im missing, do I have to point ./configure to the /usr/lib64 folder? If so Delon> how do i do it? You need the development files (typically in a package with a -devel postfix). See the `packaging/rhel/syslog-ng.spec` file for a list of dependencies (you may not need all of them, if you don't need all the modules). The "BuildRequires" lines list the packages you need to compile it. Mind you, if you installed syslog-ng from your distribution, and all you want is forward your apache logs from clients to a central server, you may not need to compile from source. -- |8]

Hi , I have used same configuration as you have sent , client IP:- 192.168.122.61 central server IP:- 192.168.122.184 my Client configuration file:- @version:3.13 @include "scl.conf" # syslog-ng configuration file. # # This should behave pretty much like the original syslog on RedHat. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # # Note: it also sources additional configuration files (*.conf) # located in /etc/syslog-ng/conf.d/ options { flush_lines (0); time_reopen (10); log_fifo_size (1000); chain_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { system(); internal(); # udp(ip(0.0.0.0) port(514)); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; # Source additional configuration files (.conf extension only) @include "/etc/syslog-ng/conf.d/*.conf" # vim:ft=syslog-ng:ai:si:ts=4:sw=4:et: # source s_apache { wildcard-file( base-dir("/var/log/httpd/") filename-pattern("*access_*") flags(no-parse) ) destination d_central { network("192.168.122.184" port(udp)) } log { source(s_apache) destination(d_central) }

...

...

...

...

...

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>ERROR<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< If I am starting syslog-ng then ,its showing below error

[root@master syslog-ng]# service syslog-ng start Error parsing source, syntax error, unexpected KW_DESTINATION, expecting ';' in /etc/syslog-ng/syslog-ng.conf at line 83, column 1: destination d_central { ^^^^^^^^^^^ syslog-ng documentation: https://www.balabit.com/support/documentation?product=syslog-ng-ose contact: https://lists.balabit.hu/mailman/listinfo/syslog-ng [root@master syslog-ng]# So,Please help to configure client and central server. *Thanks & Regards :-* *VINOD SINGH SAUD* *(M):- 09718663552* *(W):-09997645597* *(E) :- vinod.samant.123@gmail.com <vinod.samant.123@gmail.com>* On Mon, May 7, 2018 at 7:30 PM, Gergely Nagy <algernon@balabit.com> wrote:

Hi!

...

...

...

...

...

"vinod" == vinod samant <vinod.samant.123@gmail.com> writes:

vinod> 1.first one is using command yum install syslog-ng , and it has been vinod> installed ,if this way is wright then what will be the client and server vinod> side configuration,Suppose

The client should be configured to read the logs, the server to accept them and put them where you want 'em. See below for two simple examples.

vinod> apache writing custom log on vinod> /usr/local/apache/logs/xyz_access_2018-05-07.log ,Client IP:- 10.10.64.1 vinod> ,server IP:- 10.10.64.100.

It looks like you're using files with dates in them, so you'll need a wildcard source. Something along these lines may serve as a starting point:

------------------------- * -------------------------

# client config

@version: 3.15

source s_apache { wildcard-file( base-dir("/usr/local/apache/logs"); filename-pattern("*_access_*.log"); flags(no-parse); ); };

destination d_central { network("10.0.0.1" port(1234)); };

log { source(s_apache); destination(d_central); };

------------------------- * -------------------------

# server config

@version: 3.15

source s_network { network(port(1234)); };

destination d_all { file("/var/log/all.log" template("${MSG}\n"); };

log { source(s_network); destination(d_all); };

------------------------- * -------------------------

If you want to have the same filename on the server side, that becomes a bit less trivial, but still doable. You'll have to transfer the filename too, and extract it on the server side.

This should be doable, because the ${FILE_NAME} macro on the client contains the file a log line was read from, you can put this into the message sent to the server, where it can be extracted and used to construct the file the message gets saved to.

The following thread might be of use if you want to go down this path: https://lists.balabit.hu/pipermail/syslog-ng/2015-March/021906.html

I also recommend reading - or at least browsing the relevant parts of - the syslog-ng administrator's guide. It has a lot of helpful information about the configuration file syntax, options, and whatnot: https://syslog-ng.com/documents/html/syslog-ng-ose- 3.14-guides/en/syslog-ng-ose-guide-admin/html/index.html

vinod> 2. Second i have downloaded tar.gz file form github and trying to install vinod> ,But i am facing lots of dependency problem.

vinod> Can you explain difference between both way installation which i am triyng vinod> ?

If you install from a binary package, you won't have to compile anything. But you are limited to the version of syslog-ng your distribution ships with (unless you use a third-party repository, which you don't appear to be using). When compiling from source, you'll need plenty of development tools - I'd recommend checking out the docker-based building solution Laszlo Budai mentioned in his reply: https://github.com/balabit/syslog-ng/blob/master/dbld/ images/centos6.dockerfile

This should make it a *lot* easier to compile from source, as it has all the dependencies already installed. There are some notes on how to use the Dockerfile here: https://github.com/balabit/syslog-ng/tree/master/dbld

The main difference between source and binary package is like the difference between a recipe and a finished dish: if you have the recipe (source), the ingredients and tools (dependencies, compiler, etc), then you can cook the dish (binary package). Or you can order (download) the finshed food (binary package), where someone else did the cooking for you. :)

-- |8]

Hello, My suggestion would be to check the error message. "Error parsing source, syntax error, unexpected KW_DESTINATION, *expecting ';' *in /etc/syslog-ng/syslog-ng.conf at line 83, column 1:" p.s.: maybe you are missing a few ';' in the configuration file -- Kokan On Tue, May 8, 2018 at 1:38 PM vinod samant <vinod.samant.123@gmail.com> wrote:

Please suggest..

*Thanks & Regards :-*

*VINOD SINGH SAUD*

*(M):- 09718663552*

*(W):-09997645597*

*(E) :- vinod.samant.123@gmail.com <vinod.samant.123@gmail.com>*

On Tue, May 8, 2018 at 3:16 PM, vinod samant <vinod.samant.123@gmail.com> wrote:

...

Hi , I have used same configuration as you have sent , client IP:- 192.168.122.61 central server IP:- 192.168.122.184 my Client configuration file:-

@version:3.13 @include "scl.conf"

# syslog-ng configuration file. # # This should behave pretty much like the original syslog on RedHat. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # # Note: it also sources additional configuration files (*.conf) # located in /etc/syslog-ng/conf.d/

options { flush_lines (0); time_reopen (10); log_fifo_size (1000); chain_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); };

source s_sys { system(); internal(); # udp(ip(0.0.0.0) port(514)); };

destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); };

filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); };

#log { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); };

# Source additional configuration files (.conf extension only) @include "/etc/syslog-ng/conf.d/*.conf"

# vim:ft=syslog-ng:ai:si:ts=4:sw=4:et: #

source s_apache { wildcard-file( base-dir("/var/log/httpd/") filename-pattern("*access_*") flags(no-parse) )

destination d_central { network("192.168.122.184" port(udp)) }

log { source(s_apache) destination(d_central) }

...

...

...

...

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>ERROR<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< If I am starting syslog-ng then ,its showing below error

[root@master syslog-ng]# service syslog-ng start Error parsing source, syntax error, unexpected KW_DESTINATION, expecting ';' in /etc/syslog-ng/syslog-ng.conf at line 83, column 1:

destination d_central { ^^^^^^^^^^^

syslog-ng documentation: https://www.balabit.com/support/documentation?product=syslog-ng-ose contact: https://lists.balabit.hu/mailman/listinfo/syslog-ng [root@master syslog-ng]#

So,Please help to configure client and central server.

*Thanks & Regards :-*

*VINOD SINGH SAUD*

*(M):- 09718663552*

*(W):-09997645597*

*(E) :- vinod.samant.123@gmail.com <vinod.samant.123@gmail.com>*

On Mon, May 7, 2018 at 7:30 PM, Gergely Nagy <algernon@balabit.com> wrote:

...

Hi!

...

...

...

>> "vinod" == vinod samant <vinod.samant.123@gmail.com> writes:

vinod> 1.first one is using command yum install syslog-ng , and it has been vinod> installed ,if this way is wright then what will be the client and server vinod> side configuration,Suppose

The client should be configured to read the logs, the server to accept them and put them where you want 'em. See below for two simple examples.

vinod> apache writing custom log on vinod> /usr/local/apache/logs/xyz_access_2018-05-07.log ,Client IP:- 10.10.64.1 vinod> ,server IP:- 10.10.64.100.

It looks like you're using files with dates in them, so you'll need a wildcard source. Something along these lines may serve as a starting point:

------------------------- * -------------------------

# client config

@version: 3.15

source s_apache { wildcard-file( base-dir("/usr/local/apache/logs"); filename-pattern("*_access_*.log"); flags(no-parse); ); };

destination d_central { network("10.0.0.1" port(1234)); };

log { source(s_apache); destination(d_central); };

------------------------- * -------------------------

# server config

@version: 3.15

source s_network { network(port(1234)); };

destination d_all { file("/var/log/all.log" template("${MSG}\n"); };

log { source(s_network); destination(d_all); };

------------------------- * -------------------------

If you want to have the same filename on the server side, that becomes a bit less trivial, but still doable. You'll have to transfer the filename too, and extract it on the server side.

This should be doable, because the ${FILE_NAME} macro on the client contains the file a log line was read from, you can put this into the message sent to the server, where it can be extracted and used to construct the file the message gets saved to.

The following thread might be of use if you want to go down this path: https://lists.balabit.hu/pipermail/syslog-ng/2015-March/021906.html

I also recommend reading - or at least browsing the relevant parts of - the syslog-ng administrator's guide. It has a lot of helpful information about the configuration file syntax, options, and whatnot:

https://syslog-ng.com/documents/html/syslog-ng-ose-3.14-guides/en/syslog-ng-...

vinod> 2. Second i have downloaded tar.gz file form github and trying to install vinod> ,But i am facing lots of dependency problem.

vinod> Can you explain difference between both way installation which i am triyng vinod> ?

If you install from a binary package, you won't have to compile anything. But you are limited to the version of syslog-ng your distribution ships with (unless you use a third-party repository, which you don't appear to be using). When compiling from source, you'll need plenty of development tools - I'd recommend checking out the docker-based building solution Laszlo Budai mentioned in his reply:

https://github.com/balabit/syslog-ng/blob/master/dbld/images/centos6.dockerf...

This should make it a *lot* easier to compile from source, as it has all the dependencies already installed. There are some notes on how to use the Dockerfile here: https://github.com/balabit/syslog-ng/tree/master/dbld

The main difference between source and binary package is like the difference between a recipe and a finished dish: if you have the recipe (source), the ingredients and tools (dependencies, compiler, etc), then you can cook the dish (binary package). Or you can order (download) the finshed food (binary package), where someone else did the cooking for you. :)

-- |8]

______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq