From attila.szakacs at axoflow.com Mon Jan 15 15:07:45 2024 From: attila.szakacs at axoflow.com (=?UTF-8?Q?Attila_Szak=C3=A1cs?=) Date: Mon, 15 Jan 2024 16:07:45 +0100 Subject: [syslog-ng] Please Help! Syslog-ng Not Putting Logs in Destination In-Reply-To: References: Message-ID: Hello Sumanta! Your config looks good. The log about the statistics show that there are no incoming messages on 514 UDP and nothing is written to the files defined in the d_splunk destination. I think you could try to narrow down the scope of the problem with the following ideas. Try to send a message locally to 514 with: echo "foo bar" | nc -u -w0 localhost 514 If it does not work, I would suggest to change the receiving port of the network() source to something larger, like port(12345), and trying again with the following, just to see if the problem only occurs for the 514 port: echo "foo bar" | nc -u -w0 localhost 12345 You should see these kind of logs: [2024-01-15T15:58:46.037255] Incoming log entry; input='foo bar\x0a', msg='0x7f9bb0003020', rcptid='297' ... [2024-01-15T15:58:46.037655] Initializing destination file writer; template='......', filename='......', symlink_as='(null)' ... [2024-01-15T15:58:46.037872] Outgoing message; message='bar' My hunch is that this probably has something to do with SELinux, but unfortunately my knowledge of it is very limited. Regards, Attila -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.szakacs at axoflow.com Mon Jan 15 15:12:19 2024 From: attila.szakacs at axoflow.com (=?UTF-8?Q?Attila_Szak=C3=A1cs?=) Date: Mon, 15 Jan 2024 16:12:19 +0100 Subject: [syslog-ng] Please Help! Syslog-ng Not Putting Logs in Destination In-Reply-To: References: Message-ID: There is a selinux policy setup script in the syslog-ng repository, you might want to look at that to find some ideas. I have seen that some ports are getting enabled by the script with the semanage command, maybe this is what you are missing: https://github.com/syslog-ng/syslog-ng/blob/master/contrib/selinux/syslog_ng.sh#L228 Cheers, Attila On Mon, Jan 15, 2024 at 4:07?PM Attila Szak?cs wrote: > Hello Sumanta! > > Your config looks good. > The log about the statistics show that there are no incoming messages on > 514 UDP and nothing is written to the files defined in the d_splunk > destination. > > I think you could try to narrow down the scope of the problem with the > following ideas. > > Try to send a message locally to 514 with: > echo "foo bar" | nc -u -w0 localhost 514 > > If it does not work, I would suggest to change the receiving port of the > network() source to something larger, like port(12345), and trying again > with the following, just to see if the problem only occurs for the 514 port: > echo "foo bar" | nc -u -w0 localhost 12345 > > You should see these kind of logs: > [2024-01-15T15:58:46.037255] Incoming log entry; input='foo bar\x0a', > msg='0x7f9bb0003020', rcptid='297' > ... > [2024-01-15T15:58:46.037655] Initializing destination file writer; > template='......', filename='......', symlink_as='(null)' > ... > [2024-01-15T15:58:46.037872] Outgoing message; message='bar' > > My hunch is that this probably has something to do with SELinux, but > unfortunately my knowledge of it is very limited. > > Regards, > Attila > -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.szakacs at axoflow.com Mon Jan 15 15:06:00 2024 From: attila.szakacs at axoflow.com (=?UTF-8?Q?Attila_Szak=C3=A1cs?=) Date: Mon, 15 Jan 2024 16:06:00 +0100 Subject: [syslog-ng] Syslog-ng Not Working properly In-Reply-To: References: Message-ID: Hello Sumanta! Your config looks good. The log about the statistics show that there are no incoming messages on 514 UDP and nothing is written to the files defined in the d_splunk destination. I think you could try to narrow down the scope of the problem with the following ideas. Try to send a message locally to 514 with: echo "foo bar" | nc -u -w0 localhost 514 If it does not work, I would suggest to change the receiving port of the network() source to something larger, like port(12345), and trying again with the following, just to see if the problem only occurs for the 514 port: echo "foo bar" | nc -u -w0 localhost 12345 You should see these kind of logs: [2024-01-15T15:58:46.037255] Incoming log entry; input='foo bar\x0a', msg='0x7f9bb0003020', rcptid='297' ... [2024-01-15T15:58:46.037655] Initializing destination file writer; template='......', filename='......', symlink_as='(null)' ... [2024-01-15T15:58:46.037872] Outgoing message; message='bar' My hunch is that this probably has something to do with SELinux, but unfortunately my knowledge of it is very limited. Regards, Attila On Sun, Jan 14, 2024 at 9:50?PM Sumanta Banerjee wrote: > *Hi Team,* > > > > > > *I am trying to configure syslog-ng in one our linux instance to get > NGIPS/FMC data via udp connection on its default port (514). I have > configured syslog-ng.conf under /etc/syslog-ng and then we have set SE > Linux as Permissive. I am using RHEL 8.7 and syslog version 4.0. Apparently > all looked good to me however while checking in the destination path that > is mentioned I don?t see any directory or logfile from for the said udp > connection got created. * > > *Below is our observation and steps that we executed, can any of you > please help me telling where I went wrong or if I am missing something, > there is another testing in pipeline that is stalled for this ? * > > > > 1. Define source, destination and log_file in syslog-ng.conf (file > attached). > 2. Run the below SELinux command ? > > # ausearch -c 'syslog-ng' --raw | audit2allow -M my-syslogng > # semodule -X 300 -i my-syslogng.pp > > 1. Restart syslog-ng service ? > > # systemctl restart syslog-ng.service (no error message received) > > 1. Check if the service is running ? > > > > > > 1. Check if syslog-ng is listening to udp port 514 ? > > > > 1. Checked and we have incoming data stream from source using the > below command ? > > *tcpdump -i any -c10 -nn -A port 514* > > > > 1. I have went through the syslog-ng troubleshooting steps mentioned > in the link ? > > > https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.37/administration-guide/105#TOPIC-1829320 > > > > > > *syslog-ng -Fdev command output is also attached.* > > > > 1. While running the following command got the below output - *# watch > '/usr/sbin/syslog-ng-ctl stats | grep "^center"'* > > > > 1. *# journaltctl command output (first 500 lines) attached* > > > > 1. Current SE Linux status : > > > > 1. Our syslog-ng is logging to /var/log/messages and we are getting > this message in /var/log/messages ? > > > > > > Thanks & Regards, > > Sumanta Banerjee > > Splunk Admin | CISO | Aviva Group > > Tel: +91-8420892593 > > 24x7x365: +44 1603 208 582 > > sumanta.banerjee at aviva.com > > GlobalCyberSecurityEngineeringTeam at aviva.com > > www.aviva.com > > Wipro Technologies - SJP2, Bangalore, India > > > > > > Aviva: Internal > > Aviva plc, registered Office: St. Helen's, 1 Undershaft, London EC3P 3DQ. > Registered in England No. 02468686. www.aviva.com > > This message and any attachments may be confidential or legally > privileged. If you are not the intended recipient, please telephone or > e-mail the sender and delete this message and any attachments from your > system. Also, if you are not the intended recipient you must not copy this > message or attachments or disclose the contents to any other person. Any > views or opinions expressed are solely those of the author and do not > necessarily represent those of Aviva. > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 3316 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.png Type: image/png Size: 26844 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image003.png Type: image/png Size: 7580 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image004.png Type: image/png Size: 3061 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image005.png Type: image/png Size: 8575 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image006.png Type: image/png Size: 10145 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image007.png Type: image/png Size: 101145 bytes Desc: not available URL: From Peter.Czanik at oneidentity.com Thu Jan 18 11:07:13 2024 From: Peter.Czanik at oneidentity.com (Peter Czanik (pczanik)) Date: Thu, 18 Jan 2024 11:07:13 -0000 Subject: [syslog-ng] The syslog-ng Insider 2024-01: HTTP; Cloudflare; systemd-journal; Humio / Logscale; Message-ID: Dear syslog-ng users, This is the 116th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Why use a http()-based destination in syslog-ng? ------------------------------------------------ Logging is not just syslog anymore. Still, many syslog-ng users stick to using one of the syslog protocols for log transport and flat files for log storage. While most SIEMs and log analytics tools can receive syslog messages or read them using their own agents, in most cases, you can use the http() destination of syslog-ng as well to send logs to them. You gain extreme performance and an architecture that is easier to maintain. Some of the drivers in syslog-ng built on top of the http() destination include elasticsearch-http() destination for sending logs to Elasticsearch / OpenSearch. Many other services utilizing the Elasticsearch Bulk API exist, and there are also others using Sumo Logic, Splunk, and so on. Of course, there are some destinations where performance is not a major concern (or can even be a drawback, in fact). For example,various instant messaging services used for alerting (like Telegram or Slack) belong in this category. You can also read the API docs and write a new destination based on http() yourself. https://www.syslog-ng.com/community/b/blog/posts/why-use-a-http--based-destination-in-syslog-ng An overview of Cloudflare's logging pipeline -------------------------------------------- One of the roles of Cloudflare's Observability Platform team is managing the operation, improvement, and maintenance of our internal logging pipelines. These pipelines are used to ship debugging logs from every service across Cloudflare?s infrastructure into a centralised location, allowing our engineers to operate and debug their services in near real time. In this post, we?re going to go over what that looks like, how we achieve high availability, and how we meet our Service Level Objectives (SLOs) while shipping close to a million log lines per second. https://blog.cloudflare.com/an-overview-of-cloudflares-logging-pipeline/ Working with multiple systemd-journal namespaces in syslog-ng ------------------------------------------------------------- Initial support for systemd-journal namespaces is available in syslog-ng 3.29. However, only version 4.4.0 allows you to work with multiple namespaces in your syslog-ng configuration. So, what changed in the latest version of syslog-ng? Previously, you could only configure a single systemd-journal() source in syslog-ng. By default, it collected logs from all namespaces, but you could configure it to collect log messages from a single one exclusively. This means that logs from other namespaces could not be collected by syslog-ng. Version 4.4.0 allows you to use multiple systemd-journal() source drivers in the configuration, as long as each source uses a unique namespace. https://www.syslog-ng.com/community/b/blog/posts/working-with-multiple-systemd-journal-namespaces-in-syslog-ng Logging to Humio / Logscale simplified in syslog-ng --------------------------------------------------- Logging into Humio (which was recently re-branded to Falcon LogScale) was available for years, using their Elasticsearch compatible API. However, according to Humio developers, it is slightly slower than other APIs for log ingestion. Axoflow contributed a Logscale destination to syslog-ng, which uses Logscale?s native API. I did not measure if there is really a performance difference, however it is definitely easier to configure it. WEBINARS * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik From wlinn at bofa.com Fri Jan 26 13:17:48 2024 From: wlinn at bofa.com (Linn, William) Date: Fri, 26 Jan 2024 13:17:48 -0000 Subject: [syslog-ng] running RHEL 9 Message-ID: <4b9e7f9a872b4fd69b2d8a19f3a62951@bofa.com> Has anyone seen any issues with syslog-ng running on RHEL 9? Any pointers, caveats? Thanks William ---------------------------------------------------------------------- This message, and any attachment(s), is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/electronic-disclaimer. If you are not the intended recipient, please delete this message. For more information about how Bank of America protects your privacy, including specific rights that may apply, please visit the following pages: https://business.bofa.com/en-us/content/global-privacy-notices.html (which includes global privacy notices) and https://www.bankofamerica.com/security-center/privacy-overview/ (which includes US State specific privacy notices such as the http://www.bankofamerica.com/ccpa-notice). -------------- next part -------------- An HTML attachment was scrubbed... URL: