[syslog-ng] Local sources seem not to be working
Alexandre Santos
ASantos at infinera.com
Tue Jan 10 23:37:29 UTC 2023
>From the last tests with info that was provided regarding journald polling, it seems that the system is hitting this problem:
https://github.com/syslog-ng/syslog-ng/issues/2836
Still, the last time I catch the problem, I have the following:
root at machine:/ # cat /proc/sys/kernel/random/boot_id
e473e508-6bce-412a-b7b8-ff44b5dbea9b
root at machine:/ # journalctl --list-boots
-1 8f5414009b064ef68186e2b91a9f251d Tue 2023-01-10 02:41:58 CST-Tue 2023-01-10 02:42:32 CST
0 a74228741b0240ba93621f15bd58b4c4 Tue 2023-01-10 02:57:57 CST-Wed 2023-01-11 19:39:20 CST
The boot id does not match any of the journal boots. So restart of syslog-ng does not clear the problem.
Do you know how can this happen?
Regarding your questions:
1) systemd 241 (241)
2) syslog-ng 3.36.1
3) Yes
4) Yes, it is quite loaded.
Thanks, Alex
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu> On Behalf Of Balazs Scheidler
Sent: 1 de novembro de 2022 06:49
To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu>
Subject: Re: [syslog-ng] Local sources seem not to be working
Ok, I've now read the relevant code to see how the journald polling operates and now have a general understanding.
Couple of questions and notes.
1) which version of of systemd do you have? (Debian buster currently has 247.3-6~bpo10+1), is that right?
2) syslog-ng 3.36.1, right?
3) you are using flow-control + disk buffer
4) your system seems to be lightly loaded, you seem to have had ~30 messages/sec in the journal between stalling and recovering via SIGHUP (based on the "processed" counts and the last timestamp)
syslog-ng uses the libsystemd library to poll for changes, which in turn uses inotify to watch the directories and files that make up the journal. You should see the inotify fd in the lsof output. I have mine like this:
syslog-ng 3233 root 11r a_inode 0,14 0 10439 inotify
You should also have a couple of "eventpoll" fd's open, in lsof output this should look like this:
syslog-ng 3233 root 3u a_inode 0,14 0 10439 [eventpoll]
syslog-ng 3233 root 13u a_inode 0,14 0 10439 [eventpoll]
As you can see, I have two of these eventpoll fds, #3 is the main one, usually that comes first.
If you do an strace on the syslog-ng process (command was "strace -s 256 -f -p <syslogng-pid> -o aaa"), you can see the epoll being manipulated:
# we are grepping for all epoll operations with fd #3
$ grep 'epoll.*(3' aaa
3233 07:12:55.542368 epoll_wait(3, [{EPOLLIN, {u32=1319656488, u64=94426175659048}}], 10, 921) = 1 <0.462260>
3233 07:12:56.005163 epoll_ctl(3, EPOLL_CTL_DEL, 11, 0x7ffecc73736c) = 0 <0.000007>
3233 07:12:56.005265 epoll_wait(3, <unfinished ...>
625546 07:12:56.005555 epoll_ctl(3, EPOLL_CTL_MOD, 4, {EPOLLIN|EPOLLONESHOT, {u32=1319484192, u64=94426175486752}}) = 0 <0.000008>
3233 07:12:56.005778 epoll_ctl(3, EPOLL_CTL_ADD, 11, {EPOLLIN, {u32=1319656488, u64=94426175659048}} <unfinished ...>
3233 07:12:56.005864 epoll_wait(3, <unfinished ...>
625546 07:12:56.006049 epoll_ctl(3, EPOLL_CTL_MOD, 4, {EPOLLIN|EPOLLONESHOT, {u32=1319484192, u64=94426175486752}}) = 0 <0.000006>
3233 07:12:56.006114 epoll_wait(3, [{EPOLLIN, {u32=1319656488, u64=94426175659048}}], 10, -1) = 1 <0.250790>
3233 07:12:56.257450 epoll_ctl(3, EPOLL_CTL_DEL, 11, 0x7ffecc73736c) = 0 <0.000009>
3233 07:12:56.257518 epoll_wait(3, <unfinished ...>
625546 07:12:56.257828 epoll_ctl(3, EPOLL_CTL_MOD, 4, {EPOLLIN|EPOLLONESHOT, {u32=1319484192, u64=94426175486752}}) = 0 <0.000021>
3233 07:12:56.258144 epoll_ctl(3, EPOLL_CTL_ADD, 11, {EPOLLIN, {u32=1319656488, u64=94426175659048}} <unfinished ...>
3233 07:12:56.258250 epoll_wait(3, <unfinished ...>
625546 07:12:56.258411 epoll_ctl(3, EPOLL_CTL_MOD, 4, {EPOLLIN|EPOLLONESHOT, {u32=1319484192, u64=94426175486752}}) = 0 <0.000013>
3233 07:12:56.258496 epoll_wait(3, [{EPOLLIN, {u32=1319484552, u64=94426175487112}}], 10, -1) = 1 <0.203908>
3233 07:12:56.462711 epoll_wait(3, [{EPOLLIN, {u32=1319656488, u64=94426175659048}}], 10, 1000) = 1 <0.123872>
3233 07:12:56.587406 epoll_ctl(3, EPOLL_CTL_DEL, 11, 0x7ffecc73736c) = 0 <0.000015>
3233 07:12:56.587542 epoll_wait(3, <unfinished ...>
>From those events, you can see that syslog-ng deletes and adds fd #11 (which is my inotify descriptor) and fd #4 (which is an event). If I zoom in to fd 11 in addition to epoll:
$ egrep '(\(11)|(epoll.*\(3)' aaa
# first event and its reaction:
3233 07:12:55.542368 epoll_wait(3, [{EPOLLIN, {u32=1319656488, u64=94426175659048}}], 10, 921) = 1 <0.462260>
3233 07:12:56.004800 read(11, "\3\0\0\0\2\0\0\0\0\0\0\0\20\0\0\0system.journal\0\0", 272) = 32 <0.000019>
3233 07:12:56.005090 read(11, 0x7ffecc7372a0, 272) = -1 EAGAIN (Resource temporarily unavailable) <0.000006>
3233 07:12:56.005163 epoll_ctl(3, EPOLL_CTL_DEL, 11, 0x7ffecc73736c) = 0 <0.000007>
3233 07:12:56.005265 epoll_wait(3, <unfinished ...>
625546 07:12:56.005555 epoll_ctl(3, EPOLL_CTL_MOD, 4, {EPOLLIN|EPOLLONESHOT, {u32=1319484192, u64=94426175486752}}) = 0 <0.000008>
3233 07:12:56.005595 fcntl(11, F_GETFD) = 0x1 (flags FD_CLOEXEC) <0.000007>
3233 07:12:56.005618 fcntl(11, F_GETFL) = 0x800 (flags O_RDONLY|O_NONBLOCK) <0.000006>
3233 07:12:56.005659 setsockopt(11, SOL_SOCKET, SO_OOBINLINE, [1], 4) = -1 ENOTSOCK (Socket operation on non-socket) <0.000007>
3233 07:12:56.005778 epoll_ctl(3, EPOLL_CTL_ADD, 11, {EPOLLIN, {u32=1319656488, u64=94426175659048}} <unfinished ...>
3233 07:12:56.005864 epoll_wait(3, <unfinished ...>
# second event and its reaction:
625546 07:12:56.006049 epoll_ctl(3, EPOLL_CTL_MOD, 4, {EPOLLIN|EPOLLONESHOT, {u32=1319484192, u64=94426175486752}}) = 0 <0.000006>
3233 07:12:56.006114 epoll_wait(3, [{EPOLLIN, {u32=1319656488, u64=94426175659048}}], 10, -1) = 1 <0.250790>
3233 07:12:56.256994 read(11, "\3\0\0\0\2\0\0\0\0\0\0\0\20\0\0\0system.journal\0\0", 272) = 32 <0.000026>
3233 07:12:56.257416 read(11, 0x7ffecc7372a0, 272) = -1 EAGAIN (Resource temporarily unavailable) <0.000008>
3233 07:12:56.257450 epoll_ctl(3, EPOLL_CTL_DEL, 11, 0x7ffecc73736c) = 0 <0.000009>
3233 07:12:56.257518 epoll_wait(3, <unfinished ...>
625546 07:12:56.257828 epoll_ctl(3, EPOLL_CTL_MOD, 4, {EPOLLIN|EPOLLONESHOT, {u32=1319484192, u64=94426175486752}}) = 0 <0.000021>
3233 07:12:56.257940 fcntl(11, F_GETFD) = 0x1 (flags FD_CLOEXEC) <0.000011>
3233 07:12:56.257978 fcntl(11, F_GETFL) = 0x800 (flags O_RDONLY|O_NONBLOCK) <0.000009>
3233 07:12:56.258017 setsockopt(11, SOL_SOCKET, SO_OOBINLINE, [1], 4) = -1 ENOTSOCK (Socket operation on non-socket) <0.000011>
3233 07:12:56.258144 epoll_ctl(3, EPOLL_CTL_ADD, 11, {EPOLLIN, {u32=1319656488, u64=94426175659048}} <unfinished ...>
3233 07:12:56.258250 epoll_wait(3, <unfinished ...>
please notice that syslog-ng reads fd #11 right after epoll indicates there's an event available. The read() has a payload which can be decoded according to struct inotify_event.
In both cases, after the thread "625546" starts executing, we use the journald APIs to fetch the message. It's not v ery visible from the strace as sdjournal seems to access the files via mmap or something, but you should definitely see the disk buffer being written as messages get delivered, since you are using a reliable disk buffer (which you may not need btw, but that's a separate discussion).
The reasons we might stall could be:
* the file is not modified at all (this could be ruled out)
* we don't poll the inotify fd (you can confirm this via strace as above)
* we do poll the fd, but we don't read the events from it (again strace helps here)
* we do poll fd fd and read the events but somehow we don't respond to it properly (this is probably a bug)
* our worker wakes up in response to the event but then it doesn't consume messages from the journal (this is probably a bug)
If we are not polling the inotify fd, that means that the systemd-journal() source might be suspended. This can happen for example because of flow control:
* a source is suspended if some of its flow-controlled destinations are unable to consume the messages it produces. in your case, you have a disk buffer and that consumes all messages up to its capacity. I didn't see disk buffers that were full (the queued counters are all zero), you were also saying that those same files are being written, messages delivered normally.
* if for some reason we are stuck in the "suspended" state and the wakeup mechanism does not work (this is probably a bug)
With all this said, can you please confirm that we are polling the inotify fd from our main thread and that you can see the inotify event being consumed? Or you don't see that all.
Of course while syslog-ng is working, this should operate normally. The trick is to look at it when it is stalled.
Thanks and again sorry for this to take so long.
On Tue, Nov 1, 2022 at 6:36 AM Balazs Scheidler <bazsi77 at gmail.com<mailto:bazsi77 at gmail.com>> wrote:
sorry, no I didn't.
I just looked at your configs, If I see correctly then in the error situation the local logs are stalled, counters are not incremented. The journald "processed" counter stays at 85558, while immediately after reload it goes to 271377, which seems to indicate that for some reason we are not getting updates on the journals.
I am reading the code to understand how we poll the journal and I hope I'll have some input how to debug this.
Bazsi
On Mon, Oct 31, 2022 at 5:15 PM Alexandre Santos <ASantos at infinera.com<mailto:ASantos at infinera.com>> wrote:
Hi,
Did you have a chance to analyze this?
Thanks & Regards,
Alex
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu<mailto:syslog-ng-bounces at lists.balabit.hu>> On Behalf Of Alexandre Santos
Sent: 16 de setembro de 2022 14:38
To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu<mailto:syslog-ng at lists.balabit.hu>>
Subject: Re: [syslog-ng] Local sources seem not to be working
Hi Balazs,
Do you have any other idea of how to debug this?
>From the last time I was able to reproduce the problem, I found the following:
New developments, on this.
After another test this time with no remote destinations configured, the issue happened again:
In attachment I am sending:
- The stats in error condition: 42116.inerror.stats.txt, where 'src.journald;s_src#0;journal;a;stamp;1651142246'
- The stats in error condition after some time: 42116.inerror.15m.stats.txt where 'src.journald;s_src#0;journal;a;stamp;1651142246'
- The stats after recovering the system with reload: 42116.after.stats.txt
- The syslog-ng configuration: 42116.no.remote.dest.syslog-ng.conf
It seems the 'src.journald;s_src#0;journal;a;stamp;1651142246'did not changed. Does this means that last read timestamp from journal did not changed?
Logs from the UDP source are still being written to the /var/log/... files.
Cheers,
Alex
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu<mailto:syslog-ng-bounces at lists.balabit.hu>> On Behalf Of Balazs Scheidler
Sent: 1 de julho de 2022 21:02
To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu<mailto:syslog-ng at lists.balabit.hu>>
Subject: Re: [syslog-ng] Local sources seem not to be working
This means that we are indeed polling the journal ourselves and not rely on journald to forward the logs to us.
If the error occurs, try to look at the syslog-ng statistic counters (syslog-ng-ctl stats) to see if your output queue is full. This could cause back pressure and reading to stop.
Also, enabling debug/trace messages of syslog-ng could be helpful. You can also do that via syslog-ng-ctl, no restart is needed. The trace output should be excluded from your normal log processing pipeline, as it can be quite overwhelming in volume.
I don't really have any other ideas at the moment. Still on vacation. :)
On Thu, Jun 30, 2022, 19:43 Alexandre Santos <ASantos at infinera.com<mailto:ASantos at infinera.com>> wrote:
Hi Balazs,
Thanks for the feedback!
It is definitely using journald, as you can see below.
When you say "Try to remove the syslog-ng persist file and check if reading the journal restarts.", this is to do when the system in the error condition, right?
Regards,
Alex
source s_src {
#Start Block source generator system
channel {
source {
systemd-journal();
}; # source
channel {
channel {
parser {
#Start Block parser generator app-parser
channel {
junction {
channel { filter { tags('.app.doesnotexist'); }; flags(final); }; };
}
#End Block parser generator app-parser
;
};
flags(final);
};
channel { flags(final); };
};
}; # channel
#End Block source generator system
;
internal();
syslog(ip(19.88.4.17) transport("udp") port(514) keep-alive(no));
};
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu<mailto:syslog-ng-bounces at lists.balabit.hu>> On Behalf Of Balazs Scheidler
Sent: 26 de junho de 2022 06:37
To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu<mailto:syslog-ng at lists.balabit.hu>>
Subject: Re: [syslog-ng] Local sources seem not to be working
Hi,
I haven't seen anything like this. We are reading the journal files using libsystemd.
Try to remove the syslog-ng persist file and check if reading the journal restarts.
Also there's can be two ways of local messages getting to syslog-ng,
1) /dev/log forwarding
2) reading the journal files
The first one is actively done by journald. Which one syslog-ng uses is automatically detected by our system() source.
To see which one syslog-ng is trying to use, try to run it with --preprocess-into=some-file and check how system() source is expanded.
I am unable to check the source code at the moment, so this is all from the top-of-my-head, but I hope this already helps to troubleshoot the issue.
On Fri, Jun 24, 2022, 18:21 Alexandre Santos <ASantos at infinera.com<mailto:ASantos at infinera.com>> wrote:
Hi
Any news regarding this issue?
Making a recap of the findings:
* Using a Debian 10 buster with first release with 3.36.1;
* After some time "system()" source logs are not getting written to the destinations;
* The log messages from other sources, internal() and syslog(...) continue to work fine, being written to the destinations;
* One the things I noticed is that the socket to the journal seems to vanish during the error situation:
It seems that somehow syslog-ng in unable to read from linux journal.
Have you ever experienced this problem?
Do know what can be wrong with the system?
root at machine:~# lsof /run/log/journal/98101a328524447d88917bea845a8966/system*
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
systemd-j 1723 root mem REG 0,19 8388608 31745 /run/log/journal/98101a328524447d88917bea845a8966/system.journal
systemd-j 1723 root mem REG 0,19 8388608 26165 /run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal<mailto:/run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal>
systemd-j 1723 root 16u REG 0,19 8388608 26165 /run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal<mailto:/run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal>
systemd-j 1723 root 24u REG 0,19 8388608 31745 /run/log/journal/98101a328524447d88917bea845a8966/system.journal
syslog-ng 3201 root mem REG 0,19 8388608 26165 /run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal<mailto:/run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal>
syslog-ng 3201 root mem REG 0,19 8388608 31745 /run/log/journal/98101a328524447d88917bea845a8966/system.journal
syslog-ng 3201 root 14r REG 0,19 8388608 31745 /run/log/journal/98101a328524447d88917bea845a8966/system.journal
syslog-ng 3201 root 15r REG 0,19 8388608 26165 /run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal<mailto:/run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal>
journalct 6861 root mem REG 0,19 8388608 26165 /run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal<mailto:/run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal>
journalct 6861 root mem REG 0,19 8388608 31745 /run/log/journal/98101a328524447d88917bea845a8966/system.journal
journalct 6861 root 5r REG 0,19 8388608 31745 /run/log/journal/98101a328524447d88917bea845a8966/system.journal
journalct 6861 root 6r REG 0,19 8388608 26165 /run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal<mailto:/run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal>
root@ machine:~# lsof /run/log/journal/98101a328524447d88917bea845a8966/system*
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
systemd-j 1723 root mem REG 0,19 8388608 31745 /run/log/journal/98101a328524447d88917bea845a8966/system.journal
systemd-j 1723 root mem REG 0,19 8388608 26165 /run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal<mailto:/run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal>
systemd-j 1723 root 16u REG 0,19 8388608 26165 /run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal<mailto:/run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal>
systemd-j 1723 root 24u REG 0,19 8388608 31745 /run/log/journal/98101a328524447d88917bea845a8966/system.journal
journalct 6861 root mem REG 0,19 8388608 26165 /run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal<mailto:/run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal>
journalct 6861 root mem REG 0,19 8388608 31745 /run/log/journal/98101a328524447d88917bea845a8966/system.journal
journalct 6861 root 5r REG 0,19 8388608 31745 /run/log/journal/98101a328524447d88917bea845a8966/system.journal
journalct 6861 root 6r REG 0,19 8388608 26165 /run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal<mailto:/run/log/journal/98101a328524447d88917bea845a8966/system at 3721b31246e54dc0baab1ac0f68c3f43-0000000000000001-000581d7e3fe20ba.journal>
Thanks in advance,
Alex
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu<mailto:syslog-ng-bounces at lists.balabit.hu>> On Behalf Of Alexandre Santos
Sent: 19 de maio de 2022 09:25
To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu<mailto:syslog-ng at lists.balabit.hu>>
Subject: Re: [syslog-ng] Local sources seem not to be working
Hi Szilard,
There is no filter:
source syslog_ng_src {
internal();
};
destination d_localfile_syslog_ng {
program("/opt/machine/local/bin/write_with_rotation.sh /var/log/syslog-ng-internal.log 10 10"
flags(syslog-protocol)
suppress(5)
disk-buffer(
mem-buf-size(2097152)
disk-buf-size(4194304)
reliable(yes)
dir("/tmp")
)
);
};
log {
source(syslog_ng_src);
destination(d_localfile_syslog_ng);
flags(flow-control);
};
Thanks and Regards,
Alex
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu<mailto:syslog-ng-bounces at lists.balabit.hu>> On Behalf Of Szilard Parrag (sparrag)
Sent: 19 de maio de 2022 08:59
To: syslog-ng at lists.balabit.hu<mailto:syslog-ng at lists.balabit.hu>
Subject: Re: [syslog-ng] Local sources seem not to be working
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
Hi Alex,
We've checked it too and syslog-ng does not release the file descriptor of journald even with flow-control enabled.
Also, your internal logs seem rather terse, maybe there is a filter which filters out the important parts. Could you please check it?
Szilard
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=05%7C01%7Casantos%40infinera.com%7C14e11b4e57564fdd257808dabbd54209%7C285643de5f5b4b03a1530ae2dc8aaf77%7C1%7C0%7C638028821866307428%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Hx4vnC4ovnJHzP%2FOvBhwD7JynP2dgYKQg6e%2BeK7xEaY%3D&reserved=0>
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=05%7C01%7Casantos%40infinera.com%7C14e11b4e57564fdd257808dabbd54209%7C285643de5f5b4b03a1530ae2dc8aaf77%7C1%7C0%7C638028821866307428%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=jiIrysNzV3agMWDR%2FwQn%2FZ7WvyXYmUODE%2BcsF8MYzCM%3D&reserved=0>
FAQ: http://www.balabit.com/wiki/syslog-ng-faq<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=05%7C01%7Casantos%40infinera.com%7C14e11b4e57564fdd257808dabbd54209%7C285643de5f5b4b03a1530ae2dc8aaf77%7C1%7C0%7C638028821866307428%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=epn0Ap2nNzhsEahBAh2JNwLerEIaHKab6gweveZ8l1Q%3D&reserved=0>
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=05%7C01%7Casantos%40infinera.com%7C14e11b4e57564fdd257808dabbd54209%7C285643de5f5b4b03a1530ae2dc8aaf77%7C1%7C0%7C638028821866307428%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Hx4vnC4ovnJHzP%2FOvBhwD7JynP2dgYKQg6e%2BeK7xEaY%3D&reserved=0>
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=05%7C01%7Casantos%40infinera.com%7C14e11b4e57564fdd257808dabbd54209%7C285643de5f5b4b03a1530ae2dc8aaf77%7C1%7C0%7C638028821866307428%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=jiIrysNzV3agMWDR%2FwQn%2FZ7WvyXYmUODE%2BcsF8MYzCM%3D&reserved=0>
FAQ: http://www.balabit.com/wiki/syslog-ng-faq<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=05%7C01%7Casantos%40infinera.com%7C14e11b4e57564fdd257808dabbd54209%7C285643de5f5b4b03a1530ae2dc8aaf77%7C1%7C0%7C638028821866307428%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=epn0Ap2nNzhsEahBAh2JNwLerEIaHKab6gweveZ8l1Q%3D&reserved=0>
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=05%7C01%7Casantos%40infinera.com%7C14e11b4e57564fdd257808dabbd54209%7C285643de5f5b4b03a1530ae2dc8aaf77%7C1%7C0%7C638028821866307428%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Hx4vnC4ovnJHzP%2FOvBhwD7JynP2dgYKQg6e%2BeK7xEaY%3D&reserved=0>
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=05%7C01%7Casantos%40infinera.com%7C14e11b4e57564fdd257808dabbd54209%7C285643de5f5b4b03a1530ae2dc8aaf77%7C1%7C0%7C638028821866307428%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=jiIrysNzV3agMWDR%2FwQn%2FZ7WvyXYmUODE%2BcsF8MYzCM%3D&reserved=0>
FAQ: http://www.balabit.com/wiki/syslog-ng-faq<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=05%7C01%7Casantos%40infinera.com%7C14e11b4e57564fdd257808dabbd54209%7C285643de5f5b4b03a1530ae2dc8aaf77%7C1%7C0%7C638028821866307428%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=epn0Ap2nNzhsEahBAh2JNwLerEIaHKab6gweveZ8l1Q%3D&reserved=0>
--
Bazsi
--
Bazsi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20230110/dce1c460/attachment-0001.htm>
More information about the syslog-ng
mailing list