<div dir="ltr">Dear syslog-ng users,<br><br>We are pleased to announce that the 4.2.0 version of syslog-ng has been<br>released, and it is now available on GitHub: <a href="https://github.com/syslog-ng/syslog-ng/releases/tag/syslog-ng-4.2.0">https://github.com/syslog-ng/syslog-ng/releases/tag/syslog-ng-4.2.0</a><div><br></div><div>4.2.0<br>=====<br><br>Read Axoflow's [blog post](<a href="https://axoflow.com/axosyslog-release-4-2/">https://axoflow.com/axosyslog-release-4-2/</a>) for more details.<br><br>## Highlights<br><br>#### Sending messages to Splunk HEC<br>The `splunk-hec-event()` destination feeds Splunk via the [HEC events API](<a href="https://docs.splunk.com/Documentation/Splunk/9.0.4/RESTREF/RESTinput#services.2Fcollector.2Fevent.2F1.0">https://docs.splunk.com/Documentation/Splunk/9.0.4/RESTREF/RESTinput#services.2Fcollector.2Fevent.2F1.0</a>).<br><br>Minimal config:<br>```<br>destination d_splunk_hec_event {<br>  splunk-hec-event(<br>    url("<a href="https://localhost:8088">https://localhost:8088</a>")<br>    token("70b6ae71-76b3-4c38-9597-0c5b37ad9630")<br>  );<br>};<br>```<br><br>Additional options include:<br>  * `event()`<br>  * `index()`<br>  * `source()`<br>  * `sourcetype()`<br>  * `host()`<br>  * `time()`<br>  * `default-index()`<br>  * `default-source()`<br>  * `default-sourcetype()`<br>  * `fields()`<br>  * `extra-headers()`<br>  * `extra-queries()`<br>  * `content-type()`<br><br><br>The `splunk-hec-raw()` destination feeds Splunk via the [HEC raw API](<a href="https://docs.splunk.com/Documentation/Splunk/9.0.4/RESTREF/RESTinput#services.2Fcollector.2Fraw.2F1.0">https://docs.splunk.com/Documentation/Splunk/9.0.4/RESTREF/RESTinput#services.2Fcollector.2Fraw.2F1.0</a>).<br><br>Minimal config:<br>```<br>destination d_splunk_hec_raw {<br>  splunk-hec-raw(<br>    url("<a href="https://localhost:8088">https://localhost:8088</a>")<br>    token("70b6ae71-76b3-4c38-9597-0c5b37ad9630")<br>    channel("05ed4617-f186-4ccd-b4e7-08847094c8fd")<br>  );<br>};<br>```<br><br>([#4462](<a href="https://github.com/syslog-ng/syslog-ng/pull/4462">https://github.com/syslog-ng/syslog-ng/pull/4462</a>))<br><br>#### Smart multi-line for recognizing backtraces<br>`multi-line-mode(smart)`:<br>With this multi-line mode, the inherently multi-line data backtrace format is<br>recognized even if they span multiple lines in the input and are converted<br>to a single log message for easier analysis.  Backtraces for the following<br>programming languages are recognized : Python, Java, JavaScript, PHP, Go,<br>Ruby and Dart.<br><br>The regular expressions to recognize these programming languages are<br>specified by an external file called<br>`/usr/share/syslog-ng/smart-multi-line.fsm` (installation path depends on<br>configure arguments), in a format that is described in that file.<br><br>`group-lines()` parser: this new parser correlates multi-line messages<br>received as separate, but subsequent lines into a single log message.<br>Received messages are first collected into streams related messages (using<br>key()), then collected into correlation contexts up to timeout() seconds.<br>The identification of multi-line messages are then performed on these<br>message contexts within the time period.<br><br>```<br>  group-lines(key("$FILE_NAME")<br>              multi-line-mode("smart")<br>        template("$MESSAGE")<br>        timeout(10)<br>        line-separator("\n")<br>  );<br>```<br><br>([#4225](<a href="https://github.com/syslog-ng/syslog-ng/pull/4225">https://github.com/syslog-ng/syslog-ng/pull/4225</a>))<br><br>#### HYPR Audit Trail source<br>`hypr-audit-trail()` & `hypr-app-audit-trail()` source drivers are now<br>available to monitor the audit trails for [HYPR](<a href="https://www.hypr.com">https://www.hypr.com</a>) applications.<br><br>See the README.md file in the driver's directory to see usage information.<br><br>([#4175](<a href="https://github.com/syslog-ng/syslog-ng/pull/4175">https://github.com/syslog-ng/syslog-ng/pull/4175</a>))<br><br>#### `ebpf()` plugin and reuseport packet randomizer<br>A new ebpf() plugin was added as a framework to leverage the kernel's eBPF<br>infrastructure to improve performance and scalability of syslog-ng.<br><br>Example:<br><br>```<br>source s_udp {<br>        udp(so-reuseport(yes) port(2000) persist-name("udp1")<br>                ebpf(reuseport(sockets(4)))<br>        );<br>        udp(so-reuseport(yes) port(2000) persist-name("udp2"));<br>        udp(so-reuseport(yes) port(2000) persist-name("udp3"));<br>        udp(so-reuseport(yes) port(2000) persist-name("udp4"));<br>};<br>```<br><br>NOTE: The `ebpf()` plugin is considered advanced usage so its compilation is<br>disabled by default.  Please don't use it unless all other avenues of<br>configuration solutions are already tried.  You will need a special<br>toolchain and a recent kernel version to compile and run eBPF programs.<br><br>([#4365](<a href="https://github.com/syslog-ng/syslog-ng/pull/4365">https://github.com/syslog-ng/syslog-ng/pull/4365</a>))<br><br><br>## Features<br><br>  * `network` source: During a TLS handshake, syslog-ng now automatically sets the<br>    `certificate_authorities` field of the certificate request based on the `ca-file()`<br>    and `ca-dir()` options. The `pkcs12-file()` option already had this feature.<br>    ([#4412](<a href="https://github.com/syslog-ng/syslog-ng/pull/4412">https://github.com/syslog-ng/syslog-ng/pull/4412</a>))<br><br>  * `metrics-probe()`: Added `level()` option to set the stats level of the generated metrics.<br>    ([#4453](<a href="https://github.com/syslog-ng/syslog-ng/pull/4453">https://github.com/syslog-ng/syslog-ng/pull/4453</a>))<br><br>  * `metrics-probe()`: Added `increment()` option.<br><br>    Users can now set a template, which resolves to a number that modifies<br>    the increment of the counter. If not set, the increment is 1.<br>    ([#4447](<a href="https://github.com/syslog-ng/syslog-ng/pull/4447">https://github.com/syslog-ng/syslog-ng/pull/4447</a>))<br><br>  * `python`: Added support for typed custom options.<br><br>    This applies for `python` source, `python-fetcher` source, `python` destination,<br>    `python` parser and `python-http-header` inner destination.<br><br>    Example config:<br>    ```<br>    python(<br>      class("TestClass")<br>      options(<br>        "string_option" => "example_string"<br>        "bool_option" => True  # supported values are: True, False, yes, no<br>        "integer_option" => 123456789<br>        "double_option" => 123.456789<br>        "string_list_option" => ["string1", "string2", "string3"]<br>        "template_option" => LogTemplate("${example_template}")<br>      )<br>    );<br>    ```<br><br>    **Breaking change! Previously values were converted to strings if possible, now they are passed<br>    to the python class with their real type. Make sure to follow up these changes<br>    in your python code!**<br>    ([#4354](<a href="https://github.com/syslog-ng/syslog-ng/pull/4354">https://github.com/syslog-ng/syslog-ng/pull/4354</a>))<br><br>  * `mongodb` destination: Added support for list, JSON and null types.<br>    ([#4437](<a href="https://github.com/syslog-ng/syslog-ng/pull/4437">https://github.com/syslog-ng/syslog-ng/pull/4437</a>))<br><br>  * `add-contextual-data()`: significantly reduce memory usage for large CSV<br>    files.<br>    ([#4444](<a href="https://github.com/syslog-ng/syslog-ng/pull/4444">https://github.com/syslog-ng/syslog-ng/pull/4444</a>))<br><br>  * `python()`: new LogMessage methods for querying as string and with default values<br><br>    - `get(key[, default])`<br>      Return the value for `key` if `key` exists, else `default`. If `default` is<br>      not given, it defaults to `None`, so that this method never raises a<br>      `KeyError`.<br><br>    - `get_as_str(key, default=None, encoding='utf-8', errors='strict', repr='internal')`:<br>      Return the string value for `key` if `key` exists, else `default`.<br>      If `default` is not given, it defaults to `None`, so that this method never<br>      raises a `KeyError`.<br><br>      The string value is decoded using the codec registered for `encoding`.<br>      `errors` may be given to set the desired error handling scheme.<br><br>      Note that currently `repr='internal'` is the only available representation.<br>      We may implement another more Pythonic representation in the future, so please<br>      specify the `repr` argument explicitly if you want to avoid future<br>      representation changes in your code.<br>    ([#4410](<a href="https://github.com/syslog-ng/syslog-ng/pull/4410">https://github.com/syslog-ng/syslog-ng/pull/4410</a>))<br><br>  * `kubernetes()` source: Added support for json-file logging driver format.<br>    ([#4419](<a href="https://github.com/syslog-ng/syslog-ng/pull/4419">https://github.com/syslog-ng/syslog-ng/pull/4419</a>))<br><br>  * The new `$RAWMSG_SIZE` hard macro can be used to query the original size of the<br>    incoming message in bytes.<br><br>    This information may not be available for all source drivers.<br>    ([#4440](<a href="https://github.com/syslog-ng/syslog-ng/pull/4440">https://github.com/syslog-ng/syslog-ng/pull/4440</a>))<br><br>  * syslog-ng configuration identifier<br><br>    A new syslog-ng configuration keyword has been added, which allows specifying a config identifier. For example:<br>    ```<br>    @config-id: cfg-20230404-13-g02b0850fc<br>    ```<br><br>    This keyword can be used for config identification in managed environments, where syslog-ng instances and their<br>    configuration are deployed/generated automatically.<br><br>    `syslog-ng-ctl config --id` can be used to query the active configuration ID and the SHA256 hash of the full<br>    "preprocessed" syslog-ng configuration. For example:<br><br>    ```<br>    $ syslog-ng-ctl config --id<br>    cfg-20230404-13-g02b0850fc (08ddecfa52a3443b29d5d5aa3e5114e48dd465e195598062da9f5fc5a45d8a83)<br>    ```<br>    ([#4420](<a href="https://github.com/syslog-ng/syslog-ng/pull/4420">https://github.com/syslog-ng/syslog-ng/pull/4420</a>))<br><br>  * `syslog-ng`: add `--config-id` command line option<br><br>    Similarly to `--syntax-only`, this command line option parses the configuration<br>    and then prints its ID before exiting.<br><br>    It can be used to query the ID of the current configuration persisted on<br>    disk.<br>    ([#4435](<a href="https://github.com/syslog-ng/syslog-ng/pull/4435">https://github.com/syslog-ng/syslog-ng/pull/4435</a>))<br><br>  * Health metrics and `syslog-ng-ctl healthcheck`<br><br>    A new `syslog-ng-ctl` command has been introduced, which can be used to query a healthcheck status from syslog-ng.<br>    Currently, only 2 basic health values are reported.<br><br>    `syslog-ng-ctl healthcheck --timeout <seconds>` can be specified to use it as a boolean healthy/unhealthy check.<br><br>    Health checks are also published as periodically updated metrics.<br>    The frequency of these checks can be configured with the `stats(healthcheck-freq())` option.<br>    The default is 5 minutes.<br>    ([#4362](<a href="https://github.com/syslog-ng/syslog-ng/pull/4362">https://github.com/syslog-ng/syslog-ng/pull/4362</a>))<br><br>  * `$(format-json)` and template functions which support value-pairs<br>    expressions: new key transformations upper() and lower() have been added to<br>    translate the caps of keys while formatting the output template. For<br>    example:<br><br>        template("$(format-json test.* --upper)\n")<br><br>    Would convert all keys to uppercase. Only supports US ASCII.<br>    ([#4452](<a href="https://github.com/syslog-ng/syslog-ng/pull/4452">https://github.com/syslog-ng/syslog-ng/pull/4452</a>))<br><br>  * `python()`, `python-fetcher()` sources: Added a mapping for the `flags()` option.<br><br>    The state of the `flags()` option is mapped to the `self.flags` variable, which is<br>    a `Dict[str, bool]`, for example:<br>    ```python<br>    {<br>        'parse': True,<br>        'check-hostname': False,<br>        'syslog-protocol': True,<br>        'assume-utf8': False,<br>        'validate-utf8': False,<br>        'sanitize-utf8': False,<br>        'multi-line': True,<br>        'store-legacy-msghdr': True,<br>        'store-raw-message': False,<br>        'expect-hostname': True,<br>        'guess-timezone': False,<br>        'header': True,<br>        'rfc3164-fallback': True,<br>    }<br>    ```<br>    ([#4455](<a href="https://github.com/syslog-ng/syslog-ng/pull/4455">https://github.com/syslog-ng/syslog-ng/pull/4455</a>))<br><br><br>### Metrics<br>  * `network()`, `syslog()`: TCP connection metrics<br><br>    ```<br>    syslogng_socket_connections{id="tcp_src#0",driver_instance="afsocket_sd.(stream,AF_INET(0.0.0.0:5555))",direction="input"} 3<br>    syslogng_socket_max_connections{id="tcp_src#0",driver_instance="afsocket_sd.(stream,AF_INET(0.0.0.0:5555))",direction="input"} 10<br>    syslogng_socket_rejected_connections_total{id="tcp_src#0",driver_instance="afsocket_sd.(stream,AF_INET(0.0.0.0:5555))",direction="input"} 96<br>    ```<br><br>    `internal()`: `internal_events_queue_capacity` metric<br><br>    `syslog-ng-ctl healthcheck`: new healthcheck value `syslogng_internal_events_queue_usage_ratio`<br>    ([#4411](<a href="https://github.com/syslog-ng/syslog-ng/pull/4411">https://github.com/syslog-ng/syslog-ng/pull/4411</a>))<br><br>  * `metrics`: new network (TCP, UDP) metrics are available on stats level 1<br><br>    ```<br>    # syslog-ng-ctl stats prometheus<br><br>    syslogng_socket_receive_buffer_used_bytes{id="#anon-source0#3",direction="input",driver_instance="afsocket_sd.udp4"} 0<br>    syslogng_socket_receive_buffer_max_bytes{id="#anon-source0#3",direction="input",driver_instance="afsocket_sd.udp4"} 268435456<br>    syslogng_socket_receive_dropped_packets_total{id="#anon-source0#3",direction="input",driver_instance="afsocket_sd.udp4"} 619173<br><br>    syslogng_socket_connections{id="#anon-source0#0",direction="input",driver_instance="afsocket_sd.(stream,AF_INET(0.0.0.0:2000))"} 1<br>    ```<br>    ([#4374](<a href="https://github.com/syslog-ng/syslog-ng/pull/4374">https://github.com/syslog-ng/syslog-ng/pull/4374</a>))<br><br>  * New configuration-related metrics:<br><br>    ```<br>    syslogng_last_config_reload_timestamp_seconds 1681309903<br>    syslogng_last_successful_config_reload_timestamp_seconds 1681309758<br>    syslogng_last_config_file_modification_timestamp_seconds 1681309877<br>    ```<br>    ([#4420](<a href="https://github.com/syslog-ng/syslog-ng/pull/4420">https://github.com/syslog-ng/syslog-ng/pull/4420</a>))<br><br>  * destination: Introduced queue metrics.<br><br>      * The corresponding driver is identified with the "id" and "driver_instance" labels.<br>      * Available counters are "memory_usage_bytes" and "events".<br>      * Memory queue metrics are available with "syslogng_memory_queue_" prefix,<br>        `disk-buffer` metrics are available with "syslogng_disk_queue_" prefix.<br>      * `disk-buffer` metrics have an additional "path" label, pointing to the location of the disk-buffer file<br>        and a "reliable" label, which can be either "true" or "false".<br>      * Threaded destinations, like `http`, `python`, etc have an additional "worker" label.<br><br>    Example metrics<br>    ```<br>    syslogng_disk_queue_events{driver_instance="http,<a href="http://localhost:1239">http://localhost:1239</a>",id="d_http_disk_buffer#0",path="/var/syslog-ng/syslog-ng-00000.rqf",reliable="true",worker="0"} 80<br>    syslogng_disk_queue_events{driver_instance="http,<a href="http://localhost:1239">http://localhost:1239</a>",id="d_http_disk_buffer#0",path="/var/syslog-ng/syslog-ng-00001.rqf",reliable="true",worker="1"} 7<br>    syslogng_disk_queue_events{driver_instance="http,<a href="http://localhost:1239">http://localhost:1239</a>",id="d_http_disk_buffer#0",path="/var/syslog-ng/syslog-ng-00002.rqf",reliable="true",worker="2"} 7<br>    syslogng_disk_queue_events{driver_instance="http,<a href="http://localhost:1239">http://localhost:1239</a>",id="d_http_disk_buffer#0",path="/var/syslog-ng/syslog-ng-00003.rqf",reliable="true",worker="3"} 7<br>    syslogng_disk_queue_events{driver_instance="tcp,localhost:1235",id="d_network_disk_buffer#0",path="/var/syslog-ng/syslog-ng-00000.qf",reliable="false"} 101<br>    syslogng_disk_queue_memory_usage_bytes{driver_instance="http,<a href="http://localhost:1239">http://localhost:1239</a>",id="d_http_disk_buffer#0",path="/var/syslog-ng/syslog-ng-00000.rqf",reliable="true",worker="0"} 3136<br>    syslogng_disk_queue_memory_usage_bytes{driver_instance="http,<a href="http://localhost:1239">http://localhost:1239</a>",id="d_http_disk_buffer#0",path="/var/syslog-ng/syslog-ng-00001.rqf",reliable="true",worker="1"} 2776<br>    syslogng_disk_queue_memory_usage_bytes{driver_instance="http,<a href="http://localhost:1239">http://localhost:1239</a>",id="d_http_disk_buffer#0",path="/var/syslog-ng/syslog-ng-00002.rqf",reliable="true",worker="2"} 2760<br>    syslogng_disk_queue_memory_usage_bytes{driver_instance="http,<a href="http://localhost:1239">http://localhost:1239</a>",id="d_http_disk_buffer#0",path="/var/syslog-ng/syslog-ng-00003.rqf",reliable="true",worker="3"} 2776<br>    syslogng_disk_queue_memory_usage_bytes{driver_instance="tcp,localhost:1235",id="d_network_disk_buffer#0",path="/var/syslog-ng/syslog-ng-00000.qf",reliable="false"} 39888<br>    syslogng_memory_queue_events{driver_instance="http,<a href="http://localhost:1236">http://localhost:1236</a>",id="d_http#0",worker="0"} 15<br>    syslogng_memory_queue_events{driver_instance="http,<a href="http://localhost:1236">http://localhost:1236</a>",id="d_http#0",worker="1"} 14<br>    syslogng_memory_queue_events{driver_instance="tcp,localhost:1234",id="d_network#0"} 29<br>    syslogng_memory_queue_memory_usage_bytes{driver_instance="http,<a href="http://localhost:1236">http://localhost:1236</a>",id="d_http#0",worker="0"} 5896<br>    syslogng_memory_queue_memory_usage_bytes{driver_instance="http,<a href="http://localhost:1236">http://localhost:1236</a>",id="d_http#0",worker="1"} 5552<br>    syslogng_memory_queue_memory_usage_bytes{driver_instance="tcp,localhost:1234",id="d_network#0"} 11448<br>    ```<br>    ([#4392](<a href="https://github.com/syslog-ng/syslog-ng/pull/4392">https://github.com/syslog-ng/syslog-ng/pull/4392</a>))<br><br>  * `network()`, `syslog()`, `file()`, `http()`: new byte-based metrics for incoming/outgoing events<br><br>    These metrics show the serialized message sizes (protocol-specific header/framing/etc. length is not included).<br><br>    ```<br>    syslogng_input_event_bytes_total{id="s_network#0",driver_instance="tcp,127.0.0.1"} 1925529600<br>    syslogng_output_event_bytes_total{id="d_network#0",driver_instance="tcp,<a href="http://127.0.0.1:5555">127.0.0.1:5555</a>"} 565215232<br>    syslogng_output_event_bytes_total{id="d_http#0",driver_instance="http,<a href="http://127.0.0.1:8080/">http://127.0.0.1:8080/</a>"} 1024<br>    ```<br>    ([#4440](<a href="https://github.com/syslog-ng/syslog-ng/pull/4440">https://github.com/syslog-ng/syslog-ng/pull/4440</a>))<br><br>  * `disk-buffer`: Added metrics for monitoring the available space in disk-buffer `dir()`s.<br><br>    Metrics are available from `stats(level(1))`.<br><br>    By default, the metrics are generated every 5 minutes, but it can be changed in the global options:<br>    ```<br>    options {<br>      disk-buffer(<br>        stats(<br>          freq(10)<br>        )<br>      );<br>    };<br>    ```<br>    Setting `freq(0)` disabled this feature.<br><br>    Example metrics:<br>    ```<br>    syslogng_disk_queue_dir_available_bytes{dir="/var/syslog-ng"} 870109413376<br>    ```<br>    ([#4399](<a href="https://github.com/syslog-ng/syslog-ng/pull/4399">https://github.com/syslog-ng/syslog-ng/pull/4399</a>))<br><br>  * `disk-buffer`: Added metrics for abandoned disk-buffer files.<br><br>    Availability is the same as the `disk_queue_dir_available_bytes` metric.<br><br>    Example metrics:<br>    ```<br>    syslogng_disk_queue_capacity_bytes{abandoned="true",path="/var/syslog-ng/syslog-ng-00000.rqf",reliable="true"} 104853504<br>    syslogng_disk_queue_disk_allocated_bytes{abandoned="true",path="/var/syslog-ng/syslog-ng-00000.rqf",reliable="true"} 273408<br>    syslogng_disk_queue_disk_usage_bytes{abandoned="true",path="/var/syslog-ng/syslog-ng-00000.rqf",reliable="true"} 269312<br>    syslogng_disk_queue_events{abandoned="true",path="/var/syslog-ng/syslog-ng-00000.rqf",reliable="true"} 860<br>    ```<br>    ([#4402](<a href="https://github.com/syslog-ng/syslog-ng/pull/4402">https://github.com/syslog-ng/syslog-ng/pull/4402</a>))<br><br>  * `disk-buffer`: Added capacity, disk_allocated and disk_usage metrics.<br><br>      * "capacity_bytes": The theoretical maximal useful size of the disk-buffer.<br>                          This is always smaller, than `disk-buf-size()`, as there is some reserved<br>                          space for metadata. The actual full disk-buffer file can be larger than this,<br>                          as syslog-ng allows to write over this limit once, at the end of the file.<br><br>      * "disk_allocated_bytes": The current size of the disk-buffer file on the disk. Please note that<br>                                the disk-buffer file size does not strictly correlate with the number<br>                                of messages, as it is a ring buffer implementation, and also syslog-ng<br>                                optimizes the truncation of the file for performance reasons.<br><br>      * "disk_usage_bytes": The serialized size of the queued messages in the disk-buffer file. This counter<br>                            is useful for calculating the disk usage percentage (disk_usage_bytes / capacity_bytes)<br>                            or the remaining available space (capacity_bytes - disk_usage_bytes).<br><br>    Example metrics:<br>    ```<br>    syslogng_disk_queue_capacity_bytes{driver_id="d_network#0",driver_instance="tcp,localhost:1235",path="/var/syslog-ng-00000.rqf",reliable="true"} 104853504<br>    syslogng_disk_queue_disk_allocated_bytes{driver_id="d_network#0",driver_instance="tcp,localhost:1235",path="/var/syslog-ng-00000.rqf",reliable="true"} 17284<br>    syslogng_disk_queue_disk_usage_bytes{driver_id="d_network#0",driver_instance="tcp,localhost:1235",path="/var/syslog-ng-00000.rqf",reliable="true"} 13188<br>    ```<br>    ([#4356](<a href="https://github.com/syslog-ng/syslog-ng/pull/4356">https://github.com/syslog-ng/syslog-ng/pull/4356</a>))<br><br>  * `kubernetes()`: Added `input_events_total` and `input_event_bytes_total` metrics.<br><br>    ```<br>    syslogng_input_events_total{cluster="k8s",driver="kubernetes",id="#anon-source0",namespace="default",pod="log-generator-1682517834-7797487dcc-49hqc"} 25<br>    syslogng_input_event_bytes_total{cluster="k8s",driver="kubernetes",id="#anon-source0",namespace="default",pod="log-generator-1682517834-7797487dcc-49hqc"} 1859<br>    ```<br>    ([#4447](<a href="https://github.com/syslog-ng/syslog-ng/pull/4447">https://github.com/syslog-ng/syslog-ng/pull/4447</a>))<br><br>## Bugfixes<br><br>  * `pdbtool test`: fix two type validation bugs:<br><br>      1) When `pdbtool test` validates the type information associated with a name-value<br>         pair, it was using string comparisons, which didn't take type aliases<br>         into account. This is now fixed, so that "int", "integer" or "int64"<br>         can all be used to mean the same type.<br><br>      2) When type information is missing from a `<test_value/>` tag, don't<br>         validate it against "string", rather accept any extracted type.<br><br>    In addition to these fixes, a new alias "integer" was added to mean the same<br>    as "int", simply because syslog-ng was erroneously using this term when<br>    reporting type information in its own messages.<br>    ([#4405](<a href="https://github.com/syslog-ng/syslog-ng/pull/4405">https://github.com/syslog-ng/syslog-ng/pull/4405</a>))<br><br>  * `$(format-json)`: fix RFC8259 number violation<br><br>    `$(format-json)` produced invalid JSON output when it contained numeric values with leading zeros or + signs.<br>    This has been fixed.<br>    ([#4415](<a href="https://github.com/syslog-ng/syslog-ng/pull/4415">https://github.com/syslog-ng/syslog-ng/pull/4415</a>))<br><br>  * `grouping-by()`: fix `persist-name()` option not taken into account<br>    ([#4390](<a href="https://github.com/syslog-ng/syslog-ng/pull/4390">https://github.com/syslog-ng/syslog-ng/pull/4390</a>))<br><br>  * `python()`, `db-parser()`, `grouping-by()`, `add-contextual-data()`: fix typing compatibility with <4.0 config versions<br>    ([#4394](<a href="https://github.com/syslog-ng/syslog-ng/pull/4394">https://github.com/syslog-ng/syslog-ng/pull/4394</a>))<br><br>  * `python`: Fixed a crash which occurred at reloading after registering a confgen plugin.<br>    ([#4459](<a href="https://github.com/syslog-ng/syslog-ng/pull/4459">https://github.com/syslog-ng/syslog-ng/pull/4459</a>))<br><br>  * `date-parser()`: fix `%z` when system timezone has no daylight saving time<br>    ([#4401](<a href="https://github.com/syslog-ng/syslog-ng/pull/4401">https://github.com/syslog-ng/syslog-ng/pull/4401</a>))<br><br>  * Consider messages consumed into correlation states "matching": syslog-ng's<br>    correlation functionality (e.g.  grouping-by() or db-parser() with such<br>    rules) drop individual messages as they are consumed into a correlation<br>    contexts and you are using `inject-mode(aggregate-only)`.  This is usually<br>    happens because you are only interested in the combined message and not in<br>    those that make up the combination. However, if you are using correlation<br>    with conditional processing (e.g. if/elif/else or flags(final)), such<br>    messages were erroneously considered as unmatching, causing syslog-ng to<br>    take the alternative branch.<br><br>    Example:<br><br>    With a configuration similar to this, individual messages are consumed into<br>    a correlation state and dropped by `grouping-by()`:<br><br>    ```<br>    log {<br>        source(...);<br><br>        if {<br>            grouping-by(... inject-mode(aggregate-only));<br>        } else {<br>            # alternative branch<br>        };<br>    };<br>    ```<br><br>    The bug was that these individual messages also traverse the `else` branch,<br>    even though they were successfully processed with the inclusion into the<br>    correlation context. This is not correct. The bugfix changes this behaviour.<br>    ([#4370](<a href="https://github.com/syslog-ng/syslog-ng/pull/4370">https://github.com/syslog-ng/syslog-ng/pull/4370</a>))<br><br>  * `netmask6()`: fix crash when user specifies too long mask<br>    ([#4429](<a href="https://github.com/syslog-ng/syslog-ng/pull/4429">https://github.com/syslog-ng/syslog-ng/pull/4429</a>))<br><br>  * `afprog`: Fixed possible freezing on some OSes<br>    ([#4438](<a href="https://github.com/syslog-ng/syslog-ng/pull/4438">https://github.com/syslog-ng/syslog-ng/pull/4438</a>))<br><br>  * `network()`, `syslog()`, `syslog-parser()`: fix null termination of SDATA param names<br>    ([#4429](<a href="https://github.com/syslog-ng/syslog-ng/pull/4429">https://github.com/syslog-ng/syslog-ng/pull/4429</a>))<br><br>  * `python()`: fix LogMessage subscript not raising KeyError on non-existent keys<br><br>    When message fields were queried (`msg["key"]`) and the given key did not exist,<br>    `None` or an empty string was returned (depending on the version of the config).<br><br>    Neither was correct, now a KeyError occurs in such cases.<br>    ([#4410](<a href="https://github.com/syslog-ng/syslog-ng/pull/4410">https://github.com/syslog-ng/syslog-ng/pull/4410</a>))<br><br>  * `$(python)`: fix template function prefix being overwritten when using datetime types<br>    ([#4410](<a href="https://github.com/syslog-ng/syslog-ng/pull/4410">https://github.com/syslog-ng/syslog-ng/pull/4410</a>))<br><br>  * `disk-buffer`: Fixed queued messages stats counting, when a disk-buffer became corrupted.<br>    ([#4385](<a href="https://github.com/syslog-ng/syslog-ng/pull/4385">https://github.com/syslog-ng/syslog-ng/pull/4385</a>))<br><br>  * `$(format-json)`: fix escaping control characters<br><br>    `$(format-json)` produced invalid JSON output when a string value contained control characters.<br>    ([#4417](<a href="https://github.com/syslog-ng/syslog-ng/pull/4417">https://github.com/syslog-ng/syslog-ng/pull/4417</a>))<br><br>  * `disk-buffer()`: fix deinitialization when starting syslog-ng with invalid configuration<br>    ([#4418](<a href="https://github.com/syslog-ng/syslog-ng/pull/4418">https://github.com/syslog-ng/syslog-ng/pull/4418</a>))<br><br>  * `python()`: fix exception handling when LogMessage value conversion fails<br>    ([#4410](<a href="https://github.com/syslog-ng/syslog-ng/pull/4410">https://github.com/syslog-ng/syslog-ng/pull/4410</a>))<br><br>  * `json-parser()`: Fixed parsing non-string arrays.<br><br>    syslog-ng now no longer parses non-string arrays to list of strings, losing the original type<br>    information of the array's elements.<br>    ([#4396](<a href="https://github.com/syslog-ng/syslog-ng/pull/4396">https://github.com/syslog-ng/syslog-ng/pull/4396</a>))<br><br>  * `disk-buffer`: Fixed a rare race condition when calculating disk-buffer filename.<br>    ([#4381](<a href="https://github.com/syslog-ng/syslog-ng/pull/4381">https://github.com/syslog-ng/syslog-ng/pull/4381</a>))<br><br>  * `python-persist`: fix off-by-one overflow<br>    ([#4429](<a href="https://github.com/syslog-ng/syslog-ng/pull/4429">https://github.com/syslog-ng/syslog-ng/pull/4429</a>))<br><br>## Packaging<br>  * The `--with-python-venv-dir=path` configure option can be used to modify the location of syslog-ng's venv.<br>    The default is still `${localstatedir}/python-venv`.<br>    ([#4465](<a href="https://github.com/syslog-ng/syslog-ng/pull/4465">https://github.com/syslog-ng/syslog-ng/pull/4465</a>))<br><br><br>## Other changes<br><br>  * The `sdata-prefix()` option does not accept values longer than 128 characters.<br>    ([#4429](<a href="https://github.com/syslog-ng/syslog-ng/pull/4429">https://github.com/syslog-ng/syslog-ng/pull/4429</a>))<br><br>  * `grouping-by()`: Remove setting of the `${.classifier.context_id}`<br>    name-value pair in all messages consumed into a correlation context.  This<br>    functionality is inherited from db-parser() and has never been documented<br>    for `grouping-by()`, has of limited use, and any uses can be replaced by the<br>    use of the built-in macro named `$CONTEXT_ID`.  Modifying all consumed<br>    messages this way has significant performance consequences for<br>    `grouping-by()` and removing it outweighs the small incompatibility this<br>    change introduces. The similar functionality in `db-parser()` correlation is<br>    not removed with this change.<br>    ([#4424](<a href="https://github.com/syslog-ng/syslog-ng/pull/4424">https://github.com/syslog-ng/syslog-ng/pull/4424</a>))<br><br>  * `config`: Added `internal()` option to `source`s, `destination`s, `parser`s and `rewrite`s.<br><br>    Its main usage is in SCL blocks. Drivers configured with `internal(yes)` register<br>    their metrics on level 3. This makes developers of SCLs able to create metrics manually<br>    with `metrics-probe()` and "disable" every other metrics, they do not need.<br>    ([#4451](<a href="https://github.com/syslog-ng/syslog-ng/pull/4451">https://github.com/syslog-ng/syslog-ng/pull/4451</a>))<br><br>  * The following Prometheus metrics have been renamed:<br><br>    `log_path_{in,e}gress` -> `route_{in,e}gress_total`<br>    `internal_source` -> `internal_events_total`<br><br>    The `internal_queue_length` stats counter has been removed.<br>    It was deprecated since syslog-ng 3.29.<br>    ([#4411](<a href="https://github.com/syslog-ng/syslog-ng/pull/4411">https://github.com/syslog-ng/syslog-ng/pull/4411</a>))<br><br><br>## syslog-ng Discord<br><br>For a bit more interactive discussion, join our Discord server:<br><a href="https://discord.gg/E65kP9aZGm">https://discord.gg/E65kP9aZGm</a><br><br>## Credits<br><br>syslog-ng is developed as a community project, and as such it relies<br>on volunteers, to do the work necessarily to produce syslog-ng.<br><br>Reporting bugs, testing changes, writing code or simply providing<br>feedback are all important contributions, so please if you are a user<br>of syslog-ng, contribute.<br><br>We would like to thank the following people for their contribution:<br><br>Alex Becker, Attila Szakacs, Balazs Scheidler, Hofi, László Várady,<br>Muhammad Shanif, Ricfilipe, Romain Tartière<br></div><div><br><div><br></div><div><br></div></div></div>