Hi, Thanks for your contribution. I still made some additional adjustments to your patch, eliminated the shifts used for size/len variables (which was used to stuff 256k into 16 bit variables). This became unneeded as 32 bit can be more than enough. I still added an artifical limit to make NVTable instances less than 256MB, hope that's still enough for your usecase. The patches are not yet on master, but are available in the "32bit-nvtable" branch in my github repo. Also, I hope you could sign off your patch, as that's needed for proper attribution. This is the same policy that the Linux kernel uses, nothing but something similar like: Signed-off-by: Peter Eisenlohr <peter.eisenlohr@inform-software.com> beside the patch, so that I can include it in the commit message. This change caused a slightly measurable drop in performance in one of my microbenchmark, but nothing really intolerable. The per-value overhead in NVTable became 12 bytes instead of the previous 6, but I didn't have any better idea, rather than imposing stricter limits like I did previously. Also, I'd welcome any feedback you might have on the NVTable code, it's a tricky one at best, and you really went ahead and did things right. Was it easy to understand/modify? Thanks again, On Tue, 2012-01-10 at 13:14 +0100, Peter Eisenlohr wrote:
Hi,
as requested by bazsi in the forums
http://www.syslog.org/forum/index.php?topic=1218.0
here's a patch against syslog-ng enabling it to handle log messages larger than the current 256k limit. This is done by extending NVTable's data structures from 16 to 32 bit. The patch also contains simple code for automatic resizing of the receive buffer. 'log_msg_size' is still honoured, though.
I have been using this to regularly receive multi-MB XML messages and process them into log files.
I developed the patch against 3.3.1, but also tested it with 3.3.3 and on x64 as well. 'make check' runs fine, too -- but still, use at your own risk.
Peter ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Bazsi