Hello Balazs, thanks a lot for your fast response. We need a few days to consolidate our thoughts against syslog, syslog-ng and the whole stuff. We are hoping of your comments or suggestions. At the end there are some questions against syslog-ng. Maybe it's easy for you to answer.
we are observing a wondrous behavior on our central logservers.
We have an local application, that logs to the destinations above. And all messages greater that approx. 975 Bytes are truncated.
But the enigma is: If using the local Solaris 'logger' application, all messages greater than 509 Bytes are truncated. But they are truncated on different positions depends on the message size! (Say, if you log 600 Bytes, you get the first 509 and if you try to log 761, you get the first 26.) Hmm ?!
truss either syslog-ng or logger to see what it actually sends. log messages might be garbled if they are around 1024 in size, since syslog-ng might also modify it (with results in truncated messages again).
Trussing our syslog-ng isn't possible, because the daemons are heavy loaded. So we trussed the 'logger' app (using large messages) under SunOS/Linux. Under Solaris its difficult to see the msg_len sent at the first view, because Solaris uses the SVR4 log driver behavior via putmsg(2). But Linux, that uses the BSD style UNIX_DGRAM behavior, shows that the message is truncated at OS log driver layer. Solaris: open("/dev/conslog", O_WRONLY) = 1 putmsg(1, 0xFFBEE2E8, 0xFFBEE2DC, 0) = 0 Linux: socket(PF_UNIX, SOCK_DGRAM, 0) = 1 connect(1, {sin_family=AF_UNIX, path=" /dev/log"}, 16) = 0 send(1, "<11>Jul 2 16:47:57 DEBUG: 12345"..., 1016, 0) = 1016 Hmm. Locks not really good. A lock at "The Stevens (APUE)" validates, that SunOS implements a SVR4 log system with an additional BSD style syslog(3) interface to this. Anyone knows, how its possible to configure the log(7d) driver on Solaris to process large (>1024 Bytes) messages? Now, we remembered that there is a syslog 'standard' in RFC 3164. Ok, there is a message limit of 1024 over all, but its only referring to UDP. What's with the local log systems? System depended? Huargh.
Imagine the following scenario:
- message to syslog-ng (with no host and date parts), 1024 bytes in size - syslog-ng rewrites the messages, inserts host and date (internally these are stored in different variables where the 1024 limit is not appliced) - syslog-ng formats the new message (according to the template given, or the usual syslog format) where the host and date parts of the message are inserted, the message now becomes more than 1024 bytes, thus the message is truncated
We duplicated your problematic local log's´using 2 destinations. Dest_1 used template("..."), Dest_2 not. The messages are truncated for both destinations. Only for our interest: Is the string created by syslog-ng for holding the results of template(...) size limited?
If syslog-ng reads a message which exceeds 1024, the remaining is read as an independent message
It seems, that this doesn't work in our context. Seems that the syslog(3) interface or the log(7d) driver truncates the messages before they can arrive syslog-ng. We now try do write directly to /dev/*log*, to ensure that the problem is in log(7d). Is the syslog message format in RFC 3164 your reference, Balazs? Thanks for any hints. -- Best regards --Andreas Schulze [phone: +49.5246.80.1275, fax: +49.5246.80.2275] | I believe, it was Dennis Ritchie who said something like: | "C is rarely the best language for a given task, | but it's often the second-best". | The implication being that: "[...]" | | sh# cat>$$.c<<EOT | main(l,a,n,d)char**a;{for(d=atoi(a[1])/10*80-atoi(a[2])/5-596;n="@NK\ | ACLCCGZAAQBEAADAFaISADJABBA^SNLGAQABDAXIMBAACTBATAHDBANZcEMMCCCCAAhE\ | IJFAEAAABAfHJETBdFLDAANEfDNBPHdBcBBBEA_AL H E L L O, W O R L D! " | [l++-3];)for(;n-->64;)putchar(!d+++33^l&1);} | EOT | gcc -o$$ $$.c;clear;./$$ 52 8;rm -f $$*