Hi, I have spent the last two days on hacking syslog-ng with a good number of fixes and couple of features requested a long time ago. Syslog-ng 1.9.8 got into Debian sid and apart from a couple of problems the house is not burning :) In summary syslog-ng 1.9.x is getting into shape, and slowly might be ready for some more production-like environments. I would like to ask you to give a try to these snapshot releases I've just uploaded to our website. Assuming no grave problems are found in the next day or two I'll release 1.9.9 and hopefully we can have a stable 2.0.0 in the nearfuture. I'm still not happy with the way log statistics (per source/destination dropped/processed counters) are written to the system log. I'm starting to think that the logfile is not necessarily the best medium to write this information to. I'm open to suggestions. The exact versions: eventlog-0.2.4+20060212.tar.gz syslog-ng-1.9.8+20060212.tar.gz The NEWS file entry is the longest ever in the history of 1.9.x, here it is: 1.9.9 Some minor new features: * Added optional() option to pipe and unix drivers to make syslog-ng start even if the required directories/files do not exist. * Added DNS name resolution to udp and tcp targets. * Added a stats() keyword for 1.6.x compatibility. * Added processed counters for source/destination groups and the log center. * Added normalize_hostnames() option which converts all hostnames to lower case. * Added PID macro. * Added kernel flag to sources to indicate that messages coming from the source should default to 'kern.crit' instead of 'user.notice' * Added frac_digits() option which controls how many digits are printed in second fractions. * Documentation updates. Changes: * Removed stats entries for files as they never lose messages and it would only clutter the log statistics output. * The meaning of the undocumented keep_timestamp() option was changed and documented, it controls whether syslog-ng uses the time of reception or the time included in the log message. Bugfixes: * Fixed port unreachable handling for UDP destinations. * Fixed PRI macro processing as it included the local hostname in addition to the priority value because of a missing break statement. * UNIX domain sockets are kept alive across SIGHUPs by default. * Fixed a possible segmentation fault on SIGHUP. * Fixed timezone extraction from incoming messages with ISO timestamp (only this syslog-ng can send these currently) * Fixed HOST_FROM, FULLHOST_FROM, SOURCEIP, DATE, R_DATE, S_DATE macros (some were still not implemented others worked incorrectly in some circumstances) * Fixed fractions of a second processing. -- Bazsi
Hello,
I have spent the last two days on hacking syslog-ng with a good number of fixes and couple of features requested a long time ago. Syslog-ng 1.9.8 got into Debian sid and apart from a couple of problems the house is not burning :)
Congratulations and thanks for your hard work.
In summary syslog-ng 1.9.x is getting into shape, and slowly might be ready for some more production-like environments.
... if the segfaults are gone ;).
I would like to ask you to give a try to these snapshot releases I've just uploaded to our website. Assuming no grave problems are found in the next day or two I'll release 1.9.9 and hopefully we can have a stable 2.0.0 in the nearfuture.
So this is a feature freeze?
I'm still not happy with the way log statistics (per source/destination dropped/processed counters) are written to the system log. I'm starting to think that the logfile is not necessarily the best medium to write this information to. I'm open to suggestions.
You could do it the squid way (or httpd for that matter) and have an external syslog-ng client (similar to squidclient) to poll or dump internal stats. Is that more to your liking?
1.9.9
Some minor new features:
* Added optional() option to pipe and unix drivers to make syslog-ng start even if the required directories/files do not exist. * Added DNS name resolution to udp and tcp targets. * Added a stats() keyword for 1.6.x compatibility. * Added processed counters for source/destination groups and the log center. * Added normalize_hostnames() option which converts all hostnames to lower case. * Added PID macro. * Added kernel flag to sources to indicate that messages coming from the source should default to 'kern.crit' instead of 'user.notice'
Rather than fix up the kernel source?
* Added frac_digits() option which controls how many digits are printed in second fractions. * Documentation updates.
Changes:
* Removed stats entries for files as they never lose messages and it would only clutter the log statistics output. * The meaning of the undocumented keep_timestamp() option was changed and documented, it controls whether syslog-ng uses the time of reception or the time included in the log message.
Bugfixes:
* Fixed port unreachable handling for UDP destinations. * Fixed PRI macro processing as it included the local hostname in addition to the priority value because of a missing break statement. * UNIX domain sockets are kept alive across SIGHUPs by default. * Fixed a possible segmentation fault on SIGHUP.
Thank you! Could you point me to the respective patch, please, since I tried to fix that one in the past and spent 4 hours in vain. I would like to improve my debugging abilities regarding syslog-ng and understand your architecture better.
* Fixed timezone extraction from incoming messages with ISO timestamp (only this syslog-ng can send these currently) * Fixed HOST_FROM, FULLHOST_FROM, SOURCEIP, DATE, R_DATE, S_DATE macros (some were still not implemented others worked incorrectly in some circumstances) * Fixed fractions of a second processing.
I'll see if we get 1.9.9 into our testing phase, despite having decided to go with the stable 1.6.9 for our servers. Previous versions of syslog-ng-1.9.x just didn't cut it. Best regards, Roberto Nibali, ratz -- echo '[q]sa[ln0=aln256%Pln256/snlbx]sb3135071790101768542287578439snlbxq' | dc
On Mon, 2006-02-20 at 02:06 +0100, Roberto Nibali wrote:
... if the segfaults are gone ;).
I've fixed a segfault in this release, and as of now I don't know about any others.
I would like to ask you to give a try to these snapshot releases I've just uploaded to our website. Assuming no grave problems are found in the next day or two I'll release 1.9.9 and hopefully we can have a stable 2.0.0 in the nearfuture.
So this is a feature freeze?
Yes, I'd like to release a stable syslog-ng 2.0 out of the door. The more exciting features will be added to a new branch.
I'm still not happy with the way log statistics (per source/destination dropped/processed counters) are written to the system log. I'm starting to think that the logfile is not necessarily the best medium to write this information to. I'm open to suggestions.
You could do it the squid way (or httpd for that matter) and have an external syslog-ng client (similar to squidclient) to poll or dump internal stats. Is that more to your liking?
Yes, that is something I was thinking about, I originally wanted to avoid the complexity of a control channel. That might be missing from 2.0.0 though.
* Added kernel flag to sources to indicate that messages coming from the source should default to 'kern.crit' instead of 'user.notice'
Rather than fix up the kernel source?
Sometimes messages from the kernel are not prefixed by a proper priority prefix. I had this request in our bugzilla and as it was trivial I implemented it.
* Fixed a possible segmentation fault on SIGHUP.
Thank you! Could you point me to the respective patch, please, since I tried to fix that one in the past and spent 4 hours in vain. I would like to improve my debugging abilities regarding syslog-ng and understand your architecture better.
The problem was a stale reference between AFSocketSourceConnection and the AFSocketSourceDriver objects through SIGHUP. AFSocketSourceConnections are kept through SIGHUP (so that they are not closed), but AFSocketSourceDrivers are not. Similarly the LogReader associated with AFSocketSourceConnection has a reference to its LogReaderOptions which was again stored in AFSocketSourceDriver. The solution was get rid of this stale reference to point to the newly created driver object, this was archieved by adding a afsocket_sc_set_owner() function which makes sure all references are properly updated. See the changelog entry for 2006-02-11 for more details.
* Fixed fractions of a second processing.
I'll see if we get 1.9.9 into our testing phase, despite having decided to go with the stable 1.6.9 for our servers. Previous versions of syslog-ng-1.9.x just didn't cut it.
Thanks. -- Bazsi
I've fixed a segfault in this release, and as of now I don't know about any others.
Ok, all segfaults I've seen can be attributed to the SIGHUP case.
I would like to ask you to give a try to these snapshot releases I've just uploaded to our website. Assuming no grave problems are found in the next day or two I'll release 1.9.9 and hopefully we can have a stable 2.0.0 in the nearfuture.
So this is a feature freeze?
Yes, I'd like to release a stable syslog-ng 2.0 out of the door. The more exciting features will be added to a new branch.
Thanks, this is good to know. So with this in mind, I believe we can give syslog-ng 2.0 a serious thought. This would give us and you a broad test base. The idea would be to go through all our internal bugzilla entries related to logging or syslog-ng and re-test them with the new code base.
You could do it the squid way (or httpd for that matter) and have an external syslog-ng client (similar to squidclient) to poll or dump internal stats. Is that more to your liking?
Yes, that is something I was thinking about, I originally wanted to avoid the complexity of a control channel. That might be missing from 2.0.0 though.
No problem.
* Added kernel flag to sources to indicate that messages coming from the source should default to 'kern.crit' instead of 'user.notice'
Rather than fix up the kernel source?
Sometimes messages from the kernel are not prefixed by a proper priority prefix. I had this request in our bugzilla and as it was trivial I implemented it.
So long as it remains optional. However I do not agree with kern.crit as a default because kern.crit has very specific semantic meaning in the linux kernel. Why not resort to kern.info or make it configurable? Also how does the klogd alternative handle this?
* Fixed a possible segmentation fault on SIGHUP.
Thank you! Could you point me to the respective patch, please, since I tried to fix that one in the past and spent 4 hours in vain. I would like to improve my debugging abilities regarding syslog-ng and understand your architecture better.
The problem was a stale reference between AFSocketSourceConnection and the AFSocketSourceDriver objects through SIGHUP.
AFSocketSourceConnections are kept through SIGHUP (so that they are not closed), but AFSocketSourceDrivers are not. Similarly the LogReader associated with AFSocketSourceConnection has a reference to its LogReaderOptions which was again stored in AFSocketSourceDriver.
... and booom! I see the problem, nasty. At least, now you know that you have no fd leakage :).
The solution was get rid of this stale reference to point to the newly created driver object, this was archieved by adding a afsocket_sc_set_owner() function which makes sure all references are properly updated.
Nice solution.
See the changelog entry for 2006-02-11 for more details.
I'll do that, thanks. Best regards, Roberto Nibali, ratz -- echo '[q]sa[ln0=aln256%Pln256/snlbx]sb3135071790101768542287578439snlbxq' | dc
participants (2)
-
Balazs Scheidler
-
Roberto Nibali