[Bug 185] New: Use upstream ivykis (>= 0.30) instead of a fork
https://bugzilla.balabit.com/show_bug.cgi?id=185 Summary: Use upstream ivykis (>= 0.30) instead of a fork Product: syslog-ng Version: 3.3.x Platform: All OS/Version: All Status: ASSIGNED Severity: blocker Priority: unspecified Component: syslog-ng AssignedTo: algernon@balabit.hu ReportedBy: algernon@balabit.hu Type of the Report: enhancement Estimated Hours: 0.0 With upstream ivykis 0.30 having everything the syslog-ng internal fork had, and more, it's time to drop the fork, and compile with upstream ivykis as-is. This would mean that the embedded copy (which we will still ship) is replaced by ivykis 0.30, and syslog-ng will become compileable against upstream ivykis. The bulk of the work is already done on my feature/3.3/ivykis-upstream branch, only a few issues remain (see below) before this can be merged to 3.3 master, and this bug report is here to keep track of progress. While trying to merge the branch, I conducted a series of tests, and it appears that somewhere along the way the file destination broke (and most likely everything else that is based on logwriter), and it deadlocks. I do not yet know why, but this feature can't be merged until this problem is solved. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=185 Gergely Nagy <algernon@balabit.hu> changed: What |Removed |Added ---------------------------------------------------------------------------- Target Milestone|--- |3.3.6 -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=185 Jose Oliveira <jpo@di.uminho.pt> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jpo@di.uminho.pt -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=185 Matthias Runge <mrunge@fedoraproject.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mrunge@fedoraproject.org -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=185 --- Comment #1 from Gergely Nagy <algernon@balabit.hu> 2012-06-30 15:29:24 --- It appears that the deadlock I saw during test was in ivykis, and Lennert Buytenhek debugged & fixed the issue shortly after this bug was opened. I pushed a change to the feature/3.3/ivykis-upstream branch that points the lib/ivykis submodule to the commit that fixes the deadlock. I'll conduct a few more tests, and most likely merge the branch onto master tomorrow. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=185 Balazs Scheidler <bazsi@balabit.hu> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bazsi@balabit.hu --- Comment #2 from Balazs Scheidler <bazsi@balabit.hu> 2012-06-30 20:59:33 --- once this is settled I'd want to merge the same for 3.4 too. Great work, both you and Lennert. Thanks a lot. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=185 Gergely Nagy <algernon@balabit.hu> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution| |FIXED Status|ASSIGNED |RESOLVED --- Comment #3 from Gergely Nagy <algernon@balabit.hu> 2012-07-01 11:29:53 --- After a day and a half of extensive testing on multiple platforms, I found no regressions, so I merged the feature/3.3/ivykis-upstream branch to master. I triggered a snapshot tarball build, so http://packages.madhouse-project.org/syslog-ng/3.3/syslog-ng-3.3-HEAD.tar.gz is updated already. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=185 --- Comment #4 from Jose Oliveira <jpo@di.uminho.pt> 2012-07-01 15:29:31 --- (In reply to comment #3) ...
I triggered a snapshot tarball build, so http://packages.madhouse-project.org/syslog-ng/3.3/syslog-ng-3.3-HEAD.tar.gz is updated already.
New SRPM: syslog-ng-3.3.6-0.1.1.fc17.src.rpm (source tarball: syslog-ng-3.3.5-20120701-v3.3.5-30-g9020.tar.gz) Build logs: http://um-pe09-2.di.uminho.pt/fedora/build.logs/fedora-17-i386/syslog-ng-3.3... http://um-pe09-2.di.uminho.pt/fedora/build.logs/fedora-17-x86_64/syslog-ng-3... Notes: Several tests are core dumping in Fedora 17: --------- ... PASS: test_matcher /bin/sh: line 5: 9791 Segmentation fault (core dumped) ${dir}$tst FAIL: test_clone_logmsg /bin/sh: line 5: 9880 Segmentation fault (core dumped) ${dir}$tst FAIL: test_csvparser /bin/sh: line 5: 9934 Segmentation fault (core dumped) ${dir}$tst FAIL: test_serialize /bin/sh: line 5: 10020 Segmentation fault (core dumped) ${dir}$tst FAIL: test_msgparse ... ---------- -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=185 --- Comment #5 from Gergely Nagy <algernon@balabit.hu> 2012-07-01 17:41:12 --- (In reply to comment #4)
(In reply to comment #3) ...
I triggered a snapshot tarball build, so http://packages.madhouse-project.org/syslog-ng/3.3/syslog-ng-3.3-HEAD.tar.gz is updated already.
New SRPM: syslog-ng-3.3.6-0.1.1.fc17.src.rpm
Grabbed it and testing it locally, thanks! So far, I managed to reproduce the problem on Fedora 17, but not on any Debian or FreeBSD variant (I tested those before merging the branch, and double checked now too). With a bit of luck, I'll figure out what went wrong within a couple of hours. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=185 --- Comment #6 from Gergely Nagy <algernon@balabit.hu> 2012-07-01 17:56:43 --- (In reply to comment #5)
With a bit of luck, I'll figure out what went wrong within a couple of hours.
Got it. A change unrelated to ivykis broke it. Fix is available in git, and a patch at http://git.madhouse-project.org/debian/syslog-ng/patch/?id=77df2c184124d5b7b... The SRPM built successfully in my Fedora 17 VM, all tests passed. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=185 --- Comment #7 from Jose Oliveira <jpo@di.uminho.pt> 2012-07-01 18:31:56 --- (In reply to comment #6)
(In reply to comment #5)
With a bit of luck, I'll figure out what went wrong within a couple of hours.
Got it. A change unrelated to ivykis broke it. Fix is available in git, and a patch at http://git.madhouse-project.org/debian/syslog-ng/patch/?id=77df2c184124d5b7b...
The SRPM built successfully in my Fedora 17 VM, all tests passed.
The build doesn't finish with the above patch applied (in mock and locally). My building system is quad core with ht (Intel i7). ps aux ---------- ... fedora 9605 0.0 0.0 4576 952 pts/0 S+ 17:25 0:00 make check fedora 9606 0.0 0.0 9492 1204 pts/0 S+ 17:25 0:00 /bin/sh -c fail= failcom='exit 1'; \ for f in x $MAKEFLAGS; do \ case $f in \ *=* | fedora 10196 0.0 0.0 9492 632 pts/0 S+ 17:25 0:00 /bin/sh -c fail= failcom='exit 1'; \ for f in x $MAKEFLAGS; do \ case $f in \ *=* | fedora 10197 0.0 0.0 4580 944 pts/0 S+ 17:25 0:00 make check fedora 10198 0.0 0.0 9492 1184 pts/0 S+ 17:25 0:00 /bin/sh -c fail= failcom='exit 1'; \ for f in x $MAKEFLAGS; do \ case $f in \ *=* | fedora 13048 0.0 0.0 9492 632 pts/0 S+ 17:25 0:00 /bin/sh -c fail= failcom='exit 1'; \ for f in x $MAKEFLAGS; do \ case $f in \ *=* | fedora 13049 0.0 0.0 4580 932 pts/0 S+ 17:25 0:00 make check fedora 13051 0.0 0.0 4580 936 pts/0 S+ 17:25 0:00 make check-TESTS fedora 13053 0.0 0.0 9492 1220 pts/0 S+ 17:25 0:00 /bin/sh -c failed=0; all=0; xfail=0; xpass=0; skip=0; \ srcdir=.; export srcdir; \ list=' fedora 13054 0.0 0.0 65508 6588 pts/0 S+ 17:25 0:00 python ./func_test.py fedora 13077 0.1 0.0 684368 6100 pts/0 Sl+ 17:25 0:00 /home/fedora/rpms/BUILD/syslog-ng-3.3.5/syslog-ng/.libs/lt-syslog-ng -f test.conf --fd-li ... --------- Local build output: --------- ############################################## ### Starting testcase: test_input_drivers.test_catchall ############################################## ... 2012-25-01T17:07:26 Testcase start Follow-mode file source not found, deferring open; filename='log-file' syslog-ng starting up; version='3.3.5' 2012-25-01T17:07:31 Syslog-ng started 2012-25-01T17:07:31 Starting test case... 2012-25-01T17:07:31 generating 100 messages using transport unix-stream(log-stream-catchall) 2012-25-01T17:07:31 generating 100 messages using transport unix-dgram(log-dgram) 2012-25-01T17:07:32 generating 100 messages using transport unix-dgram(log-dgram) 2012-25-01T17:07:33 generating 100 messages using transport unix-dgram(log-dgram) 2012-25-01T17:07:34 generating 100 messages using transport unix-stream(log-stream) 2012-25-01T17:07:34 generating 100 messages using transport unix-stream(log-stream) 2012-25-01T17:07:34 generating 100 messages using transport udp(('localhost', 46054)) 2012-25-01T17:07:35 generating 100 messages using transport udp(('localhost', 46054)) 2012-25-01T17:07:36 generating 100 messages using transport udp(('localhost', 46054)) 2012-25-01T17:07:37 generating 100 messages using transport tcp(('localhost', 46054)) Syslog connection accepted; fd='37', client='AF_INET(127.0.0.1:38347)', local='AF_INET(0.0.0.0:46054)' 2012-25-01T17:07:37 generating 100 messages using transport tcp(('localhost', 46054)) Syslog connection accepted; fd='38', client='AF_INET(127.0.0.1:38348)', local='AF_INET(0.0.0.0:46054)' 2012-25-01T17:07:37 generating 100 messages using transport pipe(log-pipe) 2012-25-01T17:07:37 generating 100 messages using transport pipe(log-pipe) ---------- -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=185 --- Comment #8 from Jose Oliveira <jpo@di.uminho.pt> 2012-07-01 18:33:27 --- I forgot to mention that I'm using ivykis 0.30. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=185 Lennert Buytenhek <buytenh@wantstofly.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |buytenh@wantstofly.org --- Comment #9 from Lennert Buytenhek <buytenh@wantstofly.org> 2012-07-01 18:49:00 --- (In reply to comment #8)
I forgot to mention that I'm using ivykis 0.30.
You could be hitting the iv_work deadlock issue in 0.30 that will be fixed in 0.31. I'll release 0.31 shortly. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
https://bugzilla.balabit.com/show_bug.cgi?id=185 --- Comment #10 from Jose Oliveira <jpo@di.uminho.pt> 2012-07-01 20:24:03 --- (In reply to comment #9)
(In reply to comment #8)
I forgot to mention that I'm using ivykis 0.30.
You could be hitting the iv_work deadlock issue in 0.30 that will be fixed in 0.31. I'll release 0.31 shortly.
Yep. Applying the ivykis patch https://github.com/buytenh/ivykis/commit/38d7359103ab988cc4a7b8089b81a455c5a... against the 0.30 tarball fixed the problem. SRPMs: ivykis-0.30-0.3.fc17.src.rpm ---------- ivykis-0.30.tar.gz ivykis-0.31-deadlock-38d73591.patch ivykis.spec ---------- syslog-ng-3.3.6-0.1.2.fc17.src.rpm ---------- syslog-ng-3.2.5-tests-functional-control.py.patch syslog-ng-3.3.5-20120701-v3.3.5-30-g9020.tar.gz syslog-ng-3.3.6-fix-core-dumps-in-some-tests.patch syslog-ng-3.3.6-syslog-ng.service.patch syslog-ng-3.3.6-tests-functional-sql-test.patch syslog-ng.conf syslog-ng.logrotate syslog-ng.spec ---------- Build logs and binary RPMS: http://um-pe09-2.di.uminho.pt/fedora/build.logs/ We just need to bump the ivykis requirement to 0.31 :) -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
participants (1)
-
bugzilla＠bugzilla.balabit.com