Hi all, I am currently setting up a remote logging facility for a customer who requires encryption and mutual authentication. After having run Syslog NG over Stunnel for several years, I am looking into the option to do the same withot stunnel, by using Syslog NG with the TLS option. My current version is 3.2.2, installed from the binary provided on the Syslog NG web site, on CentOS 5.5. My setup on the server side (stripped to the minimum): source s_remote { tcp( ip( 192.168.42.184 ) port( 601 ) tls( key_file("/opt/syslog-ng/etc/ssl/server_key.pem") cert_file("/opt/syslog-ng/etc/ssl/server_crt.pem") ca_dir( "/opt/syslog-ng/etc/ssl" ) peer_verify( required-trusted ) ) ); }; destination d_remote { file( "/var/log/remote" owner(root) group(root) perm(0600) ); }; log { source(s_remote); destination(d_remote); }; On the client side: source s_local { internal(); unix-stream( "/dev/log" ); file( "/proc/kmsg" ); }; destination d_remote { tcp( "192.168.42.184" port( 601 ) tls( key_file("/opt/syslog-ng/etc/ssl/client1_key.pem") cert_file("/opt/syslog-ng/etc/ssl/client1_crt.pem") ca_dir( "/opt/syslog-ng/etc/ssl" ) ) ); }; log { source(s_local); destination(d_remote); }; With the appropriate and valid client cert, it works perfrctly: [root@server1 ssl]# openssl x509 -noout -text < client1_crt.pem Certificate: Data: Version: 1 (0x0) Serial Number: 2 (0x2) Signature Algorithm: sha256WithRSAEncryption Issuer: O=xxx, OU=Client 1/emailAddress=xxx, L=xxx, ST=xxx, C=xx, CN=xxx Validity Not Before: Mar 8 15:53:24 2011 GMT Not After : Mar 7 15:53:24 2014 GMT Subject: C=xx, ST=xx, O=xx, OU=xx, CN=client1 Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (2048 bit) ... However, I copied over the cert of another client machine to client1, reconfigure Syslog NG to use it, and it works as well, though IMHO it shouldn't: [root@server1 ssl]# openssl x509 -noout -text < client2_crt.pem Certificate: Data: Version: 1 (0x0) Serial Number: 3 (0x3) Signature Algorithm: sha256WithRSAEncryption Issuer: O=xxx, OU=Client 2/emailAddress=xxx, L=xxx, ST=xxx, C=xx, CN=xxx Validity Not Before: Mar 8 15:54:35 2011 GMT Not After : Mar 7 15:54:35 2014 GMT Subject: C=xx, ST=xxx, O=xxx, OU=Client 2, CN=client2 Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (2048 bit) Modulus (2048 bit): ... Swapping certificates (client2's instead of server's in this case) on the server breaks trust (as expected): Mar 8 18:19:45 client1 syslog-ng[2991]: Syslog connection established; fd='10', server='AF_INET(192.168.42.184:601)', local='AF_INET(0.0.0.0:0)' Mar 8 18:20:13 client1 syslog-ng[2991]: Certificate subject does not match configured hostname; hostname='server', certificate='client2' Mar 8 18:20:13 client1 syslog-ng[2991]: SSL error while writing stream; tls_error='SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed' Mar 8 18:20:13 client1 syslog-ng[2991]: I/O error occurred while writing; fd='10', error='Broken pipe (32)' Doing the same on the client, however, does not. The Syslog NG server does not seem to validate the CN. Is this expected behaviour or a bug? Best regards, Peter.
On Tue, 2011-03-08 at 21:25 +0100, Peter Eckel wrote:
Hi all,
I am currently setting up a remote logging facility for a customer who requires encryption and mutual authentication. After having run Syslog NG over Stunnel for several years, I am looking into the option to do the same withot stunnel, by using Syslog NG with the TLS option. My current version is 3.2.2, installed from the binary provided on the Syslog NG web site, on CentOS 5.5.
My setup on the server side (stripped to the minimum):
source s_remote { tcp( ip( 192.168.42.184 ) port( 601 ) tls( key_file("/opt/syslog-ng/etc/ssl/server_key.pem") cert_file("/opt/syslog-ng/etc/ssl/server_crt.pem") ca_dir( "/opt/syslog-ng/etc/ssl" ) peer_verify( required-trusted ) ) ); };
destination d_remote { file( "/var/log/remote" owner(root) group(root) perm(0600) ); };
log { source(s_remote); destination(d_remote); };
On the client side:
source s_local { internal(); unix-stream( "/dev/log" ); file( "/proc/kmsg" ); };
destination d_remote { tcp( "192.168.42.184" port( 601 ) tls( key_file("/opt/syslog-ng/etc/ssl/client1_key.pem") cert_file("/opt/syslog-ng/etc/ssl/client1_crt.pem") ca_dir( "/opt/syslog-ng/etc/ssl" ) ) ); };
log { source(s_local); destination(d_remote); };
With the appropriate and valid client cert, it works perfrctly:
[root@server1 ssl]# openssl x509 -noout -text < client1_crt.pem Certificate: Data: Version: 1 (0x0) Serial Number: 2 (0x2) Signature Algorithm: sha256WithRSAEncryption Issuer: O=xxx, OU=Client 1/emailAddress=xxx, L=xxx, ST=xxx, C=xx, CN=xxx Validity Not Before: Mar 8 15:53:24 2011 GMT Not After : Mar 7 15:53:24 2014 GMT Subject: C=xx, ST=xx, O=xx, OU=xx, CN=client1 Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (2048 bit) ...
However, I copied over the cert of another client machine to client1, reconfigure Syslog NG to use it, and it works as well, though IMHO it shouldn't:
[root@server1 ssl]# openssl x509 -noout -text < client2_crt.pem Certificate: Data: Version: 1 (0x0) Serial Number: 3 (0x3) Signature Algorithm: sha256WithRSAEncryption Issuer: O=xxx, OU=Client 2/emailAddress=xxx, L=xxx, ST=xxx, C=xx, CN=xxx Validity Not Before: Mar 8 15:54:35 2011 GMT Not After : Mar 7 15:54:35 2014 GMT Subject: C=xx, ST=xxx, O=xxx, OU=Client 2, CN=client2 Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (2048 bit) Modulus (2048 bit): ...
Swapping certificates (client2's instead of server's in this case) on the server breaks trust (as expected):
Mar 8 18:19:45 client1 syslog-ng[2991]: Syslog connection established; fd='10', server='AF_INET(192.168.42.184:601)', local='AF_INET(0.0.0.0:0)' Mar 8 18:20:13 client1 syslog-ng[2991]: Certificate subject does not match configured hostname; hostname='server', certificate='client2' Mar 8 18:20:13 client1 syslog-ng[2991]: SSL error while writing stream; tls_error='SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed' Mar 8 18:20:13 client1 syslog-ng[2991]: I/O error occurred while writing; fd='10', error='Broken pipe (32)'
Doing the same on the client, however, does not. The Syslog NG server does not seem to validate the CN.
I'm not sure I followed you correctly, but the syslog-ng server has no information about the client name, only its IP address, which usually doesn't match anything in the X.509 certificate. (although I know that a subjectAltName can be specified with type iPADDr), but that's usually missing. You could use the trusted_dn() option to specify which client DNs a given server accepts, but that could mean that you need separate ports for each client. What do you expect syslog-ng to behave in this case? -- Bazsi
Hi Baszi,
I'm not sure I followed you correctly, but the syslog-ng server has no information about the client name, only its IP address, which usually doesn't match anything in the X.509 certificate. (although I know that a subjectAltName can be specified with type iPADDr),
that would be one option I had in mind. The other one would be to reverse-lookup the host name and try to match the result against the CN.
but that's usually missing.
Since I create the certificates myself and control all the hosts logging to the central server, I could easily include the IP as well as some alternative form of host name (FQDN vs. simple).
You could use the trusted_dn() option to specify which client DNs a given server accepts, but that could mean that you need separate ports for each client.
You're right, that's not an option.
What do you expect syslog-ng to behave in this case?
As you already suggested: - try to lookup the host name and match the result against the CN or some subjectAltName - try to match the IP against some subjectAltName - fail if the client cannot provide any certificate proving his identity. I just did some tests aiming to prove that stunnel does it exactly like that ... unfortunately this wasn't possible, as stunnel obviously follows the same logic Syslog NG does, and so I didn't prove anything except that my own assumption was wrong. But I still think it would be feasible and a very useful functionality, too. Thank you very much and best regards, Peter.
On Wed, 2011-03-09 at 00:35 +0100, Peter Eckel wrote:
Hi Baszi,
I'm not sure I followed you correctly, but the syslog-ng server has no information about the client name, only its IP address, which usually doesn't match anything in the X.509 certificate. (although I know that a subjectAltName can be specified with type iPADDr),
that would be one option I had in mind. The other one would be to reverse-lookup the host name and try to match the result against the CN.
Well, yes, even though that'd be reliance on DNS, which may (in most cases) may not be trusted.
but that's usually missing.
Since I create the certificates myself and control all the hosts logging to the central server, I could easily include the IP as well as some alternative form of host name (FQDN vs. simple).
You could use the trusted_dn() option to specify which client DNs a given server accepts, but that could mean that you need separate ports for each client.
maybe supporting some kind of templating within trusted_dn()? Like SSLRequire in Apache, as described here: http://httpd.apache.org/docs/2.0/mod/mod_ssl.html The trouble is that syslog-ng already has a templating system, but that only works in message context, whereas we need to evaluate this _before_ accepting a message.
You're right, that's not an option.
What do you expect syslog-ng to behave in this case?
As you already suggested:
- try to lookup the host name and match the result against the CN or some subjectAltName - try to match the IP against some subjectAltName - fail if the client cannot provide any certificate proving his identity.
I just did some tests aiming to prove that stunnel does it exactly like that ... unfortunately this wasn't possible, as stunnel obviously follows the same logic Syslog NG does, and so I didn't prove anything except that my own assumption was wrong. But I still think it would be feasible and a very useful functionality, too.
Generally all SSL enabled servers do it like syslog-ng, at least in my experience.
Thank you very much and best regards,
Since there's a subject validation code in syslog-ng, one would only need to call it in the client case too. See the destination side in modules/afsocket/afsocket.c, function afsocket_dd_tls_verify_callback() That could certainly be applied to the source side too. As I see if that happens some form of refactoring could be applied, since this could become part of the TLSContext proper, rather than having to call it explicitly from each source/destination driver that may make use of SSL. for example a: tls(validate-subject(yes/no)) using defaults as it is now (as a client "yes", as a server "no") would probably work. That would expect the client IP address in the CN or in subjectAltName. Care to give it a spin? -- Bazsi
On Wed, 2011-03-09 at 18:52 +0100, Balazs Scheidler wrote:
On Wed, 2011-03-09 at 00:35 +0100, Peter Eckel wrote:
Hi Baszi,
I'm not sure I followed you correctly, but the syslog-ng server has no information about the client name, only its IP address, which usually doesn't match anything in the X.509 certificate. (although I know that a subjectAltName can be specified with type iPADDr),
that would be one option I had in mind. The other one would be to reverse-lookup the host name and try to match the result against the CN.
Well, yes, even though that'd be reliance on DNS, which may (in most cases) may not be trusted.
well, this is easy to misunderstand what I wrote. I wanted to write, it may not be trusted in most cases.
but that's usually missing.
Since I create the certificates myself and control all the hosts logging to the central server, I could easily include the IP as well as some alternative form of host name (FQDN vs. simple).
You could use the trusted_dn() option to specify which client DNs a given server accepts, but that could mean that you need separate ports for each client.
maybe supporting some kind of templating within trusted_dn()? Like SSLRequire in Apache, as described here:
http://httpd.apache.org/docs/2.0/mod/mod_ssl.html
The trouble is that syslog-ng already has a templating system, but that only works in message context, whereas we need to evaluate this _before_ accepting a message.
You're right, that's not an option.
What do you expect syslog-ng to behave in this case?
As you already suggested:
- try to lookup the host name and match the result against the CN or some subjectAltName - try to match the IP against some subjectAltName - fail if the client cannot provide any certificate proving his identity.
I just did some tests aiming to prove that stunnel does it exactly like that ... unfortunately this wasn't possible, as stunnel obviously follows the same logic Syslog NG does, and so I didn't prove anything except that my own assumption was wrong. But I still think it would be feasible and a very useful functionality, too.
Generally all SSL enabled servers do it like syslog-ng, at least in my experience.
Thank you very much and best regards,
Since there's a subject validation code in syslog-ng, one would only need to call it in the client case too. See the destination side in modules/afsocket/afsocket.c, function afsocket_dd_tls_verify_callback()
That could certainly be applied to the source side too.
As I see if that happens some form of refactoring could be applied, since this could become part of the TLSContext proper, rather than having to call it explicitly from each source/destination driver that may make use of SSL.
for example a:
tls(validate-subject(yes/no))
using defaults as it is now (as a client "yes", as a server "no") would probably work.
That would expect the client IP address in the CN or in subjectAltName.
Care to give it a spin?
-- Bazsi
Hi Baszi,
Well, yes, even though that'd be reliance on DNS, which may (in most cases) may not be trusted.
well, this is easy to misunderstand what I wrote. I wanted to write, it may not be trusted in most cases.
sure. Although I am currently rolling out DNSSEC step-by-step, which means that there is some progress to expect in the near future. But for most current installation you are of course right, it doesn't make much sense to link a certificate to a name that is not guaranteed to be correct at all.
maybe supporting some kind of templating within trusted_dn()? Like SSLRequire in Apache, as described here:
http://httpd.apache.org/docs/2.0/mod/mod_ssl.html
The trouble is that syslog-ng already has a templating system, but that only works in message context, whereas we need to evaluate this _before_ accepting a message.
Seems to be very much effort to me ... one could refer to /etc/hosts based adress resolution (which would work around DNS insecurity), but that's not very scalable as well.
Generally all SSL enabled servers do it like syslog-ng, at least in my experience.
That seems to be the case, yes. Which makes it seem very probable that actually it doesn't make much sense to try to resolve names and match them to the CN/subjectAltName, because if there was a sensible way to do it, someone else would have done it already. Last resort would be the IP address, which works fine in subjectAltName for authenticating the server to the client. Can you think of any reason why that could not be feasible as well?
Since there's a subject validation code in syslog-ng, one would only need to call it in the client case too. See the destination side in modules/afsocket/afsocket.c, function afsocket_dd_tls_verify_callback()
That could certainly be applied to the source side too.
As I see if that happens some form of refactoring could be applied, since this could become part of the TLSContext proper, rather than having to call it explicitly from each source/destination driver that may make use of SSL.
for example a:
tls(validate-subject(yes/no))
using defaults as it is now (as a client "yes", as a server "no") would probably work.
That would expect the client IP address in the CN or in subjectAltName.
Care to give it a spin?
Sure! I am currently including the IP in the subjectAltName for new certificates anyway, so it wouldn't require any change in procedure to test it. Thanks for your feedback! Best regards, Peter.
On Wed, Mar 09, 2011 at 11:57:10PM +0100, Peter Eckel wrote:
Hi Baszi,
Well, yes, even though that'd be reliance on DNS, which may (in most cases) may not be trusted.
well, this is easy to misunderstand what I wrote. I wanted to write, it may not be trusted in most cases.
sure. Although I am currently rolling out DNSSEC step-by-step, which means that there is some progress to expect in the near future. But for most current installation you are of course right, it doesn't make much sense to link a certificate to a name that is not guaranteed to be correct at all.
If DNS may not be trusted, then why do web browsers compare the DNS name with the name in the server certificate, and complain or block the connection if there is a mismatch? This argument here is not right. Sure It's less trusted without DNSSEC but certainly more trusted than just accepting any client certificate your CA issued even if it's for a totally different system (because, for example, it was stolen by an attacker and not yet added to a CRL, etc.)
Generally all SSL enabled servers do it like syslog-ng, at least in my experience.
That seems to be the case, yes. Which makes it seem very probable that actually it doesn't make much sense to try to resolve names and match them to the CN/subjectAltName, because if there was a sensible way to do it, someone else would have done it already.
Last resort would be the IP address, which works fine in subjectAltName for authenticating the server to the client. Can you think of any reason why that could not be feasible as well?
I think it makes more sense to check the DNS because all the machine's IP can reverse-resolve to the machine's primary DNS name, and it avoids hard-coding IPs into client certificates which is presumably undesirable in a large environment of the sort where client certificates would add much value. Regards, Matthew.
Hi, On Wed, 2011-03-09 at 16:49 -0800, Matthew Hall wrote:
On Wed, Mar 09, 2011 at 11:57:10PM +0100, Peter Eckel wrote:
Hi Baszi,
Well, yes, even though that'd be reliance on DNS, which may (in most cases) may not be trusted.
well, this is easy to misunderstand what I wrote. I wanted to write, it may not be trusted in most cases.
sure. Although I am currently rolling out DNSSEC step-by-step, which means that there is some progress to expect in the near future. But for most current installation you are of course right, it doesn't make much sense to link a certificate to a name that is not guaranteed to be correct at all.
If DNS may not be trusted, then why do web browsers compare the DNS name with the name in the server certificate, and complain or block the connection if there is a mismatch? This argument here is not right.
Yes, it is. In the 'client' case (e.g. web browser or syslog-ng running as a client) you know the _name_ of the server since you've entered it in the URL bar (or the config file), so that's authoritive. In the server case, all we have is an IP address, reverse-mapping it is doable, but cannot be trusted.
Sure It's less trusted without DNSSEC but certainly more trusted than just accepting any client certificate your CA issued even if it's for a totally different system (because, for example, it was stolen by an attacker and not yet added to a CRL, etc.)
It creates a false sense of security. The way DNS reverse-resolution works makes it trivial to spoof especially if this is an internet facing listener. The way it currently works, is that if you have a key (any key) it'd work. It doesn't protect against trusted clients exchanging their keys. But the suggestion in my previous email would solve that too.
Generally all SSL enabled servers do it like syslog-ng, at least in my experience.
That seems to be the case, yes. Which makes it seem very probable that actually it doesn't make much sense to try to resolve names and match them to the CN/subjectAltName, because if there was a sensible way to do it, someone else would have done it already.
Last resort would be the IP address, which works fine in subjectAltName for authenticating the server to the client. Can you think of any reason why that could not be feasible as well?
I think it makes more sense to check the DNS because all the machine's IP can reverse-resolve to the machine's primary DNS name, and it avoids hard-coding IPs into client certificates which is presumably undesirable in a large environment of the sort where client certificates would add much value.
You have a point here. I guess doing a reverse and then a forward lookup would not be bad, even though that would mean an extra DNS lookup when the connection is established. And since this is only an _additional_ restriction to having a trusted key, this may be useful. -- Bazsi
You have a point here. I guess doing a reverse and then a forward lookup would not be bad, even though that would mean an extra DNS lookup when the connection is established. And since this is only an _additional_ restriction to having a trusted key, this may be useful.
I don't think the cost of an reverse-then-forward lookup should be a problem. After all, we are using TCP when TLS is involved anyway, so the connections are permanent in most cases and the overhead of two DNS lookups is neglegible. We need an option to make the matching of the double lookup a strict requirement (or do it at all - with DNSSEC, it's not really necessary), though - there are cases where several IP addresses have a PTR to the same host name, so the lookup of the host name would probably return a different IP. I also second Matt's injection that putting the IP address in subjectAltName can be a major maintenance headache in larger environments. My current environment is a rather static and well-defined one, but keeping track of hundreds of hosts with their IPs would be a challenge, apart from the fact that generating certificates automatically would require some additional infrastructure in order to keep the process secure. Regards, Peter.
On Thu, 2011-03-10 at 10:39 +0100, Peter Eckel wrote:
You have a point here. I guess doing a reverse and then a forward lookup would not be bad, even though that would mean an extra DNS lookup when the connection is established. And since this is only an _additional_ restriction to having a trusted key, this may be useful.
I don't think the cost of an reverse-then-forward lookup should be a problem. After all, we are using TCP when TLS is involved anyway, so the connections are permanent in most cases and the overhead of two DNS lookups is neglegible. We need an option to make the matching of the double lookup a strict requirement (or do it at all - with DNSSEC, it's not really necessary), though - there are cases where several IP addresses have a PTR to the same host name, so the lookup of the host name would probably return a different IP.
I also second Matt's injection that putting the IP address in subjectAltName can be a major maintenance headache in larger environments. My current environment is a rather static and well-defined one, but keeping track of hundreds of hosts with their IPs would be a challenge, apart from the fact that generating certificates automatically would require some additional infrastructure in order to keep the process secure.
anyone with the interest of implementing this? -- Bazsi
Balazs Scheidler <bazsi@balabit.hu> writes:
On Thu, 2011-03-10 at 10:39 +0100, Peter Eckel wrote:
You have a point here. I guess doing a reverse and then a forward lookup would not be bad, even though that would mean an extra DNS lookup when the connection is established. And since this is only an _additional_ restriction to having a trusted key, this may be useful.
I don't think the cost of an reverse-then-forward lookup should be a problem. After all, we are using TCP when TLS is involved anyway, so the connections are permanent in most cases and the overhead of two DNS lookups is neglegible. We need an option to make the matching of the double lookup a strict requirement (or do it at all - with DNSSEC, it's not really necessary), though - there are cases where several IP addresses have a PTR to the same host name, so the lookup of the host name would probably return a different IP.
I also second Matt's injection that putting the IP address in subjectAltName can be a major maintenance headache in larger environments. My current environment is a rather static and well-defined one, but keeping track of hundreds of hosts with their IPs would be a challenge, apart from the fact that generating certificates automatically would require some additional infrastructure in order to keep the process secure.
anyone with the interest of implementing this?
*raises a hand* Unless someone else beats me to it, of course. I'll see if I can squeeze this into my afternoon. -- |8]
participants (4)
-
Balazs Scheidler
-
Gergely Nagy
-
Matthew Hall
-
Peter Eckel