fwm
Full Member
Posts: 105
|
Post by fwm on Jun 1, 2018 14:16:54 GMT -5
Even better! It didn't occur to me until now that going with one wrapper for both non-TLS and TLS would actually be essential, but I've just realised that it is, for example, when I connect to my mail server, some ports are "hard-wired" to use SSL/TLS and don't accept non-secure connections, but then other ports are set up for optional STARTTLS, where the connection begins as a non-secure conversation but then gets upgraded to secure, whilst remaining open. This wouldn't have been possible if we had to start the non-secure conversation with mesock and then change over to use your (fantastic!) DLL to take over the secure parts of the conversation. Well, I say it wouldn't have been possible... maybe there would have been a way to do it but I'm sure it would have been very messy to do. Anyway, I have just written a test program that uses the new wrapper DLL to have three SMTP conversations, one all non-secure, one all secure, and the third was a STARTTLS conversation where the upgrade took place "on the fly". They all work successfully. Yes, the timeout was just something I thought might be useful to put in so it's there as it was in mesock Keith.
|
|
|
Post by Chris Iverson on Jun 1, 2018 14:57:53 GMT -5
Changed my mind about the separate call, it turned out to be easy enough to integrate as-is. Modifications made, and published to GitHub.
New parameter at the end of the Connect call - msTimeout. Requested timeout in milliseconds.
Specify a timeout of 0 to use the OS default value.
The one thing I will note is that this timeout will be per connection attempt, not overall. My code will only make once connection attempt per possible connection interface, but it is possible for the system to return multiple interfaces, and my code will try them all before giving up entirely. Each one of those attempts will be made with the specified timeout value.
If that becomes burdensome, I can modify the code to lower the connection timeout per each attempt. The only reason I currently haven't is that the system returns the possible interfaces in a linked list, and doesn't specify how many possibilities there are. You just go down the list until there is no more next one. Determining how many there are would require traversing the list twice(once for the count; again to test connecting to them all with modified timeout value).
You can see how the timeout works by changing the timeout value from a 0 to a 1 in my test code. Since it's physically impossible to make any internet connection in less than a millisecond, you'll see that it returns connection failed, and that the error value is WSAETIMEDOUT.
|
|
fwm
Full Member
Posts: 105
|
Post by fwm on Jun 1, 2018 19:33:50 GMT -5
Hi Chris, Thanks for adding the timeout - I can't get it to work as expected. When the timeout is 0, everything is fine but if I try, say 1000 for 1 second, I get a connection error 10060. In fact if I try anything above 0, it produces that connection error. There doesn't appear to be a delay of the timeout set while it tries to connect, it just shows the error straight away. I was trying to work out why in the C++ code. I think it might be something to do with the blocking/non-blocking, but couldn't quite put my finger on it... Keith.
|
|
|
Post by Chris Iverson on Jun 1, 2018 21:54:55 GMT -5
You're absolutely right. Just a small oversight on my part. There's two parts in the code that call ioctlsocket(), which(among other things) is what controls the blocking/non-blocking setting for sockets. The third argument to that function is supposed to be a pointer to a value that's treated as a boolean in this case, with zero being "off/blocking", and anything else being "on/non-blocking". Here, you can see the first instance of that, and you can see I had it right: I create a variable, set the variable to the value I want(TRUE), and pass a pointer to that variable(&block). Later on, at the second call to ioctlsocket(), I goof up. Here, you can see the mistake. I create the variable, set it to the intended value(FALSE)... and then pass the value FALSE in directly to the function call, instead of a pointer to the variable I just made. Function tries to dereference the value I passed in as a pointer, fails, and bombs out. Here, you can see the one line fix.Oops. I didn't notice this before because I didn't test with a proper timeout, and I didn't test the returned error value to check to make sure that it was actually returning the timed-out error, and not the "hey you screwed up a parameter" error.
|
|
fwm
Full Member
Posts: 105
|
Post by fwm on Jun 2, 2018 4:01:33 GMT -5
Hi Chris, That all makes sense and your explanation allowed me to understand exactly what was going on. I've downloaded and compiled the DLL again, tested, and of course it works flawlessly as far as I can tell. As I already said, I have it working for an SMTP connection in non-secure, secure and non-secure upgraded to secure (STARTTLS) conversations, and all are working just as expected. For me, that's exactly what I was looking to be able to do - but now I'm interested to hear what else you plan on adding in Do you think that adding certificate exchange and verification is possible, for example? I'm excited to hear what other LB users might be able to use this for... Keith.
|
|
fwm
Full Member
Posts: 105
|
Post by fwm on Jun 3, 2018 16:35:09 GMT -5
Hi Chris,
I think there may still be an issue with the timeout. If I run your test.bas program and change the timeout from 0 to, say, 1000, I am still getting an error. I was testing it myself with local connections on my local network and getting a positive instant connection, but over the internet I am unable to connect. Not sure if you get the same result, or if it is working for you and I am just doing something daft - which is very possible!
Let me know what you think...
Keith.
|
|
|
Post by Chris Iverson on Jun 4, 2018 9:46:15 GMT -5
Hmm, I think I see that, too. Even though the error it returns is the timed out error(this time), it seems to return far too fast. And yes, the next step I intend to implement is validating the connection(which should be fairly simple, if I just leave things at the default). EDIT: Figured it out. I was misusing the TIMEVAL structure. In the select() call, the parameter you pass to indicate how much time you want to wait for before timing out is not just milliseconds. It's a structure like this: struct TIMEVAL,_ tv_sec as long,_ tv_usec as long TIMEVAL.tv_sec is the number of seconds you want to wait, and TIMEVAL.tv_usec is the number of milliseconds. The select() call combines the two and waits for that amount of time. I thought that just taking the passed-in milliseconds value and setting TIMEVAL.tv_usec to that would be fine. Turns out, that doesn't quite seem to be the case. It seems that if TIMEVAL.tv_sec is zero, it assumes the whole thing is zero, and doesn't wait at all; instead, it immediately returns. This is documented behavior(mostly). In the MSDN page for the select() call here, they document that if you don't pass a TIMEVAL structure at all, and pass NULL, select() waits/blocks indefinitely, until at least one of the specified criteria is met. If you pass a TIMEVAL structure of {0, 0}, it returns immediately, acting as a poll of available sockets, and not waiting for one to become available. It seems that leaving tv_sec as zero is triggering this second part. So instead, I divide the passed-in milliseconds value into full seconds and the remaining part of milliseconds. That seems to work now.
|
|
fwm
Full Member
Posts: 105
|
Post by fwm on Jun 4, 2018 11:50:21 GMT -5
Hi Chris, Ah - I had a look at the code and wondered if it may have been something to do with the TIMEVAL - I was reading up on it and SELECT() on MSDN but couldn't quite get my head around it. It's great that you spotted the issue - I'll compile the latest code and give it a go Keith
|
|
fwm
Full Member
Posts: 105
|
Post by fwm on Jun 4, 2018 12:18:10 GMT -5
Yes that definitely seems to have sorted it now Thank you for your devotion to this! I will look forward to seeing the progress of implementing the validation Keith
|
|
|
Post by Brandon Parker on Jun 4, 2018 21:58:33 GMT -5
Chris, Just an FYI.... The timeval.tv_usec member is actually microseconds not milliseconds. There's a fair bit of difference between the two. Also, another idea to provide an overall timeout might be to set a variable (using your preferred method for setting a timestamp) prior to entering into the For Loop and check it at each iteration; exit the For Loop if the time has elapsed. So you "could" provide the user with an overall timeout as well as an individual one per interface connection attempt. That is if I'm understanding the discussion above. {:0) Brandon Parker
|
|
fwm
Full Member
Posts: 105
|
Post by fwm on Jun 5, 2018 0:08:09 GMT -5
Hi Chris, I've just noticed that in the PerformClientHandshake DLL call, a URL is passed into it, however if I change this to another URL, the connection is still successful. Looking at the code on GitHub and the MSDN information about InitializeSecurityContext, am I right in thinking this URL is for future use when the time comes to validate the connection, and in the current form it doesn't matter what this is set to? Or am I on the wrong track? Hope you don't mind me asking Keith.
|
|
|
Post by Chris Iverson on Jun 5, 2018 9:45:21 GMT -5
Brandon Parker: ... wow, I completely missed that. Yeah, that would probably be why it didn't seem to be working properly. I would still need to adjust each individual call's timeout to a smaller value, but that might work. fwm: That's exactly right, and I don't mind you asking at all. I'll answer any questions you have about this stuff; I enjoy it! It's actually used by BOTH ends of the connection. Client-side, so it knows what domain you're trying to connect to, and can verify that the server sent the proper certificate for that domain, AND server-side, so it knows which certificate it needs to send. With how limited IPv4 addresses have recently become, there needed to be a way to host multiple domains on one IP address. While this was already supported in HTTP, with the Host: header that's required for HTTP 1.1, the problem is, the server doesn't see that Host: header until after the TLS negotiation. If you have multiple sites hosted on the same server/address, and someone tries to come in via HTTPS to browse, the server wouldn't know which cert to present to the client. That led to the Server Name Indication extension to TLS, which modified the TLS protocol to send the requested server name as part of the ClientHello message, so the server would know which cert to respond with. Heck, even my own server, the one that hosts chrisiverson.net, will respond with different certs depending on which address you use to access it. chrisiverson.net and thearcaneanomaly.net both point to the same server, but you will get a valid, corresponding cert depending on how you access it. And if you go to the IP directly, https://24.14.7.47(you have to go to HTTPS explicitly), you'll get a third certificate. It won't be a valid one, because 1) you didn't use the right hostname, so that check fails, and 2) that certificate won't be trusted by your system, anyway. It's a cert from an internal CA I created myself for testing purposes, and is only trusted by my systems. (Since that's an internal page, anyway, and trying to access it from anywhere besides my internal network will result in HTTP 403 Forbidden, trust isn't an issue. I know what certs MY CA has issued.)
|
|
fwm
Full Member
Posts: 105
|
Post by fwm on Jun 5, 2018 10:20:56 GMT -5
Chris,
Of course - that makes complete sense! I have several servers and they all use SNI, and I was aware of having to send the URL in the ClientHello message but the penny just hadn't dropped for me... until I read your post above. Yes, now it seems obvious so thanks for helping me understand that.
Keith.
|
|
|
Post by Chris Iverson on Jun 5, 2018 16:22:30 GMT -5
Updated to now do server validation by default(as that is default in SChannel).
Begin TLS negotiation with BeginTLSClient() instead of BeginTLSClientNoValidation() to handle it.
You can test various methods of TLS failure by testing against the test valid and invalid hosts provided by badssl.com. Check their website to see various types of failures, as well as the hostnames tied to those failures, and try to connect to those sites in LB.
You'll get an error code back. Different error codes for different types of issues, such as untrusted root, expired cert, invalid host name, etc.
|
|
fwm
Full Member
Posts: 105
|
Post by fwm on Jun 6, 2018 10:11:42 GMT -5
Hi Chris,
I've given the new version a try and it all seems to work for me - the "no validation" routine works exactly as it did before, and the "with validation" seems spot on.
Great job again!
What else are you thinking of adding in?
Keith.
|
|