SSL deferred handshake

[ma261065]

I'm trying to create an SSL connection from an ESP32 to a server using a non-blocking socket.

Maybe I'm misunderstanding something, but I can't get it to work with deferred handshake...

conn = socket.socket()
addr = socket.getaddrinfo(host, port)[0][-1]
conn.connect(addr)
conn.setblocking(False)
sock = ssl.wrap_socket(conn, server_hostname=host, do_handshake=False)
num = sock.write(bytes(f"GET {path} HTTP/1.1\r\nHost: {host}\r\n\r\n", "utf8"))

Based on the documentation, I expected the SSL handshake to happen when I called the sock.write(), but I keep getting num as None when I run that code.

If I instead change do_handshake=True, then the SSL negotiation happens when I do the wrap_socket() and the sock.write() succeeds...

Am I missing something?

[sophiedegran]

Hi
Did you look at http_client_ssl.py on micropython/examples/network?

[ma261065]

Thanks, but I'm not sure how that helps, as it is pretty much identical to my code snippet above, does not use non-blocking sockets and doesn't use deferred handshakes.

[sophiedegran]

Several examples with no handshakes at micropython/tests/
/net_hosted/
/net_inet/
/multi_net/

[dpgeorge]

The asyncio code supports non-blocking SSL connections: https://github.com/micropython/micropython/blob/master/extmod/asyncio/stream.py#L99-L123

You might need to set the socket to non blocking before connecting the socket. Also be aware that a read/write to a non-blocking SSL socket may return without finishing, by virtue of being non-blocking. So keep reading/writing until it does go through.

[ma261065]

Thanks, but setting the socket to non-blocking before connecting the socket results in a OSError: [Errno 119] EINPROGRESS

So keep reading/writing until it does go through

For read that makes sense.

For write, I can't keep writing the same data to the socket or it will get sent multiple times.

I didn't show the next part of my code in the snippet above where I do in fact keep doing a sock.readline() in a loop until it doesn't get a None result.

But if I defer the SSL handshake (do_handshake=False), it never gets any data back from the server. If I use do_handshake=True in the wrap_socket, it works as expected.

[dpgeorge]

setting the socket to non-blocking before connecting the socket results in a OSError: [Errno 119] EINPROGRESS

That's actually the correct behaviour, because it's a non-blocking connect the call to s.connect() will return immediately and the connection will happen in the background.

For write, I can't keep writing the same data to the socket or it will get sent multiple times.

Yes, actually, that's what you need to do. The return value tells you how much was written, and you must try to write the remaining data again.

Here's some non-blocking SSL code that works for me (when run on the unix port of MicroPython):

from errno import EINPROGRESS
import select
import socket
import ssl


def main(host, path):
    ai = socket.getaddrinfo(host, 443)[0]
    addr = ai[-1]
    print("Connect address:", addr)

    s = socket.socket(ai[0], ai[1], ai[2])
    s.setblocking(False)
    try:
        s.connect(addr)
    except OSError as er:
        if er.errno != EINPROGRESS:
            raise er

    ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
    s = ctx.wrap_socket(s, server_hostname=host, do_handshake_on_connect=False)
    s.setblocking(False)

    poller = select.poll()
    poller.register(s, select.POLLOUT)
    data = b"GET %s HTTP/1.0\r\nHost: %s\r\n\r\n" % (path, host)
    while data:
        poller.poll()
        n = s.write(data)
        print("wrote", n)
        if n is not None:
            data = data[n:]

    poller.register(s, select.POLLIN)
    poller.poll()

    print(s.read(4096))

    s.close()


main("micropython.org", "/ks/test.html")

Note that you can still connect in blocking mode if you don't want to handle the EINPROGRESS error, and then set the socket to non-blocking after it's connected. But, that connect() may then block for some time. It depends on your application as to whether you want to block or not on the connect.

[ma261065]

Thank you so much! That works perfectly for my application.

One question - does closing the wrapped SSL socket also close the underlying socket? Or is extra cleanup required there?

[dpgeorge]

does closing the wrapped SSL socket also close the underlying socket?

Yes.