Add an option to test links for an HTTPS alternative

[MaximeKjaer]

Lately, there's been a big push for wider HTTPS support. Following that trend, I think it could be a good idea to have an additional option that tests if HTTP links have an HTTPS alternative.

This option wouldn't be as harsh as enforce_https, which fails all insecure links. Obviously, from a security standpoint, if you're on HTTPS, then all your links should be HTTPS too. But that may be a bit of a high bar for many people, as it requires major changes to their sites. Instead, a more flexible option would be to only fail insecure links that actually have a secure alternative. This option would heighten the security of many sites considerably, especially considering the relatively low commitment involved.

I'm filing this as an issue, mainly because I wanted to gauge the interest in a such feature — do you think that this is within the scope of this project?

[gjtorikian]

Instead, a more flexible option would be to only fail insecure links that actually have a secure alternative.

Interesting. So you're proposing, say, if you have a link pointing to http://www.wikipedia.com, it should actually point to https://www.wikipedia.com? I guess the question is if it's possible to detect whether a URL has a secure alternative or not.

[MaximeKjaer]

Yes, exactly.

It's actually not as straightforward as sending an HTTPS request and expecting a 200 in response, because some websites redirect their HTTPS version to their HTTP version (like https://www.wired.com for instance):

And others have a redirect but keep the HTTPS (like https://facebook.com, which redirects to https://www.facebook.com)

This means that if you want to know what's on the end of the HTTPS version of the link, you have to sit through the redirects until you land on either:

• A connection with no response (not a secure alternative &mdash we can only presume that this means that the server isn't responding to HTTPS requests)
• An invalid certificate (self-signed or otherwise; I imagine that this would be a part of the request API. This wouldn't be a secure alternative)
• The http:// scheme (no matter the response code, if it starts loading HTTP, it's not a secure alternative)
• The https:// with a 2xx response code (which should be the only acceptable response, perhaps along with some odd exceptions like 418)

I think it should be feasible using only the following information:

• Whether the certificate is OK (if not, then it's not a valid HTTPS link)
• The scheme of our request (if we have the link, we know if it's asking for http:// or https://)
• The server's response headers

Here's how I'm thinking it should go about it, in pseudocode:

def exists_https_alternative(link)
    link = change the link from "http://example.com" to "https://example.com"
    follow the link using a HEAD request

    while 3xx response (and maybe while we're below a max number of redirects)
        if the redirect location starts with http://
            return false
        link = redirect location
        follow the link using a HEAD request

    return (link starts with https && returns a 2xx or 418 or something else valid)

Something like that. I haven't written it in, but at any point, the certificate is invalid or the request isn't met with a response, it should return false.

I think that should work.

[gjtorikian]

Note to self, there's a totally separate check_img_http function too, which is confusing.

[fulldecent]

This issue is proposing to add recommendations when checking HTTP links.

We saw you have a link to HTTP but we recommend that you upgrade to HTTPS and we have checked real quick to make sure that link is live.

Such recommendations are beyond the scope of a "checker" and beyond the scope of this project. Every issue here could also implement recommendations and these could add significantly more complexity to the project.

[fulldecent]

I recommend closing this issue based on proposed project scope at #422. This is still a great idea and if people are interested it could be implemented as a third-party pluggin and mentioned from the wiki here.

[fulldecent]

Related issue: #427

---

P.S. I have working process that accomplishes what you are looking for in this wiki:

https://github.com/fulldecent/html-website-template/wiki/Upgrade-http-links-to-https

[MaximeKjaer]

Fantastic! It does make more sense as a third-party plugin now that there is an interface for those, so I'll close this issue. Thank you for your great work!