Lookup additional LDAP user info

[manics]

Adds a new property user_info_attributes that lists additional LDAP fields to lookup and return in auth_state for a valid user.

Example use: Run a containerised singleuser server as the numeric user ID from LDAP so that files created on a shared external mount have the expected owner ID.

[meeseeksmachine]

This pull request has been mentioned on Jupyter Community Forum. There might be relevant details there:

https://discourse.jupyter.org/t/starting-single-user-notebook-with-our-custom-ldap-docker-image/881/4

[pricezt]

Just FYI, I made it here from jupyterhub/kubespawner#298 and this is exactly what we needed for deployment in OpenShift

[marcusianlevine]

@minrk any possibility of this getting reviewed/merged at this point?

The community has worked around it up to this point by copy-pasting a customized Authenticator class, but that is obviously not sustainable

[marcusianlevine]

@yuvipanda @dhirschfeld anyone maintaining this repo? How can we get this moved forward?

Seems like a simple feature, and a copy-pasted version of this code is already used in production by many in the community

[dhirschfeld]

I have literally zero time to work on this repo and although I have commit rights I don't really consider myself a maintainer.

I could merge this but because this repo doesn't have any tests it might break for other people and I wouldn't have the time to fix it (except possibly revert). Also, unlike most users I'm on Windows so I wouldn't even be able to test locally against the setup most people have

So, unfortunately development is at a bit of a standstill and if people need additional features they have to effectively deploy from a fork which isn't an ideal place to be.

If someone wanted to figure out and contribute a way to test existing/new features I would be ok stepping up to merge new PRs but failing that I probably won't do so until I can carve out time to look at this properly - which won't be in the immediate future.

[marcusianlevine]

Thanks for clarifying the situation Dave

Running from a fork is causing my company some headaches - I might be able to negotiate time to contribute a CI setup for this repo

It looks like the main JupyterHub repo uses Travis - I could set up something similar and write some simple tests for the existing functionality

[marcusianlevine]

@dhirschfeld I opened a PR with a Travis lint + test setup and managed to get 80% coverage with some simple tests based on the auth tests in the main JupyterHub repo: #134

If we were to get that reviewed and merged, and then I add a test case for the feature in this PR, would you be comfortable merging it?

[manics]

@marcusianlevine Thanks for #134 ! I'll look into rebasing this PR and adding a test, I'll ping you for help if I can't figure out rroemhild/test-openldap

[manics]

I've merged in master and added a test for this feature.

[manics]

No one disagreed so I've renamed user_info_attributes to auth_state_attributes as suggested by @dhirschfeld

[dhirschfeld]

@manics - I'll take a proper look Monday 12th. FWIW it looks good to me so I've "approved" the changes in case anyone wants to merge before then...

[marcusianlevine]

@dhirschfeld think we could cut a new release once we get this merged?

[CraigInches]

Any update on when this might be merged?

[ramkrishnan8994]

@manics - Is it possible to please share a version of ldapauthenticator.py with these changes for the one used for jupyterhub v0.7.0
This is the file used in v0.7.0 -

`import re

from jupyterhub.auth import Authenticator
import ldap3
from ldap3.utils.conv import escape_filter_chars
from tornado import gen
from traitlets import Unicode, Int, Bool, List, Union


class LDAPAuthenticator(Authenticator):
    server_address = Unicode(
        config=True,
        help="""
        Address of the LDAP server to contact.

        Could be an IP address or hostname.
        """
    )
    server_port = Int(
        config=True,
        help="""
        Port on which to contact the LDAP server.

        Defaults to `636` if `use_ssl` is set, `389` otherwise.
        """
    )

    def _server_port_default(self):
        if self.use_ssl:
            return 636  # default SSL port for LDAP
        else:
            return 389  # default plaintext port for LDAP

    use_ssl = Bool(
        False,
        config=True,
        help="""
        Use SSL to communicate with the LDAP server.

        Deprecated in version 3 of LDAP. Your LDAP server must be configured to support this, however.
        """
    )

    bind_dn_template = Union(
        [List(),Unicode()],
        config=True,
        help="""
        Template from which to construct the full dn
        when authenticating to LDAP. {username} is replaced
        with the actual username used to log in.

        If your LDAP is set in such a way that the userdn can not
        be formed from a template, but must be looked up with an attribute
        (such as uid or sAMAccountName), please see `lookup_dn`. It might
        be particularly relevant for ActiveDirectory installs.

        Unicode Example:
            uid={username},ou=people,dc=wikimedia,dc=org

        List Example:
            [
            	uid={username},ou=people,dc=wikimedia,dc=org,
            	uid={username},ou=Developers,dc=wikimedia,dc=org
        	]
        """
    )

    allowed_groups = List(
        config=True,
        allow_none=True,
        default=None,
        help="""
        List of LDAP group DNs that users could be members of to be granted access.

        If a user is in any one of the listed groups, then that user is granted access.
        Membership is tested by fetching info about each group and looking for the User's
        dn to be a value of one of `member` or `uniqueMember`, *or* if the username being
        used to log in with is value of the `uid`.

        Set to an empty list or None to allow all users that have an LDAP account to log in,
        without performing any group membership checks.
        """
    )

    # FIXME: Use something other than this? THIS IS LAME, akin to websites restricting things you
    # can use in usernames / passwords to protect from SQL injection!
    valid_username_regex = Unicode(
        r'^[a-z][.a-z0-9_-]*$',
        config=True,
        help="""
        Regex for validating usernames - those that do not match this regex will be rejected.

        This is primarily used as a measure against LDAP injection, which has fatal security
        considerations. The default works for most LDAP installations, but some users might need
        to modify it to fit their custom installs. If you are modifying it, be sure to understand
        the implications of allowing additional characters in usernames and what that means for
        LDAP injection issues. See https://www.owasp.org/index.php/LDAP_injection for an overview
        of LDAP injection.
        """
    )

    lookup_dn = Bool(
        False,
        config=True,
        help="""
        Form user's DN by looking up an entry from directory

        By default, LDAPAuthenticator finds the user's DN by using `bind_dn_template`.
        However, in some installations, the user's DN does not contain the username, and
        hence needs to be looked up. You can set this to True and then use `user_search_base`
        and `user_attribute` to accomplish this.
        """
    )

    user_search_base = Unicode(
        config=True,
        default=None,
        allow_none=True,
        help="""
        Base for looking up user accounts in the directory, if `lookup_dn` is set to True.

        LDAPAuthenticator will search all objects matching under this base where the `user_attribute`
        is set to the current username to form the userdn.

        For example, if all users objects existed under the base ou=people,dc=wikimedia,dc=org, and
        the username users use is set with the attribute `uid`, you can use the following config:

        ```
        c.LDAPAuthenticator.lookup_dn = True
        c.LDAPAuthenticator.lookup_dn_search_filter = '({login_attr}={login})'
        c.LDAPAuthenticator.lookup_dn_search_user = 'ldap_search_user_technical_account'
        c.LDAPAuthenticator.lookup_dn_search_password = 'secret'
        c.LDAPAuthenticator.user_search_base = 'ou=people,dc=wikimedia,dc=org'
        c.LDAPAuthenticator.user_attribute = 'sAMAccountName'
        c.LDAPAuthenticator.lookup_dn_user_dn_attribute = 'cn'
        ```
        """
    )

    user_attribute = Unicode(
        config=True,
        default=None,
        allow_none=True,
        help="""
        Attribute containing user's name, if `lookup_dn` is set to True.

        See `user_search_base` for info on how this attribute is used.

        For most LDAP servers, this is uid.  For Active Directory, it is
        sAMAccountName.
        """
    )

    lookup_dn_search_filter = Unicode(
        config=True,
        default_value='({login_attr}={login})',
        allow_none=True,
        help="""
        How to query LDAP for user name lookup, if `lookup_dn` is set to True.
        """
    )

    lookup_dn_search_user = Unicode(
        config=True,
        default_value=None,
        allow_none=True,
        help="""
        Technical account for user lookup, if `lookup_dn` is set to True.

        If both lookup_dn_search_user and lookup_dn_search_password are None, then anonymous LDAP query will be done.
        """
    )

    lookup_dn_search_password = Unicode(
        config=True,
        default_value=None,
        allow_none=True,
        help="""
        Technical account for user lookup, if `lookup_dn` is set to True.
        """
    )

    lookup_dn_user_dn_attribute = Unicode(
        config=True,
        default_value=None,
        allow_none=True,
        help="""
        Attribute containing user's name needed for  building DN string, if `lookup_dn` is set to True.

        See `user_search_base` for info on how this attribute is used.

        For most LDAP servers, this is username.  For Active Directory, it is cn.
        """
    )

    escape_userdn = Bool(
        False,
        config=True,
        help="""
        If set to True, escape special chars in userdn when authenticating in LDAP.

        On some LDAP servers, when userdn contains chars like '(', ')', '\' authentication may fail when those chars
        are not escaped.
        """
    )

    def resolve_username(self, username_supplied_by_user):
        if self.lookup_dn:
            server = ldap3.Server(
                self.server_address,
                port=self.server_port,
                use_ssl=self.use_ssl
            )

            search_filter = self.lookup_dn_search_filter.format(
                login_attr=self.user_attribute,
                login=username_supplied_by_user
            )
            self.log.debug(
                "Looking up user with search_base={search_base}, search_filter='{search_filter}', attributes={attributes}".format(
                    search_base=self.user_search_base,
                    search_filter=search_filter,
                    attributes=self.user_attribute
                )
            )

            conn = ldap3.Connection(server, user=self.escape_userdn_if_needed(self.lookup_dn_search_user), password=self.lookup_dn_search_password)
            is_bound = conn.bind()
            if not is_bound:
                self.log.warn("Can't connect to LDAP")
                return None

            conn.search(
                search_base=self.user_search_base,
                search_scope=ldap3.SUBTREE,
                search_filter=search_filter,
                attributes=[self.lookup_dn_user_dn_attribute]
            )

            if len(conn.response) == 0 or 'attributes' not in conn.response[0].keys():
                self.log.warn('username:%s No such user entry found when looking up with attribute %s', username_supplied_by_user,
                              self.user_attribute)
                return None
            return conn.response[0]['attributes'][self.lookup_dn_user_dn_attribute]
        else:
            return username_supplied_by_user

    def escape_userdn_if_needed(self, userdn):
        if self.escape_userdn:
            return escape_filter_chars(userdn)
        else:
            return userdn

    search_filter = Unicode(
        config=True,
        help="LDAP3 Search Filter whose results are allowed access"
    )

    attributes = List(
        config=True,
        help="List of attributes to be searched"
    )


    @gen.coroutine
    def authenticate(self, handler, data):
        username = data['username']
        password = data['password']
        # Get LDAP Connection
        def getConnection(userdn, username, password):
            server = ldap3.Server(
                self.server_address,
                port=self.server_port,
                use_ssl=self.use_ssl
            )
            self.log.debug('Attempting to bind {username} with {userdn}'.format(
                    username=username,
                    userdn=userdn
            ))
            conn = ldap3.Connection(
                server,
                user=self.escape_userdn_if_needed(userdn),
                password=password,
                auto_bind=self.use_ssl and ldap3.AUTO_BIND_TLS_BEFORE_BIND or ldap3.AUTO_BIND_NO_TLS,
            )
            return conn

        # Protect against invalid usernames as well as LDAP injection attacks
        if not re.match(self.valid_username_regex, username):
            self.log.warn('username:%s Illegal characters in username, must match regex %s', username, self.valid_username_regex)
            return None

        # No empty passwords!
        if password is None or password.strip() == '':
            self.log.warn('username:%s Login denied for blank password', username)
            return None

        isBound = False
        self.log.debug("TYPE= '%s'",isinstance(self.bind_dn_template, list))

        resolved_username = self.resolve_username(username)
        if resolved_username is None:
            return None

        if self.lookup_dn:
            if str(self.lookup_dn_user_dn_attribute).upper() == 'CN':
                # Only escape commas if the lookup attribute is CN
                resolved_username = re.subn(r"([^\\]),", r"\1\,", resolved_username)[0]

        bind_dn_template = self.bind_dn_template
        if isinstance(bind_dn_template, str):
            # bind_dn_template should be of type List[str]
            bind_dn_template = [bind_dn_template]

        for dn in bind_dn_template:
            userdn = dn.format(username=resolved_username)
            msg = 'Status of user bind {username} with {userdn} : {isBound}'
            try:
                conn = getConnection(userdn, username, password)
            except ldap3.core.exceptions.LDAPBindError as exc:
                isBound = False
                msg += '\n{exc_type}: {exc_msg}'.format(
                    exc_type=exc.__class__.__name__,
                    exc_msg=exc.args[0] if exc.args else ''
                )
            else:
                isBound = conn.bind()
            msg = msg.format(
                username=username,
                userdn=userdn,
                isBound=isBound
            )
            self.log.debug(msg)
            if isBound:
                break

        if isBound:
            if self.allowed_groups:
                self.log.debug('username:%s Using dn %s', username, userdn)
                for group in self.allowed_groups:
                    groupfilter = (
                        '(|'
                        '(member={userdn})'
                        '(uniqueMember={userdn})'
                        '(memberUid={uid})'
                        ')'
                    ).format(userdn=escape_filter_chars(userdn), uid=escape_filter_chars(username))
                    groupattributes = ['member', 'uniqueMember', 'memberUid']
                    if conn.search(
                        group,
                        search_scope=ldap3.BASE,
                        search_filter=groupfilter,
                        attributes=groupattributes
                    ):
                        return username
                # If we reach here, then none of the groups matched
                self.log.warn('username:%s User not in any of the allowed groups', username)
                return None
            elif self.search_filter:
                conn.search(
                    search_base=self.user_search_base,
                    search_scope=ldap3.SUBTREE,
                    search_filter=self.search_filter.format(userattr=self.user_attribute,username=username),
                    attributes=self.attributes
                )
                if len(conn.response) == 0:
                    self.log.warn('User with {userattr}={username} not found in directory'.format(
                        userattr=self.user_attribute, username=username))
                    return None
                elif len(conn.response) > 1:
                    self.log.warn('User with {userattr}={username} found more than {len}-fold in directory'.format(
                        userattr=self.user_attribute, username=username, len=len(conn.response)))
                    return None
                return username
            else:
                return username
        else:
            self.log.warn('Invalid password for user {username}'.format(
                username=username,
            ))
            return None


if __name__ == "__main__":
    import getpass
    c = LDAPAuthenticator()
    c.server_address = "ldap.organisation.org"
    c.server_port = 636
    c.bind_dn_template = "uid={username},ou=people,dc=organisation,dc=org"
    c.user_attribute = 'uid'
    c.user_search_base = 'ou=people,dc=organisation,dc=org'
    c.attributes = ['uid','cn','mail','ou','o']
    # The following is an example of a search_filter which is build on LDAP AND and OR operations
    # here in this example as a combination of the LDAP attributes 'ou', 'mail' and 'uid'
    sf = "(&(o={o})(ou={ou}))".format(o='yourOrganisation',ou='yourOrganisationalUnit')
    sf += "(&(o={o})(mail={mail}))".format(o='yourOrganisation',mail='yourMailAddress')
    c.search_filter = "(&({{userattr}}={{username}})(|{}))".format(sf)
    username = input('Username: ')
    passwd = getpass.getpass()
    data = dict(username=username,password=passwd)
    rs=c.authenticate(None,data)
    print(rs.result())
` 

[manics]

@ramkrishnan8994 I guess this is related to jupyterhub/zero-to-jupyterhub-k8s#1402 ? Have you tried using this version of LDAPAuthenticator with the old Z2JH chart? If it doesn't work you'll either have to port these changes, or upgrade your Z2JH version.

[ramkrishnan8994]

@ramkrishnan8994 I guess this is related to jupyterhub/zero-to-jupyterhub-k8s#1402 ? Have you tried using this version of LDAPAuthenticator with the old Z2JH chart? If it doesn't work you'll either have to port these changes, or upgrade your Z2JH version.

That issue has been solved. It isn't exactly related. The only relation is that there are some dependency issues that are stopping us from deploying the new version of Jhub.

I'm trying to use the ldapauthenticator.py from the MR to v0.7.0. But that has its own challenges.
There is no option to provide Image pull secrets for Hub. It is only available in singleuser.
Also. I'm not exactly sure if the new version of ldapauthenticator.py will work with the older version Jupyterhub. There are a number of changes to the file.

Also the documentation in - https://gist.github.com/manics/c4bcf53a210d444db9e64db7673e8580
has to be updated. The variable user_info_attributes does not exist anymore but is there in the wiki. Also please update anything else if needed. The details are good but needs to updated. Request you to do it soon please

[kfox1111]

There any reason not to merge this? We would really like to use zero to k8s with ldap based uids and a shared home. This seems to be a prereq?

[manics]

@jupyterhub/jupyterhubteam can we decide how to take this forward? It seems to be generally useful.

[titansmc]

Hi guys,
There has been any progress on this? This will definitely help our organization to move the deployment to production.
Cheers.

[yuvipanda]

I think we need more maintainers for this repo! I don't know who has LDAP experience in the Jupyter world...

I've just merged this. Would a release be useful?

[marcusianlevine]

Thanks Yuvi, a release would be great! 🙌

I'd be happy to volunteer to become a maintainer. I wrote the test suite and CI setup for this repo, and I use it in production at my company.

[titansmc]

How would I tell jupyterhub to use a specific release of ldapauthenticator ?
Cheers.

[consideRatio]

@titansmc, where jupyterhub runs, the specific release of ldapauthenticator that you want to use should be installed. But where does it run, how does it run, etc - it depends on how you have installed jupyterhub. Is it by a custom installation, a zero-to-jupyterhub-k8s installation, or the-littelest-jupyterhub installation etc.

Please open a new issue to discuss this where you think is most suitable.

[meeseeksmachine]

This pull request has been mentioned on Jupyter Community Forum. There might be relevant details there:

https://discourse.jupyter.org/t/ldap-uid-gid-integration-with-jupyterhub-k8s-hub-0-9-0-beta-2/3095/2