Release 0.5.1

README.md

@@ -10,21 +10,23 @@ To support 'service principal' with certificate, ADAL depends on the 'cryptograp
 
 * For Windows and macOS
 
-Upgrade to the latest pip (8.1.2 as of June 2016) and just do `pip install adal`.
+  Upgrade to the latest pip (8.1.2 as of June 2016) and just do `pip install adal`.
 
 * For Linux
 
-Upgrade to the latest pip (8.1.2 as of June 2016).
+  Upgrade to the latest pip (8.1.2 as of June 2016).
 
-You'll need a C compiler, libffi + its development headers, and openssl + its development headers. Refer to [cryptography installation](https://cryptography.io/en/latest/installation/)
+  You'll need a C compiler, libffi + its development headers, and openssl + its development headers.
+  Refer to [cryptography installation](https://cryptography.io/en/latest/installation/)
 
 * To install from source:
 
-Upgrade to the latest pip (8.1.2 as of June 2016).
-Before run `python setup.py install`, to avoid dealing with compilation errors from cryptography, run `pip install cryptography` first to use statically-linked wheels.
-If you still like build from source, refer to [cryptography installation](https://cryptography.io/en/latest/installation/).
+  Upgrade to the latest pip (8.1.2 as of June 2016).
+  To avoid dealing with compilation errors from cryptography, first run `pip install cryptography` to use statically-linked wheels.
+  Next, run `python setup.py install`
 
-For more context, starts with this [stackoverflow thread](http://stackoverflow.com/questions/22073516/failed-to-install-python-cryptography-package-with-pip-and-setup-py).
+  If you still like to build from source, refer to [cryptography installation](https://cryptography.io/en/latest/installation/).
+  For more context, start with this [stackoverflow thread](http://stackoverflow.com/questions/22073516/failed-to-install-python-cryptography-package-with-pip-and-setup-py).
 
 ### Acquire Token with Client Credentials
 
@@ -45,6 +47,19 @@ Find the `Main logic` part in the [sample](sample/device_code_sample.py#L49-L54)
 ### Acquire Token with authorization code
 Find the `Main logic` part in the [sample](sample/website_sample.py#L107-L115) for a complete bare bones web site that makes use of the code below.
 
+## Logging
+
+#### Personal Identifiable Information (PII) & Organizational Identifiable Information (OII)
+
+Starting from ADAL Python 0.5.1, by default, ADAL logging does not capture or log any PII or OII.  The library allows app developers to turn this on by configuring the `enable_pii` flag on the AuthenticationContext. By turning on PII or OII, the app takes responsibility for safely handling highly-sensitive data and complying with any regulatory requirements.
+
+```python
+//PII or OII logging disabled. Default Logger does not capture any PII or OII.
+auth_context = AuthenticationContext(...)
+
+//PII or OII logging enabled
+auth_context = AuthenticationContext(..., enable_pii=True)
+```
 
 ## Samples and Documentation
 We provide a full suite of [sample applications on GitHub](https://github.com/azure-samples?utf8=%E2%9C%93&q=active-directory&type=&language=) and an [Azure AD developer landing page](https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-developers-guide) to help you get started with learning the Azure Identity system. This includes tutorials for native clients and web applications. We also provide full walkthroughs for authentication flows such as OAuth2, OpenID Connect and for calling APIs such as the Graph API.
@@ -78,16 +93,16 @@ This project has adopted the [Microsoft Open Source Code of Conduct](https://ope
 ``` $ pip install adal ```
 
 ### http tracing/proxy
-If need to bypass self-signed certificates, turn on the environment variable of `ADAL_PYTHON_SSL_NO_VERIFY`
+If you need to bypass self-signed certificates, turn on the environment variable of `ADAL_PYTHON_SSL_NO_VERIFY`
 
 
 ## Note
 
 ### Changes on 'client_id' and 'resource' arguments after 0.1.0
-The convinient methods in 0.1.0 have been removed, and now your application should provide parameter values to `client_id` and `resource`.
+The convenient methods in 0.1.0 have been removed, and now your application should provide parameter values to `client_id` and `resource`.
 
 2 Reasons:
 
-* Each adal client should have an Application ID representing an valid application registered in a tenant. The old methods borrowed the client-id of [azure-cli](https://github.com/Azure/azure-xplat-cli), which is never right. It is simple to register your application and get a client id. Many walkthroughs exist. You can follow [one of those](https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-integrating-applications). Do check out if you are new to AAD.
+* Each adal client should have an Application ID representing a valid application registered in a tenant. The old methods borrowed the client-id of [azure-cli](https://github.com/Azure/azure-xplat-cli), which is never right. It is simple to register your application and get a client id. You can follow [this walkthrough](https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-integrating-applications). Do check out if you are new to AAD.
 
 * The old method defaults the `resource` argument to 'https://management.core.windows.net/', now you can just supply this value explictly. Please note, there are lots of different azure resources you can acquire tokens through adal though, for example, the samples in the repository acquire for the 'graph' resource. Because it is not an appropriate assumption to be made at the library level, we removed the old defaults.

adal/__init__.py

@@ -27,7 +27,7 @@
 
 # pylint: disable=wrong-import-position
 
-__version__ = '0.5.0'
+__version__ = '0.5.1'
 
 import logging
 

adal/authentication_context.py

@@ -47,7 +47,7 @@ class AuthenticationContext(object):
 
     def __init__(
             self, authority, validate_authority=None, cache=None,
-            api_version='1.0', timeout=None):
+            api_version='1.0', timeout=None, enable_pii=False):
         '''Creates a new AuthenticationContext object.
 
         By default the authority will be checked against a list of known Azure
@@ -73,6 +73,8 @@ def __init__(
         :param timeout: (optional) requests timeout. How long to wait for the server to send
             data before giving up, as a float, or a `(connect timeout,
             read timeout) <timeouts>` tuple.
+        :param enable_pii: (optional) Unless this is set to True,
+            there will be no Personally Identifiable Information (PII) written in log.
         '''
         self.authority = Authority(authority, validate_authority is None or validate_authority)
         self._oauth2client = None
@@ -93,7 +95,8 @@ def __init__(
             'options': GLOBAL_ADAL_OPTIONS,
             'api_version': api_version,
             'verify_ssl': None if env_value is None else not env_value, # mainly for tracing through proxy
-            'timeout':timeout
+            'timeout':timeout,
+            "enable_pii": enable_pii,
             }
         self._token_requests_with_user_code = {}
         self.cache = cache or TokenCache()

adal/token_cache.py

@@ -81,8 +81,9 @@ def remove(self, entries):
         with self._lock:
             for e in entries:
                 key = _get_cache_key(e)
-                self._cache.pop(key)
-            self.has_state_changed = True
+                removed = self._cache.pop(key, None)
+                if removed is not None:
+                    self.has_state_changed = True
 
     def add(self, entries):
         with self._lock:

adal/wstrust_request.py

@@ -145,7 +145,9 @@ def acquire_token(self, username, password):
 
         operation = "WS-Trust RST"
         resp = requests.post(self._wstrust_endpoint_url, headers=options['headers'], data=rst,
-                             allow_redirects=True, verify=self._call_context.get('verify_ssl', None))
+                             allow_redirects=True,
+                             verify=self._call_context.get('verify_ssl', None),
+                             timeout=self._call_context.get('timeout', None))
 
         util.log_return_correlation_id(self._log, operation, resp)
 

adal/wstrust_response.py

@@ -55,6 +55,35 @@ def scrub_rstr_log_message(response_str):
 
     return 'RSTR Response: ' + scrubbed_rstr
 
+def findall_content(xml_string, tag):
+    """
+    Given a tag name without any prefix,
+    this function returns a list of the raw content inside this tag as-is.
+
+    >>> findall_content("<ns0:foo> what <bar> ever </bar> content </ns0:foo>", "foo")
+    [" what <bar> ever </bar> content "]
+
+    Motivation:
+
+    Usually we would use XML parser to extract the data by xpath.
+    However the ElementTree in Python will implicitly normalize the output
+    by "hoisting" the inner inline namespaces into the outmost element.
+    The result will be a semantically equivalent XML snippet,
+    but not fully identical to the original one.
+    While this effect shouldn't become a problem in all other cases,
+    it does not seem to fully comply with Exclusive XML Canonicalization spec
+    (https://www.w3.org/TR/xml-exc-c14n/), and void the SAML token signature.
+    SAML signature algo needs the "XML -> C14N(XML) -> Signed(C14N(Xml))" order.
+
+    The binary extention lxml is probably the canonical way to solve this
+    (https://stackoverflow.com/questions/22959577/python-exclusive-xml-canonicalization-xml-exc-c14n)
+    but here we use this workaround, based on Regex, to return raw content as-is.
+    """
+    # \w+ is good enough for https://www.w3.org/TR/REC-xml/#NT-NameChar
+    pattern = r"<(?:\w+:)?%(tag)s(?:[^>]*)>(.*)</(?:\w+:)?%(tag)s" % {"tag": tag}
+    return re.findall(pattern, xml_string, re.DOTALL)
+
+
 class WSTrustResponse(object):
 
     def __init__(self, call_context, response, wstrust_version):
@@ -157,6 +186,7 @@ def _parse_token(self):
 
             # Adjust namespaces (without this they are autogenerated) so this is understood
             # by the receiver.  Then make a string repr of the element tree node.
+            # See also http://blog.tomhennigan.co.uk/post/46945128556/elementtree-and-xmlns
             ET.register_namespace('saml', 'urn:oasis:names:tc:SAML:1.0:assertion')
             ET.register_namespace('ds', 'http://www.w3.org/2000/09/xmldsig#')
 
@@ -178,6 +208,15 @@ def _parse_token(self):
         if self.token is None:
             raise AdalError("Unable to find any tokens in RSTR.")
 
+    @staticmethod
+    def _parse_token_by_re(raw_response):
+        for rstr in findall_content(raw_response, "RequestSecurityTokenResponse"):
+            token_types = findall_content(rstr, "TokenType")
+            tokens = findall_content(rstr, "RequestedSecurityToken")
+            if token_types and tokens:
+                return tokens[0].encode('us-ascii'), token_types[0]
+
+
     def parse(self):
         if not self._response:
             raise AdalError("Received empty RSTR response body.")
@@ -195,7 +234,12 @@ def parse(self):
                 str_fault_message = self.fault_message or 'NONE'
                 error_template = 'Server returned error in RSTR - ErrorCode: {} : FaultMessage: {}'
                 raise AdalError(error_template.format(str_error_code, str_fault_message))
-            self._parse_token()
+
+            token_found = self._parse_token_by_re(self._response)
+            if token_found:
+                self.token, self.token_type = token_found
+            else:  # fallback to old logic
+                self._parse_token()
         finally:
             self._dom = None
             self._parents = None

tests/test_wstrust_response.py

@@ -36,6 +36,7 @@
 
 from adal.constants import XmlNamespaces, Errors, WSTrustVersion
 from adal.wstrust_response import WSTrustResponse
+from adal.wstrust_response import findall_content
 
 _namespaces = XmlNamespaces.namespaces
 _call_context = {'log_context' : {'correlation-id':'test-corr-id'}}
@@ -101,5 +102,33 @@ def test_rstr_unparseable_xml(self):
             wstrustResponse = WSTrustResponse(_call_context, '<This is not parseable as an RSTR', WSTrustVersion.WSTRUST13)
             wstrustResponse.parse()
 
+    def test_findall_content_with_comparison(self):
+        content = """
+            <saml:Assertion xmlns:saml="SAML:assertion">
+                <ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
+                foo
+                </ds:Signature>
+            </saml:Assertion>"""
+        sample = ('<ns0:Wrapper xmlns:ns0="namespace0">'
+            + content
+            + '</ns0:Wrapper>')
+
+        # Demonstrating how XML-based parser won't give you the raw content as-is
+        element = ET.fromstring(sample).findall('{SAML:assertion}Assertion')[0]
+        assertion_via_xml_parser = ET.tostring(element)
+        self.assertNotEqual(content, assertion_via_xml_parser)
+        self.assertNotIn(b"<ds:Signature>", assertion_via_xml_parser)
+
+        # The findall_content() helper, based on Regex, will return content as-is.
+        self.assertEqual([content], findall_content(sample, "Wrapper"))
+
+    def test_findall_content_for_real(self):
+        with open(os.path.join(os.getcwd(), 'tests', 'wstrust', 'RSTR.xml')) as f:
+            rstr = f.read()
+        wstrustResponse = WSTrustResponse(_call_context, rstr, WSTrustVersion.WSTRUST13)
+        wstrustResponse.parse()
+        self.assertIn("<X509Data>", rstr)
+        self.assertIn(b"<X509Data>", wstrustResponse.token)  # It is in bytes
+
 if __name__ == '__main__':
     unittest.main()