Merge pull request #27 from tomquirk/voyager-update

Voyager update

.env.example

@@ -1,2 +1,4 @@
 LINKEDIN_USERNAME="test@mailnator.com"
-LINKEDIN_PASSWORD="coolpasswordhere"
+LINKEDIN_PASSWORD="coolpasswordhere"
+TEST_PROFILE_ID="ACoAABQ11fIBQLGQbB1V1XPBZJsRwfK5r1U2Rzt"
+TEST_CONVERSATION_ID="6419123050114375168"

.gitignore

@@ -6,7 +6,7 @@
 *.pyc
 .vscode
 /playground.py
-.cookies
+.cookie.jr
 .pytest_cache
 **__pycache__/
 

DOCS.md

@@ -366,6 +366,7 @@ Perform a Linkedin search and return the results.
 - `network_depth <str>` - the network depth to search within. One of {`F`, `S`, or `O`} (first, second and third+ respectively)
 - `regions <list>` - list of Linkedin region ids
 - `industries <list>` - list of Linkedin industry ids
+- ... more (see code)
 
 **Return**
 

Pipfile

@@ -6,14 +6,11 @@ verify_ssl = true
 [dev-packages]
 ipdb = "*"
 black = "*"
+pytest = "*"
 pylint = "*"
 
 [packages]
 requests = "*"
-pytest = "*"
-
-[requires]
-python_version = "3.7"
 
 [pipenv]
 allow_prereleases = true

README.md

@@ -2,22 +2,22 @@
 
 👨‍💼 Python Wrapper for the Linkedin API
 
-![v0.2.0](https://img.shields.io/badge/PyPI-v0.2.0-blue.svg)
+![v1.0.0](https://img.shields.io/badge/PyPI-v0.2.0-blue.svg)
 
 > No "official" API access required - just use a valid Linkedin account!
 
-Programmatically send messages, perform searchs, get profile data and more, all with only your Linkedin account!
+Programmatically send messages, perform searches, get profile data and more, all with a standard Linkedin account!
 
 ##### USE AT YOUR OWN RISK 😉
-This project should only be used as a learning project. Using it would violate Linkedin's Terms of Use. I am not responsible for your account being blocked (which they will definitely do. Hint: **don't use your personal Linkedin account**)
+This project should only be used as a learning project. Using it would violate Linkedin's User Agreement. I am not responsible for your account being blocked (which they will definitely do - see User Agreement section 8.2). Hint: **don't use a Linkedin account that you care about**)
 
 ## Overview
 
 This project attempts to provide a simple Python interface for the Linkedin API.
 
 > Do you mean the [legit Linkedin API](https://developer.linkedin.com/)?
 
-NO! To retrieve structured data, the [Linkedin Website](https://linkedin.com) uses a service they call **Voyager**. Voyager endpoints give us access to pretty much everything we could want from Linkedin: profiles, companies, connections, messages, etc.
+NO! To retrieve structured data, the [Linkedin Website](https://linkedin.com) uses a service they call **Voyager**. Voyager endpoints give us access to pretty much everything we could want from Linkedin: profiles, companies, connections, messages, etc. - anything that you can see on linkedin.com, we can get from Voyager.
 
 So specifically, this project aims to provide complete coverage for Voyager.
 
@@ -52,26 +52,44 @@ connections = api.get_profile_connections('1234asc12304', max_connections=200)
 ## Documentation
 For a complete reference documentation, see the [DOCS.md](https://github.com/tomquirk/linkedin-api/blob/master/DOCS.md)
 
-## Setup
+## Development Setup
 
 ### Dependencies
 
-* Python 3
+* Python 3.7
 * A valid Linkedin user account (don't use your personal account, if possible)
 * Pipenv (optional)
 
-1. Using pipenv...
+### Installation
+
+1. Create a `.env` config file. An example is provided in `.env.example` - you include at least all of the settings set there.
+2. Using pipenv...
 
     ```
     $ pipenv install
     $ pipenv shell
     ```
 
+### Running tests
+
+```
+$ python -m pytest tests
+```
+
 ### Troubleshooting
 
-#### > I can't authenticate - I keep getting a CHALLENGE!?!
+#### > I keep getting a CHALLENGE!?!
+
+Linkedin will throw you a curve ball in the form of a Challenge URL. We currently don't handle this, and so you're kinda screwed. We think it could be only IP-based (i.e. logging in from different location). Your best chance at resolution is to log out and log back in on your browser.
+
+##### Known reasons for Challenge:
+- 2FA
+- Rate-limit - "It looks like you’re visiting a very high number of pages on LinkedIn.". Note - n=1 experiment where this page was hit after ~900 contiguous requests in a single session (within the hour) (these included random delays between each request), as well as a bunch of testing, so who knows the actual limit.
+
+Please add more as you come across them.
 
-Linkedin will throw you a curve ball in the form of a challenge URL. We currently don't handle this, and so you're kinda screwed. We think it could be only IP-based (i.e. logging in from different location). Your best chance at resolution is to log out and log back in on your browser.
+#### Search woes
+- Mileage may vary when searching general keywords like "software" using the standard `search` method. They've recently added some smarts around search whereby they group results by people, company, jobs etc. if the query is general enough. Try to use an entity-specific search method (i.e. search_people) where possible.
 
 ## In-depth overview
 

linkedin_api/__init__.py

@@ -4,7 +4,7 @@
 from .linkedin import Linkedin
 
 __title__ = "linkedin_api"
-__version__ = "0.2.0"
+__version__ = "1.0.0"
 __description__ = "Python Wrapper for the Linkedin API"
 
 __license__ = "MIT"

linkedin_api/client.py

@@ -30,7 +30,11 @@ class Client(object):
                 "Chrome/66.0.3359.181 Safari/537.36",
             ]
         ),
+        # "accept": "application/vnd.linkedin.normalized+json+2.1",
+        "accept-language": "en-AU,en-GB;q=0.9,en-US;q=0.8,en;q=0.7",
+        "x-li-lang": "en_US",
         "x-restli-protocol-version": "2.0.0",
+        # "x-li-track": '{"clientVersion":"1.2.6216","osName":"web","timezoneOffset":10,"deviceFormFactor":"DESKTOP","mpName":"voyager-web"}',
     }
 
     # Settings for authenticating with Linkedin