Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added functionality to use persistent session #871

Merged
merged 1 commit into from
Sep 2, 2022

Conversation

lucasmrdt
Copy link
Contributor

@lucasmrdt lucasmrdt commented Sep 2, 2022

Hi guys, I'm submitting this small change to allow using cookies and credentials simultaneously.

To my knowledge:

If you want to scrap with a logged account, you can either give your credentials or your cookies.

2 issues quickly arise if you use a logged account for a long period of time (e.g. scrapping things on a private FB group every day) :

  • If you use the credentials property, Facebook will quickly flag you as a suspect as you log in too often, making you suspicious.
  • If you use the cookies  property, you must manually refresh the content of the cookies to avoid the expiration of your session.

The use_persistent_session  method allows you to log in before querying private information by only providing your credentials. Under the hood, the cookies are automatically stored, used, and refreshed to avoid as far as possible using your credentials and being flagged as suspicious by Facebook.

Usage:

>>> facebook_scraper.use_persistent_session(FB_USERNAME, FB_PASSWORD)

>>> for post in get_posts('private_group', pages=1):
...           print(post['text'])
...
Lorem ipsum dolor sit amet consectetur adipisicing elit.
Maxime mollitia molestiae quas vel sint commodi (...)

I remain open to all discussions 😄.

@neon-ninja neon-ninja merged commit fe03cf3 into kevinzg:master Sep 2, 2022
@jasonren0403
Copy link

I tried the new commit and the resulted .pckl file always load with error. The file was acting like this from binary file viewer:

00000000: 8004 4e2e

I have reviewed the source code and found that login function in facebook_scraper.py indeed does not return anything...

@lucasmrdt
Copy link
Contributor Author

I tried the new commit and the resulted .pckl file always load with error. The file was acting like this from binary file viewer:

00000000: 8004 4e2e

I have reviewed the source code and found that login function in facebook_scraper.py indeed does not return anything...

Indeed, you are right, I forget to update the files correctly. Just open a new PR to provide a fix for this. Thank you for pointing me this error.

@liuying12138
Copy link

liuying12138 commented Sep 14, 2022

Hello, I encountered some problems when using this method:

import facebook_scraper
facebook_scraper.use_persistent_session(FB_USERNAME, FB_PASSWORD)
for post in facebook_scraper.get_posts("VientianeMai", page_limit=None,
                      options={"allow_extra_requests": False, "posts_per_page": 200}):
    print(post)

Traceback (most recent call last): File "/root/project/Facebook/test.py", line 4, in <module> facebook_scraper.use_persistent_session("xxxxxxx", "xxxxxxx") File "/root/lib/python3.8/site-packages/facebook_scraper/__init__.py", line 552, in use_persistent_session set_cookies(cookies) File "/root/lib/python3.8/site-packages/facebook_scraper/__init__.py", line 53, in set_cookies raise exceptions.InvalidCookies(f"Cookies are not valid") facebook_scraper.exceptions.InvalidCookies: Cookies are not valid

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants