[BUG] Instance has been ratelimited

[eddydc]

Describe the bug
Getting

Error

Instance has been ratelimited

Continue your search with Farside
Whoogle:
https://farside.link/whoogle/search?q=dit is een test&lang_interface=lang_en&country=BE&theme=system

Searx:
https://farside.link/searx/search?q=dit is een test

Return Home

To Reproduce
Just do a search with Whoogle

Deployment Method

• Docker-compose

Version of Whoogle Search
lastest

Desktop (please complete the following information):
Win 10, latest FF

[T3chHunt3r]

I too have this issue on version 0.7.0

update: This issue seems to be resolved for me, I re-composed the docker container as well as changed locations for my VPN. I'm not sure which of the two fixed the issue however. It's more likely that it was a VPN issue as I was also blocked from certain sites due to the IP that I was under at the time.

[ShlomiD83]

same issue for me.
can't resolve the issue no matter what I do.

[sysoppl]

Same for me, just deployed and it's not working at all.

[ironboy1]

Same on my side for version 0.7.1 and 0.7.2

[bruvv]

This has been answered already: #211

[pieslinger]

I am having rate limiting issues occasionally and I am the only user of whoogle running in a container. I do not see how #211 has answered this already. I would install this for more people in my life as well if this issue goes away. Is there just something I need to modify in the setup?

[pieslinger]

I also only noticed it in 0.7.1 and 0.7.2 - although 0.7 was my first time using this. Just installed 0.7.3 tonight.

update: I use Pihole as well

[aw-jansen]

Same here, currently on 0.7.3. Getting the rate limiting message when connected via a NordVPN container. Would it still be a possibility to implement a workaround? On a different topic someone mentioned changing the query: searxng/searxng#159

[aindriu80]

I'm having the same error, only making a few requests and get this error, I'm using Pi-Hole and wonder has that anything to do with it?

[whaler-ragweed]

Same here on 0.7.4 using a public IP with password. Getting limited for average one or two days. Restarting the docker can fix it, but the problem always appears again.

[jojolll]

I use my own Whoogle instance (alone). Almost every day I reach the ratelimit after 10 to 15 searches during the day.

I have the chance with my LiveBox to be able with a simple click in the interface to renew my IP, that solves the problem for 1-2 days.
It's still very annoying and makes the solution of a personal Whoogle server very unviable.

[Albonycal]

there isn't much you can do about this issue, google blocks the requests it thinks are suspicious. If your IP is dynamic, try rebooting or change your VPS's IP.

[dr460nf1r3]

This might in fact also be caused by bots crawling the instances. At least I noticed that the rate limits became less after blocking most known bots via Cloudflare WAF (12k events over 24h) 🤔

[CptanPanic]

I am having this issue today also. I am not sure what #211 has to do with this, is there any debug to look at to see what is causing the issue?

[bruvv]

For me. A restart of the docket container fixes this.

[whaler-ragweed]

Same here on 0.7.4 using a public IP with password. Getting limited for average one or two days. Restarting the docker can fix it, but the problem always appears again.

I moved my VPS form Linode to Amazon Lightsail 5 months ago and never have the issue again.

[ForceConstant]

It is not clear from the error message, but looking at the code, the ratelimit is external to whoogle, and it is coming from a service that whoogle is querying. And if it gets a captcha, then it reports this. So I couldn't imaging how @bruvv restarting your container would affect this.

[0xspade]

For me. A restart of the docket container fixes this.

Doesn't work on my side.

[CptanPanic]

For me, it turns out it was the proxy I was using. I guess there was too many people at that IP address.

…

On Wed, Jan 11, 2023 at 4:45 AM 0xspade ***@***.***> wrote: For me. A restart of the docket container fixes this. Doesn't work on my side. — Reply to this email directly, view it on GitHub <#707 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA7F3ATE3JW43U4YK52W263WRZ6MTANCNFSM5SHZHBZQ> . You are receiving this because you commented.Message ID: ***@***.***>

[Paulie420]

So... I get this with both whoogle-search and googler - only when connected to my Private Internet Access VPN on some of their locations. Canada being one of them.... I'd bet a lot of folks seeing this issue are simply behind a VPN that Google is blocking - if this describes your setup, try disconnecting from your VPN and running whoogle-search again; viola, fixed.

[Snake16547]

Same issue here after 2 months of whoogle usage.
Tried to restart docker container - still rate limit exceeded

So basically moving to different machine would solve the problem? Any recommendations?

[dr460nf1r3]

From my understanding it's the IP getting rate-limited, so using another one should work.

[Snake16547]

Easy to say to change the IP when you have a VPS

[brokoler]

Since the last 2 days all of the public Whoogle instances are affected by this, most times only one of them works and yesterday it happened that even with Farside all instances were rate limited.

To me it seems like Google is going against Whoogle, but I still hope it's just a bug in the latest Whoogle software that can be fixed.

@benbusby do you have a clue what is happening right now?

[Snake16547]

I switched to the Oracle free tier and until now I'm free of any ratelimit messages.

[benbusby]

@brokoler public instances go through waves of all being rate limited, and then being clear again. It looks like some are available now. There's no simple fix to get around it unfortunately, but it isn't a bug in Whoogle. I always recommend using your own Whoogle instance, or running it locally on your machine (behind a VPN if you want) if rate limiting becomes too much of a problem.

[MaverickTN]

I just had this issue for the first time after updating to 0.8.2. I last used it 3 days ago, and it's rate-limited on all queries. Restarted the container, no change.
Private instance, closed network, docker container.

[Johnnii360]

I'm joining this issue. Same problem on my instance. Maybe it could be a workaround to use an API key in the developer console but it has to be implemented by @benbusby . It's a common thing that Google shows an CAPTCHA request if there too many search request of one and the same IP. I think it's their "DDoS protection".

Edit 1:
Another solution could be the usage of Tor service. But there must be an function that every new visitor is bind to a new Tor connection. If Whoogle needs also one and the same Tor connection it will also end in this issue. Otherwise the Tor service or connection has to be refreshed every day or hour or whatever. I will give it a try.

Edit 2:
Okay, bad idea. :D

Edit 3:
Uhm... I just changed the country now to my country (German) and also the search language. Now it works again. But the question ist: How long? :)

[robertpartridge]

I've run into this problem today with my deployment of version 0.8.2. My instance is behind basic HTTP auth with a fairly strong unique password so that it's not open to just anyone to use. Any advice?

Edit - reading through more of the responses and wondering if the problem might be related to the instance being deployed on Oracle's Cloud. Will investigate.

[seirdotexe]

Getting this too now on my own IP address, locally hosted. Already restarted the container and even reinstalled it.

[akovia]

Just installed whoogle 0.8.2-1 for the first time ever from Arch aur, so have no frame of reference of how it's supposed to even work.

Is there anything I can do to get this working?
I'm behind a VPN, so is that an actual issue to run a self hosted instance?

[Deviantgit]

Have also started running into this--I would get it occasionally over the past year and a half I've been using Whoogle, but generally it'd go away after a few hours. For the past two weeks or so I have only been getting the rate limited message despite taking down my docker container and waiting over 24 hours, etc. This is with it both behind a VPN as well as a non-VPN IP. It's rather odd.

[imTHAI]

Today I face the same problem. But I use it on an internal docker container, and I'm the only one who use it in the family. How is it possible that I have reached the rate limit of search requests for google, from a single home user ?! 🤔

EDIT: Even if I make the whoogle container goes out on internet through the internet connection of my parents (retired people using very little internet, living in a country other than me), I get this rate limit error.

[winhex]

Same problem. I'm deploying the docker container with whoogle 0.8.4 at my own home server. No public access, only personal using. Instance has been rate limited after 20-30 min of using

[Drallas]

I also have been rate limited, my Whoogle runs in a tteck LXC container. Updating the OS doesn't solve it!

[zidanpragata]

whatever time, if direct search method is not changed to the brilliant idea of "yacy search", error message "rate is limited" will still occur frequently.. the easiest solution to overcome this is to change the IP..

failed :)

[aindriu80]

I tried changing IP and using VPN but still getting rate limited, very frustrating

[akovia]

I bit the bullet and signed up for Kagi now that they made unlimited searches more affordable. I find it better than Google in so many ways and doubt I'll ever go back. I have no skin in the game, but they have a free trial if you want to check it out.

Either way I hope you get it figured out.

[Snake16547]

I had it running on Oracles free tier with zero problems. Several clients used my instance.
Now I switched to Hetzner Cloud and running smoothly as well.

[imTHAI]

Could someone explain why a whoogle instance can be sometimes rate-limited when the other computers (behind the same public ip) aren't when using google ?
Are the requests from whoogle so special (and harmful?) that Google identifies them and limits them ?

[Drallas]

I bit the bullet and signed up for Kagi now that they made unlimited searches more affordable. I find it better than Google in so many ways and doubt I'll ever go back. I have no skin in the game, but they have a free trial if you want to check it out.

Either way I hope you get it figured out.

Whoogle is nice, but the search results from Google feel manipulated and without still optimized to show payed results first. Would be better if Whoogle could include more search options. Kagi might be an alternative too!

[MuubNito]

ALL instances i am trying or doesnt work or just straight up give me another issue... Ouch

[imTHAI]

I don't know about you, but I can use my whoogle container again without any problems.
Maybe google changed something on his side ?

[seirdotexe]

Since a week, my instance is getting rate limited constantly now, only after 20 searches. Makes it impossible to use Whoogle

[C8opmBM]

happened to me as well 2 times today.

[EasternPA]

I just got this for the first time ever. I am the sole user of a docker-compose stack on Oracle Free Tier. I recently lost SSH access due to rebuilding my PC, so I need to rebuild the cloud instance anyway. Perhaps a rebuild will clear it up.

[Zundrium]

Same here, CHAPTA seems to be the issue

[byertsy]

Same issue here. Downloaded today and was bust setting up a new CSS theme and got this issue.

[mlause]

Same here. Is there anyway to passthrough the captcha?

[working-name]

I run whoogle both behind a residential connection and datacenter (so I can switch between them when one gets rate limited).

There's no difference in getting rate limited based on IP or IP type (residential vs data center).
I'm the only one using both instances, checked logs.

I do aggressively google things though (still manual, not automated, so definitely not more than 1 search per second), sometimes with minor tweaks (like "sle crescentic" which later changed to "psgn crescentic") and that's when I notice a higher incidence of rate limiting.

Update: switching to searxng on the same rate limited residential IP gives me instant google results. No rate limiting. Worth checking this out, @benbusby : searxng/searxng#1642

🙏 @unixfox