Mouser crawling is broken

[hillsandales]

I've tried Digikey, Mouser, and LCSC, but none of them work.

Digikey brings me to what looks like a Lynx interactive for their login page. I can type in username and password, but I'm not sure why I need to if it's using an API private key. Also Lynx won't let me select Login (assuming it's a button) anyway. So there's no way to continue.

Mouser does return a list of parts with my API key, but when I select the part I want, it says warning: failed to get page, retrying in 3.0s via 'www2.mouser.com' with new session and user agent. It tries another eu. url too, but no luck. Then it returns warning: failed to finalize part specifications from 'https://www.mouser.com/ProductDetail/Microchip-Technology/2N4001?qs=7D1LtPJG0i0bXRytgh1wrg%3D%3D' (blocked) and exits saying it failed to import.

LCSC doesn't return anything and says failed to get webpage.

This tool looks like it's a lot simpler than others, which is nice, but I can't get it to work for some reason.

Also, I tried it on WIndows with Git Shell, and the tool doesn't work because the shell doesn't recognize selection with Spacebar and Enter. Tried it on linux, and the script in the bash shell works.

[30350n]

Digikey brings me to what looks like a Lynx interactive for their login page. I can type in username and password, but I'm not sure why I need to if it's using an API private key. Also Lynx won't let me select Login (assuming it's a button) anyway. So there's no way to continue.

Not sure what you mean with "Lynx interactive" exactly (google also didn't come up with any useful results). Are you just talking about the OAuth browser popup? If so, I'm honestly also not sure why that's required in addition to the API key, but I think that's just how it is. (See the digikey-api python library and the DigiKey API docs).

Mouser does return a list of parts with my API key, but when I select the part I want, it says warning: failed to get page, retrying in 3.0s via 'www2.mouser.com' with new session and user agent. It tries another eu. url too, but no luck. Then it returns warning: failed to finalize part specifications from 'https://www.mouser.com/ProductDetail/Microchip-Technology/2N4001?qs=7D1LtPJG0i0bXRytgh1wrg%3D%3D' (blocked) and exits saying it failed to import.

Can confirm, this is happening for me too. The problem here is that Mousers API sucks ass, which I've been circumventing by just crawling the page directly. Seems like they recently "hardened" their crawling protection again. That's great.

LCSC doesn't return anything and says failed to get webpage.

Similar thing, LCSC doesn't really offer an API directly, but there's API endpoints which URLs got recently changed (see #47 and #48).

Also, I tried it on WIndows with Git Shell, and the tool doesn't work because the shell doesn't recognize selection with Spacebar and Enter. Tried it on linux, and the script in the bash shell works.

I'm using/developing this from Windows, mainly using the VSCode terminal and Cmder. Standard cmd should also work I'm pretty sure.

[hillsandales]

I see. Good to know it's not just me then. Thanks!

[30350n]

For future reference: You can simply install the newest git version via pipx install git+https://github.com/30350n/inventree_part_import (you might have to uninstall any old installed versions first).

[30350n]

Also, yes LCSC import was fixed in #48. I think I wanted to wait until Mouser is fixed too before creating a new release, but that might take some more time ...

[30350n]

This should be resolved with v1.7. I'm using the parameters provided by the Mouser API now (so if they ever decide to fix it 🙄 , that part should work out of the box). More importantly though, if crawling fails now, the part will still be imported (you might just have to define all the parameters by hand).