I get these errors every day and often during the day

[mfabiani53]

Every day i get these errors in the log:

2023-03-22 07:00:08.782 ERROR (MainThread) [homeassistant.helpers.entity] Update for sensor.playlists_sensor fails
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 878, in urlopen
    return self.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 878, in urlopen
    return self.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 878, in urlopen
    return self.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 868, in urlopen
    retries = retries.increment(method, url, response=response, _pool=self)
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.spotify.com', port=443): Max retries exceeded with url: /v1/me/playlists?limit=40&offset=0 (Caused by ResponseError('too many 502 error responses'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/spotipy/client.py", line 264, in _internal_call
    response = self._session.request(
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 556, in send
    raise RetryError(e, request=request)
requests.exceptions.RetryError: HTTPSConnectionPool(host='api.spotify.com', port=443): Max retries exceeded with url: /v1/me/playlists?limit=40&offset=0 (Caused by ResponseError('too many 502 error responses'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 541, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 739, in async_device_update
    raise exc
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/config/custom_components/spotcast/sensor.py", line 116, in update
    resp = self.hass.data[DOMAIN]["controller"].get_playlists(
  File "/config/custom_components/spotcast/spotcast_controller.py", line 333, in get_playlists
    resp = client.current_user_playlists(limit=limit)
  File "/usr/local/lib/python3.10/site-packages/spotipy/client.py", line 635, in current_user_playlists
    return self._get("me/playlists", limit=limit, offset=offset)
  File "/usr/local/lib/python3.10/site-packages/spotipy/client.py", line 321, in _get
    return self._internal_call("GET", url, payload, kwargs)
  File "/usr/local/lib/python3.10/site-packages/spotipy/client.py", line 305, in _internal_call
    raise SpotifyException(
spotipy.exceptions.SpotifyException: http status: 429, code:-1 - /v1/me/playlists?limit=40&offset=0:
 Max Retries, reason: too many 502 error responses

Make sure to validate all the elements before submitting the ticket (Exception to the steps marked as optional)

• [x ] Using latest version of spotcast
• Using latest stable version of Home Assistant
• [x ] I have setup the Spotify integration in Home Assistant
• [x ] I have renewed my sp_dc and sp_key values and restarted Home Assistant (see README)
• [x ] (optional) I have Spotify Premium
• (optional) I am using multiple accounts
• (optional) I'm attaching relevant logs with level debug for component spotcast (see README)
• (optional) I'm using entity_id in the service call and have tried device_name but the issue remains

Environment

• Installation type: [HA_OS]
• HA version: [2022.3.5 ]
• spotcast version: [latest ]

[fcusson]

@mfabiani53, the weird thing is 502 is a server side error. That should mean that Spotify experiencing issues, but that wouldn't make sense if you are the only one with the problem.

Are you using any unconventional networking gear like personal DNS (pihole, adguard) are using any network wide VPN?

[mfabiani53]

Not at all... no unconventional networking, neither VPN.
You mean i am not the only one with this problem?
Just few minutes ago i got this:

2023-03-24 16:01:06.386 ERROR (SyncWorker_29) [spotipy.client] Max Retries reached
2023-03-24 16:01:06.402 ERROR (MainThread) [homeassistant.helpers.entity] Update for sensor.playlists_sensor fails
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 878, in urlopen
    return self.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 878, in urlopen
    return self.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 878, in urlopen
    return self.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 868, in urlopen
    retries = retries.increment(method, url, response=response, _pool=self)
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.spotify.com', port=443): Max retries exceeded with url: /v1/me/playlists?limit=40&offset=0 (Caused by ResponseError('too many 504 error responses'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/spotipy/client.py", line 264, in _internal_call
    response = self._session.request(
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 556, in send
    raise RetryError(e, request=request)
requests.exceptions.RetryError: HTTPSConnectionPool(host='api.spotify.com', port=443): Max retries exceeded with url: /v1/me/playlists?limit=40&offset=0 (Caused by ResponseError('too many 504 error responses'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 541, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 739, in async_device_update
    raise exc
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/config/custom_components/spotcast/sensor.py", line 116, in update
    resp = self.hass.data[DOMAIN]["controller"].get_playlists(
  File "/config/custom_components/spotcast/spotcast_controller.py", line 333, in get_playlists
    resp = client.current_user_playlists(limit=limit)
  File "/usr/local/lib/python3.10/site-packages/spotipy/client.py", line 635, in current_user_playlists
    return self._get("me/playlists", limit=limit, offset=offset)
  File "/usr/local/lib/python3.10/site-packages/spotipy/client.py", line 321, in _get
    return self._internal_call("GET", url, payload, kwargs)
  File "/usr/local/lib/python3.10/site-packages/spotipy/client.py", line 305, in _internal_call
    raise SpotifyException(
spotipy.exceptions.SpotifyException: http status: 429, code:-1 - /v1/me/playlists?limit=40&offset=0:
 Max Retries, reason: too many 504 error responses

I changed in sensor.py file the limit of the playlists to 40 and the language to 'it'. Could this be in some way the cause of the error?

[fcusson]

You mean i am not the only one with this problem?

Technically you are not at fault based on the error code (anything >=500 is server side error), but Spotify could be giving bad error codes or you could be triggering an edge case they didn't expect.

I changed in sensor.py file the limit of the playlists to 40 and the language to 'it'. Could this be in some way the cause of the error?

That would have been a very good information to have from the start. Can you provide me your modified version of the sensor.py file. Also I would need your configuration for spotcast. If you hardcoded your sp_dc and sp_key, first anonymise then and please use !secret.

[mfabiani53]

This is modified version of sensor.py:

"""Sensor platform for Chromecast devices."""
from __future__ import annotations

import collections
import json
import logging
from datetime import timedelta
import homeassistant.core as ha_core

from homeassistant.components.sensor import SensorEntity
from homeassistant.const import STATE_OK, STATE_UNKNOWN
from homeassistant.util import dt

from .helpers import get_cast_devices
from .const import (
    DOMAIN,
    CONF_SPOTIFY_COUNTRY
)

_LOGGER = logging.getLogger(__name__)

SENSOR_SCAN_INTERVAL_SECS = 60
SCAN_INTERVAL = timedelta(seconds=SENSOR_SCAN_INTERVAL_SECS)


def setup_platform(hass:ha_core.HomeAssistant, config:collections.OrderedDict, add_devices, discovery_info=None):

    try:
        country = config[CONF_SPOTIFY_COUNTRY]
    except KeyError:
        country = None

    add_devices([ChromecastDevicesSensor(hass)])
    add_devices([ChromecastPlaylistSensor(hass, country)])


class ChromecastDevicesSensor(SensorEntity):
    def __init__(self, hass):
        self.hass = hass
        self._state = STATE_UNKNOWN
        self._chromecast_devices = []
        self._attributes = {"devices_json": [], "devices": [], "last_update": None}
        _LOGGER.debug("initiating sensor")

    @property
    def name(self):
        return "Chromecast Devices"

    @property
    def state(self):
        return self._state

    @property
    def extra_state_attributes(self):
        """Return the state attributes."""
        return self._attributes

    def update(self):
        _LOGGER.debug("Getting chromecast devices")

        known_devices = get_cast_devices(self.hass)
        _LOGGER.debug("sensor devices %s", known_devices)

        chromecasts = [
            {
                "uuid": str(cast_info.cast_info.uuid),
                "model_name": cast_info.cast_info.model_name,
                "name": cast_info.cast_info.friendly_name,
                "manufacturer": cast_info.cast_info.manufacturer,
                "cast_type": cast_info.cast_info.cast_type,
            }
            for cast_info in known_devices
        ]

        self._attributes["devices_json"] = json.dumps(chromecasts, ensure_ascii=False)
        self._attributes["devices"] = chromecasts
        self._attributes["last_update"] = dt.now().isoformat("T")
        self._state = STATE_OK


class ChromecastPlaylistSensor(SensorEntity):
    def __init__(self, hass: ha_core, country=None):
        self.hass = hass
        self._state = STATE_UNKNOWN
        self.country = country
        self._attributes = {"playlists": [], "last_update": None}
        _LOGGER.debug("initiating playlist sensor")

    @property
    def name(self):
        return "Playlists sensor"

    @property
    def state(self):
        return self._state

    @property
    def extra_state_attributes(self):
        """Return the state attributes."""
        return self._attributes

    def update(self):
        _LOGGER.debug("Getting playlists")

        if self.country is not None:
            country_code = self.country
        else:
            # kept the country code to SE if not provided by the user for retrocompatibility
            country_code = "SE"

        playlist_type = "user"
        locale = "it"
        limit = 40
        account = None

        resp = self.hass.data[DOMAIN]["controller"].get_playlists(
            account, playlist_type, country_code, locale, limit
        )
        self._attributes["playlists"] = [{ "uri": x['uri'], "name": x['name']} for x in resp['items'] ]

        self._attributes["last_update"] = dt.now().isoformat("T")
        self._state = STATE_OK

And this my config about spotcast:

spotcast:
  sp_dc: !secret sp_dc
  sp_key: !secret sp_key
  country: IT

Please note that i changed the locale only now as before it was set to 'en' while the limit was also changed many days ago... I still do not restart HA to see if the changing of the locale will affect this error.

[fcusson]

I tested on my end and the new limit doesn't affect the calls. As predicted, the limit Spotify has is 50 and when you go over the error is 403 unauthorized, not 504 which is a timeout error.

I dug a liitle deeper and saw this in your log:

spotipy.exceptions.SpotifyException: http status: 429, code:-1 - /v1/me/playlists? Limit=40&offset=0: Max Retries, reason: too many 504 error responses

I was too focused on the 504 and missed the 429 error. Now that is a very different story. 429 is a too many request error meaning your are busting the rate limit of Spotify API.

Can you look at your integration dashboard on the developpper portal and provide a screen capture of your total request charts, here's an example:

I want to see if you have an abnormally high endpoint being used. Normally /v1/me/player should be the main endpoint used. Also make sure that your monthly active user makes sense. You said you are not using multiple account, so there should only be one.

[mfabiani53]

What you mean with this:

 your are busting the rate limit of Spotify API

About requests, you mean this:

Also make sure that your monthly active user makes sense

[fcusson]

What you mean with this:

your are busting the rate limit of Spotify API

All API have rate limits they don't want a script kitty who thiks he's a hacker send a bunch a distributed queries and clog the API endpoint fro 4chans points.

You can read more on the rate limit here, but with the number of queries you have I doubt that you have a generally busy api query rate. You seems to have more of a burst problem, basically you have rolling 30 sec. of number of queries and if your rolling sum of queries go over the limit you get a 429 error.

Before we get stuck in a circle for no reason. Could you revert the changes you made in the sensor.py to verify if it resolves the problem or not? At least we'll know if it is the cause.

[mfabiani53]

Ok, i'll let you know...

[mfabiani53]

I switched back to the default values, but this morning i got the same errors as before:

2023-03-27 07:00:14.658 ERROR (MainThread) [homeassistant.helpers.entity] Update for sensor.playlists_sensor fails
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 878, in urlopen
    return self.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 878, in urlopen
    return self.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 878, in urlopen
    return self.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 868, in urlopen
    retries = retries.increment(method, url, response=response, _pool=self)
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.spotify.com', port=443): Max retries exceeded with url: /v1/me/playlists?limit=10&offset=0 (Caused by ResponseError('too many 502 error responses'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/spotipy/client.py", line 264, in _internal_call
    response = self._session.request(
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 556, in send
    raise RetryError(e, request=request)
requests.exceptions.RetryError: HTTPSConnectionPool(host='api.spotify.com', port=443): Max retries exceeded with url: /v1/me/playlists?limit=10&offset=0 (Caused by ResponseError('too many 502 error responses'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 541, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 739, in async_device_update
    raise exc
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/config/custom_components/spotcast/sensor.py", line 116, in update
    resp = self.hass.data[DOMAIN]["controller"].get_playlists(
  File "/config/custom_components/spotcast/spotcast_controller.py", line 333, in get_playlists
    resp = client.current_user_playlists(limit=limit)
  File "/usr/local/lib/python3.10/site-packages/spotipy/client.py", line 635, in current_user_playlists
    return self._get("me/playlists", limit=limit, offset=offset)
  File "/usr/local/lib/python3.10/site-packages/spotipy/client.py", line 321, in _get
    return self._internal_call("GET", url, payload, kwargs)
  File "/usr/local/lib/python3.10/site-packages/spotipy/client.py", line 305, in _internal_call
    raise SpotifyException(
spotipy.exceptions.SpotifyException: http status: 429, code:-1 - /v1/me/playlists?limit=10&offset=0:
 Max Retries, reason: too many 502 error responses

[mfabiani53]

FYI after some hours of default sensor.py worked, the errors are not anymore in the log. So, i think, the errors lie in the modified sensor.py.

[fcusson]

Ok good to k kw. I wanted to give you an update but had a crazy week at work.

I did some documentation reading and the playlists API is on a different rate limit than the rest of the api. The problem is they don't say what it is.

I have a feeling it is based on the number of playlist fetched and not calls. Would explain the problem with your 429 error. This is still a guess at the moment.

[mfabiani53]

Ok i'll stay tuned....
But.... sad to say... this morning the errors came up again... always at the same hour as previously...

[CubicrootXYZ]

I also experience them a lot since a few weeks. When they come up spotcast is not working for a few hours.

[fcusson]

The problem seems to be specific to some regions. I did some research online and some developpers are stuck with always receiving 502 errors while others don't. Can you verify from your side if this endpoint works in the developper console?

[fcusson]

V5 handles sensors very differently and should no longer break on 502 error. This should improve the experience a lot.