uRequests and Pi PicoW

[captaincaden]

I'm having both success, and an intermittent problem with a code I've written on a Raspberry Pi PicoW utilizing network, and urequests and ujson. I've implemented try and exception blocks in the code, but yet, I seem to keep having a stall in getting data from urequests. Apologies in advance if this isn't the appropriate place to post this issue, I've tried StackOverflow but no responses.

I'm pretty sure I've narrowed the issue down to urequests being the problem, as I have exceptions everywhere else, and it seems that the only place that could be causing this issue is after the code calls urequests.get()

To summarize, this project is using a Pi Pico W, with a 4 digit, 7 segment display, running MicroPython. The goal of the project is to gather temperature data from an API for an industrial oven, and the multiplex that data to display on the 7 segment display. Using both cores with _thread to acheive both a constant check for temperature, and a constant looping of the temperature data on to the display to multiplex it. This all works great, and works consistently for a long time. However:

An issue continues to arise pretty much every time I use this project, but at unpredictable times, where the temperature data received stalls, but doesn't throw an exception. So the temperature value freezes (even though the network, and API are still both alive), but the multiplexing loop continues. Eventually it will begin to work again after a long amount of time passed, or the board is unplugged and restarted.

My suspicion is that it for some reason is taking an excessive amount of time to call urequests.get(), or that something happens that causes it to get hung up, but not throw an exception to cause the machine.reset().

Please note this code is still a major work in progress, so likely a lot of unneccesary things hanging around in places still that I haven't came back and tidied up yet. This project took a lot more testing and altering than I expected.

Heres the code as it runs currently for reference. The get_temp() function is where I'm 99% certain the issue is coming from. But, mostly asking if anyone has had any issues similar with urequests after running for a long period of time

`from machine import Pin
import _thread
import utime as time
import network
import urequests as requests
import ujson as json

led = Pin("LED", Pin.OUT)
displaydash = Pin(8, Pin.OUT) #dashpin -
display_list = [2, 3, 4, 5, 6, 7, 8] #a, b, c, d, e, f, g segments
colonpin = Pin(9, Pin.OUT) 
display_obj = [] #to create the display object
dig1 = Pin(10, Pin.OUT) #digit 1 
dig2 = Pin(11, Pin.OUT)
dig3 = Pin(12, Pin.OUT)
dig4 = Pin(13, Pin.OUT) #digit 4
global temp
temp = 0 

for seg in display_list:
    display_obj.append(Pin(seg, Pin.OUT))

arrSeg = [[1,1,1,1,1,1,0], # -> arrSeq[0] displays 0
          [0,1,1,0,0,0,0], # -> arrSeq[1] displays 1
          [1,1,0,1,1,0,1], # -> arrSeq[2] displays 2
          [1,1,1,1,0,0,1], # -> arrSeq[3] displays 3
          [0,1,1,0,0,1,1], # -> arrSeq[4] displays 4
          [1,0,1,1,0,1,1], # -> arrSeq[5] displays 5
          [1,0,1,1,1,1,1], # -> arrSeq[6] displays 6
          [1,1,1,0,0,0,0], # -> arrSeq[7] displays 7
          [1,1,1,1,1,1,1], # -> arrSeq[8] displays 8
          [1,1,1,1,0,1,1]]  # -> arrSeq[9] displays 9

def SegDisplay(toDisplay):
    numDisplay = int(toDisplay.replace(".", ""))
    for a in range(7):
        display_obj[a].value(arrSeg[numDisplay][a])

    if toDisplay.count("-") == 1:
         Pin(3,Pin.OUT).value(1)
    else:
        colonpin.value(0)
        
def clear(): #sets all display output pins to zero 
    Pin(2, Pin.OUT).value(0)
    Pin(3, Pin.OUT).value(0)
    Pin(4, Pin.OUT).value(0)
    Pin(5, Pin.OUT).value(0)
    Pin(6, Pin.OUT).value(0)
    Pin(7, Pin.OUT).value(0)
    Pin(8, Pin.OUT).value(0)
    Pin(9, Pin.OUT).value(0)
    colonpin.value(0)

def error(): #displays EEEE
    clear()
    dig1.value(1)
    dig2.value(1)
    dig3.value(1)
    dig4.value(1)
    Pin(9, Pin.OUT).value(0)
    Pin(8,Pin.OUT).value(1)
    Pin(7,Pin.OUT).value(1)
    Pin(6,Pin.OUT).value(1)
    Pin(5,Pin.OUT).value(1)
    Pin(2,Pin.OUT).value(1)
    dig1.value(0)
    dig2.value(0)
    dig3.value(0)
    dig4.value(0)
        
def dash(): #displays ----
    clear()
    dig1.value(1)
    dig2.value(1)
    dig3.value(1)
    dig4.value(1)
    Pin(9, Pin.OUT).value(0)
    Pin(8,Pin.OUT).value(1)
    dig1.value(0)
    dig2.value(0)
    dig3.value(0)
    dig4.value(0)
    
def outside_dash(): #displays -  -
    clear()
    dig1.value(1)
    dig2.value(1)
    dig3.value(1)
    dig4.value(1)
    Pin(9, Pin.OUT).value(0)
    Pin(8,Pin.OUT).value(1)
    dig1.value(0)
    dig2.value(1)
    dig3.value(1)
    dig4.value(0)
    

def display():
    refresh = 0.0025
    dig1.value(1)
    dig2.value(1)
    dig3.value(1)
    dig4.value(1)
    while True:
        global temp
        value = temp
        if value != None or value != 9998 or value != 9999:
            try:
                value = float(value)
                value = int(value) #not rounding, just getting whole number.
                value = str(value)
                if len(value)==1:
                    pl1 = "0"
                    pl2 = "0"
                    pl3 = "0"
                    pl4 = value[0]
                    dig1.value(1)
                    dig2.value(1)
                    dig3.value(1)
                    dig4.value(1)
                    SegDisplay(pl4)
                    dig4.value(0)
                    time.sleep(refresh)
                    dig4.value(1)
                elif len(value)==2:
                    pl1 = "0"
                    pl2 = "0"
                    pl3 = value[0]
                    pl4 = value[1]
                    dig1.value(1)
                    dig2.value(1)
                    dig3.value(1)
                    dig4.value(1)
                    SegDisplay(pl3)
                    dig3.value(0)
                    time.sleep(refresh)
                    dig3.value(1)
                    SegDisplay(pl4)
                    dig4.value(0)
                    time.sleep(refresh)
                    dig4.value(1)
                elif len(value)==3:
                    pl1 = "0"
                    pl2 = value[0]
                    pl3 = value[1]
                    pl4 = value[2]
                    dig1.value(1)
                    dig2.value(1)
                    dig3.value(1)
                    dig4.value(1)
                    SegDisplay(pl2)
                    dig2.value(0)
                    time.sleep(refresh)
                    dig2.value(1)
                    SegDisplay(pl3)
                    dig3.value(0)
                    time.sleep(refresh)
                    dig3.value(1)
                    SegDisplay(pl4)
                    dig4.value(0)
                    time.sleep(refresh)
                    dig4.value(1)
                    time.sleep(refresh)
                elif len(value)==4:
                    pl1 = value[0]
                    pl2 = value[1]
                    pl3 = value[2]
                    pl4 = value[3]
                    SegDisplay(pl1)
                    dig1.value(0)
                    time.sleep(refresh)
                    dig1.value(1)
                    SegDisplay(pl2)
                    dig2.value(0)
                    time.sleep(refresh)
                    dig2.value(1)
                    SegDisplay(pl3)
                    dig3.value(0)
                    time.sleep(refresh)
                    dig3.value(1)
                    SegDisplay(pl4)
                    dig4.value(0)
                    time.sleep(refresh)
                    dig4.value(1)
            except:
                dash()
                time.sleep(1)
                pass
        elif value == None or value == 9999:
            dash()
            time.sleep(1)
        elif value == 9998:
            error()
            time.sleep(1)
                
def connect():
    #Connect to WLAN
        clear()
        ssid = "networkname" #set up wifi network
        password = 'password' #set up wifi password
        wlan = network.WLAN(network.STA_IF)
        wlan.active(True)
        wlan.config(pm = 0xa11140) 
        wlan.connect(ssid, password)
        max_wait = 10
        while max_wait > 0:
            if wlan.status() < 0 or wlan.status() >= 3:
                break
            max_wait -= 1
            print('Waiting for connection...')
            led.on()
            time.sleep(.5)
            led.off()
            time.sleep(.5)

 # Handle connection error
        if wlan.status() != 3:
            raise RuntimeError('Network connection failed')
            error()
            led.off()
        else:
            led.on()
            print('Connected')
            status = wlan.ifconfig()
            print( 'IP = ' + status[0] )
            
def get_temp():
    temp_url = "the api url goes here" 
    while True:
        global temp
        wlan = network.WLAN(network.STA_IF)
        if wlan.status() != 3: #check if connected first
            led.off()
            print("Lost connection")
            temp = wlan.status()
            connect()
            time.sleep(0.25)
        elif wlan.status() == 3:
            try: #get temp if connected
                r = requests.get(temp_url).json()
                temp = r.get('ispoint')
                time.sleep(1)
            except: #if requests fail, reset 
                print("Couldn't gather data")
                temp = wlan.status()
                outside_dash()
                print(wlan.status())
                machine.reset()
                break
    

connect() #connect to wifi first       
_thread.start_new_thread(display, ()) #start the display thread
get_temp()`

[jimmo]

@captaincaden sorry about the delay responding.

Could you please confirm that you're using a recent build (i.e. a nightly from at least December). When you say it "stalls" what does that mean exactly?

I suspect you could be right about the request timing out. Have you tried passing the timeout kwarg to the requests.get() method?

i.e.

r = requests.get(temp_url, timeout=1)  # 1 second

[captaincaden]

Hi @jimmo ! No worries. I appreciate your response!

I believe I’m using a recent build, but will double check.

Stalling implying that the temp variable stops updating, and never starts again until the machine is restarted manually, or until the code forces a restart from an exception of requests.get() (which sometimes happens, and sometimes doesn’t) . The temp variable is updated every second from the requests.get() to the API url. The code itself continues, which is obvious because the 7 segment display loop still continues to run, (it’s multiplexed, so if it stopped, only one digit would display) but the temp variable stays constant, despite the API still being accessible, and the temp value changing.

I have not tried the timeout kwarg, but I will add it later and see what happens. That sounds like what will resolve this problem. I’m unfamiliar with the requests library in general, and even more unfamiliar with all of the kwargs for urequests, and I wasn’t able to find documentation easily, so I was unaware that kwarg existed, but will update when I add it!

This problem has been very intermittent, so proving that it’s not happening anymore may take some time.

Again, thank you for responding, and for the help!

[captaincaden]

Adding the timeout kwarg to .get() and then using machine.reset() if it does timeout has seemed to correct the issue. When it freezes, it very quickly resets and displays the temp correctly again. Just have to do some testing. Thank you again!

[MartinH-Armadeus]

Hi, captaincaden,
came across your post, having the same problem. After a day or two, response = urequests.get(status_url) seems to stall for some hours.
More than a year later than the first post, I am on FW version RPI_PICO_W-20240602-v1.23.0 and it still looks the same.
Now trying RPI_PICO_W-20240904-v1.24.0-preview.286.g042693496. No clue yet, always takes more than a day to see the issue.

Did you find a solution in the meantime?

[captaincaden]

@MartinH-Armadeus I’ve made a couple pico w projects that use requests since and they all seem to eventually have this issue at some point.

Using the built in watch dog timer in machine has been the only reliable way to deal with this problem for me. Read up on machine.WDT. Make it feed the watch dog timer every time you call to make a request. If it hangs up and doesn’t feed the watchdog the timer will run out and force the pico to reboot.

[MartinH-Armadeus]

Thanks for the reply. If it hangs again with the latest FW, I will do the same. If it does not hang, I will let you know.

[captaincaden]

Thank you! In case you wanted an idea of how I use it in reference to one of my earlier comments:

Somewhere in my constants I set:
from machine import WDT
wdt = WDT(timeout = 7900) #in milliseconds
wdt.feed()

and then in the get_temp() while loop:
r = requests.get(temp_url).json()
temp = r.get('ispoint')
wdt.feed() #sometime after we've sent the request, that way if it hangs on that line the watchdog never gets fed, and timer runs out.
time.sleep(1)

The watchdog timer fully restarts the program. I have some other checks I added that help keep an eye on the connection, but ultimately, requests will still hang at some point, even using the timeout in requests. Try and except blocks (including with specific exceptions for response and connection errors with a machine.reset) never reliably resolved it either.

[MartinH-Armadeus]

Following your advice, I have set up the WDT. This is a great fallback.
Meanwhile I am on FW version RPI_PICO_W-20240924-v1.24.0-preview.349.g51974f20e.uf2
With this version, the following code is now running since 5 days without a single WDT event:

wdt = machine.WDT(timeout = 8388) # in in milliseconds, max value

def get_dect200_response(self):
    status_url = f'{self.FRITZBOX_URL}/webservices/homeautoswitch.lua?sid={self.sid}&switchcmd=getswitchstate&ain={self.AIN}'
    #print ("Query switch:",status_url )
    wdt.feed()
    try:
        print(":", end="")
        response = urequests.get(status_url, timeout=7)
        #print("State Response:", response.status_code,  response.text)
        if response is None:
            print("-", end="")
        return response

    except OSError  as e:
        if e.errno == 12:  # ENOMEM
            self.dectstst = 0
            print("m", end="")
            myserver.memory_errors += 1
            return None
        else:
            print("EXCEPTION Status Reqest No: ", e.errno)
            print("EXCEPTION ",repr(e))
            self.login_fritzbox(self.USERNAME, self.PASSWORD)
            if self.sid:
                wdt.feed()
                try:
                    response = urequests.get(status_url, timeout=7)
                    print("Retry Switch Response:", response.status_code,  response.text)
                    return response
                except: # Exception as e:
                    print("EXCEPTION Status Retry Reqest: ")
                    self.dectstst = 0
                    print("r", end="")
                    return None
            else:
                print("No SID")
                self.dectstst = 0
                return None

The exception handler is now ctaching 3 types of errors:

• OSError 12, if the RP2040 is getting overloaded, e.g. by 2 clients calling the web interface simulatneously
• OSError -2 , 17 times in 5 days.
• OSError 110: 7 times in 5 days
  For OSError other than 12, I re-connect my server. This seems to work. No hangup any more, that triggered the WDT.

[sosi-deadeye]

Just an improvement of your code, to reduce the source code size. It does not solve the other problems.
For example, if you create Pin objects, then reuse them.

...

# definition of pins and setting
# explicit the state of the pin with value
dig1 = Pin(10, mode=Pin.OUT, value=0)
dig2 = Pin(11, mode=Pin.OUT, value=0)
dig3 = Pin(12, mode=Pin.OUT, value=0)
dig4 = Pin(13, mode=Pin.OUT, value=0)

# put the pins in a list for later use:
dig_pins = [dig1, dig2, dig3, dig4]

# the other pins in a dict, to address them with their p
other_pins = [
    Pin(2, mode=Pin.OUT, value=0),
    Pin(3, mode=Pin.OUT, value=0),
    Pin(4, mode=Pin.OUT, value=0),
    Pin(5, mode=Pin.OUT, value=0),
    Pin(6, mode=Pin.OUT, value=0),
    Pin(7, mode=Pin.OUT, value=0),
    Pin(8, mode=Pin.OUT, value=0),
    Pin(9, mode=Pin.OUT, value=0),
]

# pay attention, that the ERROR_MASK has the same amount
# of elements as other_pins
ERROR_MASK = (0, 1, 1, 1, 1, 1)


def error():
    """
    displays EEEE
    """
    clear()
    for pin in dig_pins:
        pin.value(1)
    
    for pin, value in zip(other_pins, ERROR_MASK):
        pin.value(value)
    
    for pin in dig_pins:
        pin.value(0)