Async callback with custom object causes invalid reference counting handling

[haukero]

Hi All,

the following code makes RPyC 3.3.0 crash on my machine because of too early reference counting decrease.

Please note the following conditions:

• Synchronous client call that passes a callback function to the server.
• Asynchronous callback (event) that calls the received callback function.
• The event passes an object back to the client whose class definition is unknown on the client side.

After debugging the code and tracing the protocol, it seems that the callback function is requested to be deleted from the local objects dictionary on the client side,
before the callback on this function is actually executed. And the callback call is actually delayed because the parameter class definition is unknown on the client side.

Unfortunately, I haven't figured out where to fix the error.

I would assume that the delete request needs to be delayed until the callback finished. Note that the callback is asynchronous.

Thanks for any help,
Rico

Server code:

import rpyc
import os

class URL (str):
    pass

class DownloadService (rpyc.Service):
    def exposed_download_url (self, url, finished):
        a = rpyc.async(finished)
        a(URL(url))

if __name__ == '__main__':
    print os.getpid()
    from rpyc.utils.server import ThreadedServer
    serv = ThreadedServer(DownloadService, port=18812, protocol_config={"allow_all_attrs": True})
    serv.start()

Client code:

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
import logging
import rpyc

def finished (self, url):
    print "Finished."

if __name__ == '__main__':
    url = "http://www.car-it.com/wp-content/uploads/2014/02/13C1150_092.jpg"
    logging.basicConfig(level=logging.DEBUG)
    log = logging.getLogger(__name__)
    con = rpyc.connect("localhost", port=18812, config={"allow_all_attrs": True, "logger": log})
    con.root.download_url(url, finished)
    # Keep alive.
    while True:
        pass

Client side exception:

pydev debugger: starting (pid: 1988)
DEBUG:__main__:Exception caught
Traceback (most recent call last):
  File "C:\Program Files (x86)\Python27\lib\site-packages\rpyc\core\protocol.py", line 305, in _dispatch_request
    res = self._HANDLERS[handler](self, *args)
  File "C:\Program Files (x86)\Python27\lib\site-packages\rpyc\core\protocol.py", line 535, in _handle_call
    return self._local_objects[oid](*args, **dict(kwargs))
  File "C:\Program Files (x86)\Python27\lib\site-packages\rpyc\lib\colls.py", line 86, in __getitem__
    return self._dict[key][0]
KeyError: 40805104

Protocol trace (seq id, msg id):

Client:                                                                         Server:
-------                                                                           -------

--> REQ                                                                        (0, 1): GetRoot()
(0, 2): (41243912, 'DownloadService', 'pkgs.dlserv')           <-- RESP

--> REQ                                                                        (1, 1): Inspect(41243912)
(1, 2): <List of methods>                                                <-- RESP

--> REQ                                                                        (2, 1): GetAttr(41243912, 'download_url')
(2, 2): (38883696, 'instancemethod', '__builtin__')              <-- RESP

--> REQ                                                                        (3, 1): Call(38883696, url, finished=(40805104, 'function', '__builtin__'))

(0, 1): Call(40805104, url=(41284640, 'URL', '__main__'))    <-- REQ

(3, 2): None                                                                    <-- RESP

(1, 1): Del(40805104)                                                       <-- REQ

--> REQ                                                                         (4, 1): Inspect(41284640)
(4, 2): <Class description>                                               <-- RESP

--> RESP                                                                       (1, 2): None

--> EXCP                                                                       (0, 3): KeyError(40805104)

[coldfix]

Seems to have been solved by 4c8f004 which will be part of the next release. Not sure how reliably.

BTW: client should read:

def finished (url):
    print "Finished."

(without self parameter)

Let me know, if you continue to have problems.

Best, Thomas