tm1project_put in GitService not working properly

[nicolasbisurgi]

Describe the bug
When attempting to put a tm1project using the tm1project_put function under GitService, I get an error message stating:

  File "C:\Python381\lib\site-packages\TM1py\Services\RestService.py", line 68, in wrapper
    raise ValueError(f"Failed to retrieve async_id from request {func.__name__} '{url}'")
ValueError: Failed to retrieve async_id from request PUT 'https://mycompany.planning-analytics.cloud.ibm.com/tm1/api/tm1/api/v1/!tm1project'

This happens ONLY when async_requests_mode=True.

To Reproduce

tm1_project_source_file = r'C:\tm1project.json'
with TM1Service(
        base_url='https://mycompany.planning-analytics.ibmcloud.com/tm1/api/tm1/',
        user="non_interactive_user",
        namespace="LDAP",
        password="U3lSn5QLwoQZY2",
        ssl=True,
        verify=True,
        async_requests_mode=True) as tm1:
    tm1_project = GitProject.TM1Project().from_file(tm1_project_source_file)
    tm1.git.tm1project_put(tm1_project)

To Avoid This Behavior
Under the GitService.py you can force the PUT function to use async_requests_mode=False:

def tm1project_put(self, tm1_project: TM1Project) -> TM1Project:
    url = '/api/v1/!tm1project'
    body_json = tm1_project.body
    
    response = self._rest.PUT(url=url, data=body_json, async_requests_mode=False)
    return TM1Project.from_dict(response.json())

Expected behavior
The response from the server does not have a Location section in its headers, so I'm not sure if the response if wrong or if we should not expect it at all thus, enforce the async_requests_mode=False in the function above.

At any point, the value of async_request_mode should not be relevant for this operation.

Version
TM1py 1.10.2
TM1 Server Version: 11.8.01700.1

[MariusWirtz]

Thanks for raising the issue.
Does TM1 raise the same error when doing a put with async_requests_mode=True against an on-premise TM1 instance?

I agree. Your suggested fix seems the most appropriate solution to the problem.

[nicolasbisurgi]

Good question: I checked against our OnPrem (same PA version as cloud but running on RHEL) and it does not fail, the only connection parameter that differs from the one to cloud is the ssl as it is set to False in our OnPrem installations.
Do you think the OS or the SSL param could have some relation with this?

[MariusWirtz]

I doubt it's related to the SSL. I think most likely due to the implementation of the TM1 API in PAoC. I have seen on multiple occasions that the API behaved slightly more fragile than on-premise. For instance, when using write_async with a few threads it occasionally fails to return a response (I tend to work around it with a retry). I hadn't reported it to IBM yet for a lack of a concise reproducible example, but I understand there is some extra layer in that Architecture that is a weak link in the chain.
My guess would be that the PUT isn't implemented well in that layer for the async_requests_mode.
This PUT request should be reproducible. I think we could raise a ticket with IBM to address it.

[nicolasbisurgi]

Hi @MariusWirtz ,
I'm sorry but I was incorrect in my prior statement: it also fails if executed against an OnPrem instance (I used a wrong config.ini file when testing).
Just to note, the PUT call that places the tm1project is successfully completed, but the Location header is missing from the response.
I'll raise a ticket to IBM to see what they say. In the meantime I can put this workaround in place. Sounds good?

[MariusWirtz]

Thank you! Please open a PR for the fix.

Yes, raising a ticket with IBM will be a good long-term solution.

[nicolasbisurgi]

Hi @MariusWirtz ,
I raised a ticket with IBM and below is their answer:

respond-async is a PREFERENCE, the service is free to ignore it. This is standard HTTP protocol. If the service honors the preference it will set the Preference-Applied header accordingly (which the client could check) and also, in the case of respond-async, return a 202 Accepted status code instead of another success or failure code. The client will have to act accordingly.
Putting to the project file resource is one of those operations, which typically happens instantaneously anyway, that the service doesn't execute asynchronously. So this is expected.

Though I understand the PREFERENTIAL behavior of the asyn-request, I find it a bit hard to understand why this function (PUT tm1project) is the only one facing this issue; does it mean that all other functions that TM1Py executes with respond-async are honored by the service and this one does not? Do we have a proper handling of the 202 return code for async requests?

[MariusWirtz]

Thank you.

Does it mean that all other functions that TM1Py executes with respond-async are honored by the service and this one does not?

Yes!

I think we must change the implementation of the async response handling in TM1py so that, potentially, the request returns a normal response instead of async-id, even though this is the only function that behaves this way.

[MariusWirtz]

Solved with 6105399