Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Part I: Deprecate & Rename Old Routes #64

Merged
merged 2 commits into from
Sep 7, 2023

Conversation

bh2smith
Copy link
Collaborator

@bh2smith bh2smith commented Sep 5, 2023

the functions touched here are:

  • refresh --> run_query
  • refresh_csv --> run_query_csv
  • refresh_into_data_frame --> run_query_dataframe
  • get_status --> get_execution_status
  • execute --> execute_query
  • get_result --> get_execution_results

Maybe a couple others I forgot.

We had to temporarily add a pylint: ignore too-many-public-methods but this will be lifted when we split the client into

DuneExecutionsAPI, DuneQueryAPI, DuneUploadAP - See #72

cc @TheEdgeOfRage & @diegoximenes for additional review.

@bh2smith bh2smith mentioned this pull request Sep 5, 2023
@bh2smith bh2smith force-pushed the deprecate-old-routes branch 3 times, most recently from 7b96972 to 4b34406 Compare September 5, 2023 20:20
@bh2smith bh2smith requested a review from msf September 6, 2023 08:13
Copy link
Collaborator

@msf msf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for this, this is in the right direction, I didn't aprove yet because I've found myself with some questions that I left in the PR.

response = self._get(route=route, raw=True)
response.raise_for_status()
return ExecutionResultCSV(data=BytesIO(response.content))
return self.get_execution_results_csv(job_id)

def get_latest_result(self, query: Union[QueryBase, str, int]) -> ResultsResponse:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deprecate and replace w/ get_query_result() ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is different than get_query_result. Get latest, fetches the latest results buy query ID (as opposed to execution/job ID).

I suppose we could do as you suggest, but I personally find the two to be rather different.

cf:

https://dune.com/docs/api/api-reference/latest_results/

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so I don't forget, I need to review this again, but we should have a get_query_result() that uses a Query and simply returns the latest available query result (using the API that exists for that).
This method has the clear difference that it doesnt use up any execution credits, which is important.

raise ImportError(
"dependency failure, pandas is required but missing"
) from exc
data = self.refresh_csv(query, performance=performance).data
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the other way around, make refresh_csv() use this function instead.

also, what do you think about changing the ordering of the methods in the class?
shall we move:

  • old deprecated functions go AFTER the new official ones
  • internal helper methods or functions go to the end of the file (or to a separate helper class altogether?)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the other way around, make refresh_csv() use this function instead.

Yes, this was just overlooked, will fix.

  • I was saving the reordering for the larger refactor (when we split the class into multiple subclasses). This is already proposed in PR Refactor Part II: Split DuneClient Class #72 (where the deprecated functions have been moved to the bottom of their respective new locations).

  • Also, internal helper methods have been moved (in Refactor Part II: Split DuneClient Class #72) to a class called ExtendedAPI (naming can easily be changed upon request).

@@ -148,7 +148,7 @@ def test_internal_error(self):
query.query_id = 9999999999999

with self.assertRaises(DuneError) as err:
dune.execute(query)
dune.execute_query(query)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now, looking at the code and having refreshed my memory:

question for you:

  • if we have a QueryBase or Query object, shouldn't we have the methods on that class:
  • query.execute()
  • query.result()
  • etc.. ?

with functions like dune.execute_query() it feels more idiomatic this way:

res = dune.execute_query(query_id=1234)

instead of:

res = dune.execute_query(Query(1234))

for me, the two idiomatic options would be:

  • dune.execute_query(1234) # or query_id=1234
  • dune.Query(1234).execute()

Right now, I see bits of OO in here, but inconsistently:

  • Executions are simply job_id
  • Queries are objects, with a query_id inside (it should be Query.id, right?)
    • but we don't have methods on Queries..
    • if we did, it would add structure to expand the library for varying types of APIs and do a cleaner split between concepts.

this is just food for thought, the goal here is to improve the code for readability and dev experience in a simple, pragmatic way.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I definitely agree some things like this could be approached differently. In all of the ways suggested above. These things have been on my mind for quite some time. I believe the reason we went for query_id is that some of these low level methods are never actually used by the end user and none of the additional query attributes are used. Most consumers of the client are usually calling refresh.

Maybe it can be made compatible with multiple approaches (i.e. introduce some method overrides where users can pass Query object or query_id). I would have to think a bit further if we can extend the functionality to the Query object itself in a backwards compatible way, but I would be wiling to give it a try.

@bh2smith bh2smith force-pushed the deprecate-old-routes branch from 202f26d to 6bc5f80 Compare September 6, 2023 12:33
@bh2smith
Copy link
Collaborator Author

bh2smith commented Sep 6, 2023

I've found myself with some questions that I left in the PR.

I hope some of the questions are now answered (or in some cases at least postponed to the next PR).

@bh2smith bh2smith force-pushed the deprecate-old-routes branch from 10b15a0 to c21da35 Compare September 7, 2023 07:31
@bh2smith bh2smith changed the title Deprecate and Rename Old Routes Refactor Part I: Deprecate & Rename Old Routes Sep 7, 2023
@bh2smith bh2smith requested a review from msf September 7, 2023 08:48
@bh2smith bh2smith merged commit 36cb0d0 into main Sep 7, 2023
@bh2smith bh2smith deleted the deprecate-old-routes branch September 7, 2023 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants