Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement CALL_FUNCTION adaptive interpreter optimizations #88691

Closed
Fidget-Spinner opened this issue Jun 28, 2021 · 11 comments
Closed

Implement CALL_FUNCTION adaptive interpreter optimizations #88691

Fidget-Spinner opened this issue Jun 28, 2021 · 11 comments
Labels
3.11 only security fixes pending The issue will be closed if no feedback is provided performance Performance or resource usage

Comments

@Fidget-Spinner
Copy link
Member

Fidget-Spinner commented Jun 28, 2021

BPO 44525
Nosy @markshannon, @pablogsal, @sobolevn, @Fidget-Spinner, @thesamesam
PRs
  • bpo-44525: Specialize CALL_FUNCTION for C function calls #26934
  • bpo-44525: Specialize simple Python calls. #29033
  • bpo-44525: Add recursive checks for CALL_FUNCTION_BUILTIN_O #29271
  • bpo-44525: Copy freevars in bytecode to allow calls to inner functions to be specialized #29595
  • bpo-44525: Specialize for calls to type and other builtin classes with 1 argument. #29942
  • bpo-44525: Split calls into PRECALL and CALL #30011
  • bpo-44525: Document new call opcodes for 3.11 #30107
  • bpo-44525: Specialize calls to normal python classes #30415
  • bpo-46709: fix race conditions in unittest/test_brake #31273
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2021-06-28.14:22:18.192>
    labels = ['3.11', 'performance']
    title = 'Implement CALL_FUNCTION adaptive interpreter optimizations'
    updated_at = <Date 2022-02-11.11:31:07.338>
    user = 'https://github.com/Fidget-Spinner'

    bugs.python.org fields:

    activity = <Date 2022-02-11.11:31:07.338>
    actor = 'sobolevn'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = []
    creation = <Date 2021-06-28.14:22:18.192>
    creator = 'kj'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 44525
    keywords = ['patch']
    message_count = 9.0
    messages = ['396639', '404376', '404470', '405108', '405192', '406831', '408554', '408609', '409779']
    nosy_count = 5.0
    nosy_names = ['Mark.Shannon', 'pablogsal', 'sobolevn', 'kj', 'thesamesam']
    pr_nums = ['26934', '29033', '29271', '29595', '29942', '30011', '30107', '30415', '31273']
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'performance'
    url = 'https://bugs.python.org/issue44525'
    versions = ['Python 3.11']

    Linked PRs

    @Fidget-Spinner
    Copy link
    Member Author

    CALL_FUNCTION can be specialized. Copying from Mark's comments faster-cpython/ideas#54 (comment)

    There are a number of specializations of CALL_FUNCTION that make sense:
    
    1. Calls to a Python function where the arguments and parameters match exactly.
    2. Calls to a Python function with keyword arguments, or defaults, when the argument shuffle can be pre-computed.
    3. Calls to builtin functions.
    4. Calls to specific builtin functions, specifically len and instance.
    5. Calls to type with a single argument.
    6. Instantiation of a "normal" class, that is a class that does not redefine __new__ and whose metaclass is type.
    

    I've prepared a PR that should speed up non-keyword calls to PyCFunction. It covers specializations 3 and 4. Stable pyperf microbenchmarks show 5-15% less call overhead for some PyCFunctions. Please see faster-cpython/ideas#54 (comment) for the benchmark script and results.

    This issue is also tied to bpo-44207 (Add a version number to Python functions) which will be required for specializations 1 and 2.

    @Fidget-Spinner Fidget-Spinner added 3.11 only security fixes performance Performance or resource usage labels Jun 28, 2021
    @markshannon
    Copy link
    Member

    New changeset 3163e68 by Ken Jin in branch 'main':
    bpo-44525: Specialize CALL_FUNCTION for C function calls (GH-26934)
    3163e68

    @markshannon
    Copy link
    Member

    New changeset 8863a0f by Mark Shannon in branch 'main':
    bpo-44525: Specialize simple Python calls. (GH-29033)
    8863a0f

    @pablogsal
    Copy link
    Member

    Unfortunately, PR 26934 has broken thes 390x RHEL7 LTO 3.x buildbot as you can see before. As per the buildbot maintenance procedures, we will need to revert this PR unless is fixed in 24 hours.

    @markshannon @Fidget-Spinner

    @markshannon
    Copy link
    Member

    New changeset 0a1a36b by Ken Jin in branch 'main':
    bpo-44525: Add recursive checks for CALL_FUNCTION_BUILTIN_O (GH-29271)
    0a1a36b

    @markshannon
    Copy link
    Member

    New changeset 135cabd by Mark Shannon in branch 'main':
    bpo-44525: Copy free variables in bytecode to allow calls to inner functions to be specialized (GH-29595)
    135cabd

    @markshannon
    Copy link
    Member

    New changeset 9f8f451 by Mark Shannon in branch 'main':
    bpo-44525: Split calls into PRECALL and CALL (GH-30011)
    9f8f451

    @markshannon
    Copy link
    Member

    New changeset 3a60bfe by Mark Shannon in branch 'main':
    bpo-44525: Specialize for calls to type and other builtin classes with 1 argument. (GH-29942)
    3a60bfe

    @markshannon
    Copy link
    Member

    See faster-cpython/ideas#210

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @iritkatriel
    Copy link
    Member

    Is there anything left to do on this?

    @iritkatriel iritkatriel added the pending The issue will be closed if no feedback is provided label Sep 9, 2022
    @pablogsal
    Copy link
    Member

    Closing. Please feel free to re-open if we missed anything.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.11 only security fixes pending The issue will be closed if no feedback is provided performance Performance or resource usage
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants