Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

convert_lightgbm appears to be stuck in an infinite loop for a multiclass model #482

Closed
cabillin opened this issue Jul 21, 2021 · 1 comment · Fixed by #491
Closed

convert_lightgbm appears to be stuck in an infinite loop for a multiclass model #482

cabillin opened this issue Jul 21, 2021 · 1 comment · Fixed by #491

Comments

@cabillin
Copy link

Python 3.8.8
onnxmltools 1.7.0 (last we checked, higher versions break the output data type we need--the was bug fixed and then reintroduced)

We have an example that has been running/stuck for 4.5 days.
I started another run yesterday with a debugger attached. When I pause the process, I see a deep stack with modify_tree_for_rule_in_set & it's recursive_call method:
image

The lightgbm model.txt file is 251MB (Attachment of the full file failed--I assume because it was too big, but I've attached a file containing just the 1st 10 trees. Please reach out for the model.txt if needed for repro).
multiclass_lgbm_model_truncated_trees.txt

@xadupre
Copy link
Collaborator

xadupre commented Aug 3, 2021

I was able to convert the first 10 trees. I think there come from another tree. Is it possible to change the following lines in the lightgbm.py (see PR #487).

    if 'tree_info' in gbm:
        for it, tree in enumerate(gbm['tree_info']):
            print("modify_tree_for_rule_in_set: tree %d" % it)
            modify_tree_for_rule_in_set(tree, use_float=use_float)
        return

And copy paste the tree corresponding to the last number. It should be the tree for which the infinite loop happens.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants