-
Notifications
You must be signed in to change notification settings - Fork 102
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce Python plugin #549
base: master
Are you sure you want to change the base?
Conversation
Set AstNodeInfo entity hash in service.
Fix python dependecnies and narrowing error.
… bin folder to PATH.
…ions when using CodeCompass.
Fix declaration inside except block Handled starred variable in with statement
It seems like the original source has gone. The project https://github.com/threatstack/libmagic/ was deleted or made private.
(cherry picked from commit cd7d319)
Introduce process_file_content.
Remove unnecessary inner class usage persisting.
# Conflicts: # .gitlab/build-codecompass.sh # .gitlab/cc-env.sh # webgui/scripts/codecompass/view/codeBites.js # webgui/scripts/codecompass/view/component/Text.js
…Compass into rmfcnb-pythonplugin-symlink # Conflicts: # plugins/python/parser/src/scripts/cc_python_parser/parser.py
Update branch
Remove unnecessary inner class usage persisting. Add missing imported declaration to ImportedDeclaration
bcca4c9
to
52327cd
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First, I'd like to tell that this Python parser is a great work, I appreciate it. I think we should continue the review process and this will be a good language plugin.
plugins/python/parser/src/scripts/cc_python_parser/common/hashable_list.py
Outdated
Show resolved
Hide resolved
|
||
class HashableList(Generic[T], List[T]): | ||
def __hash__(self): | ||
return hash(e for e in self) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
return hash(e for e in self) | |
return hash(tuple(e for e in self)) |
In the previous version a generator object was hashed, so in practice every HashableList
object had the same hash value. We should create a tuple instead which is hashable and the hash value is really generated based on its members.
I think, however, that we should eliminate this HashableList
completely. In Python only hashable objects can be used for dictionary keys. And this HashableList
is used in this context. But it's not an elegant way of using lists as keys, because changing list elements ruins the dictionary. The only difference between tuple and list in Python is that tuples are not mutable thus can be used as keys. I think we should introduce this change:
# This is the current usage.
my_list = HashableList()
my_dict = {my_list: 42}
# We should do this.
my_list = []
my_dict = {tuple(my_list): 42}
Conversion to tuple would happen in this __hash__()
function anyways, so it is not a performance loss. I haven't reviewed the part here this HashableList
is used, so I'm not sure if using a tuple as key is really necessary.
plugins/python/parser/src/scripts/cc_python_parser/common/parser_tree.py
Show resolved
Hide resolved
plugins/python/parser/src/scripts/cc_python_parser/common/unique_list.py
Outdated
Show resolved
Hide resolved
plugins/python/parser/src/scripts/cc_python_parser/function_symbol_collector.py
Outdated
Show resolved
Hide resolved
plugins/python/parser/src/scripts/cc_python_parser/scope_manager.py
Outdated
Show resolved
Hide resolved
plugins/python/parser/src/scripts/cc_python_parser/symbol_collector.py
Outdated
Show resolved
Hide resolved
plugins/python/parser/src/scripts/cc_python_parser/symbol_collector.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have tested functionally the Python plugin with the following single-file Python script:
https://gist.github.com/mcserep/9df2edbb6767b9190e438f9ed7f3a944
Remarks for the parser:
- Parsing went smoothly, although took more time than expected. (
requests
anddatetime
modules were imported.) - During parsing a lot of debug messages were shown, which should be hidden by default.
Remarks for the service:
- Multiline documentation comments are not handled properly and treated as Python code (Invalid PythonAstNode ID message shown when clicking inside).
- Clicking on the
audit_submission
function works properly, but clicking on theget_user_name
function results in segfault. - Clicking on imported symbols (e.g.
requests
orrequests.get
) also displays an Invalid PythonAstNode ID. - CodeBites diagram is not shown properly.
Introduce skip file Small refactors
Introduce Interpret BuildAction type
No description provided.