-
Notifications
You must be signed in to change notification settings - Fork 651
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DOCS-#7144: Add information about logging from user defined function #7155
Conversation
…ned functions Signed-off-by: arunjose696 <arunjose696@gmail.com>
def Udf(df, partition_shape, *dfs): | ||
from modin.config import LogMode, LogMemoryInterval, LogFileSize | ||
LogMode.enable() | ||
# User code goes here | ||
return |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's declare a udf function and pass it in map or apply method like the user interacts with pandas API. The udf will be executed in a remote function.
def udf(x):
logger stuff
return x + 1
modin_df.map(udf)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
|
||
.. code-block:: python | ||
|
||
@ray.remote |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's declare a udf function and pass it in map or apply method like the user interacts with pandas API. The udf will be executed in a remote function.
def udf(x):
logger stuff
return x + 1
modin_df.map(udf)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
e7779e8
to
8661f8b
Compare
Co-authored-by: Iaroslav Igoshev <Poolliver868@mail.ru> Signed-off-by: arunjose696 <arunjose696@gmail.com>
8661f8b
to
9c252d1
Compare
|
||
|
||
.. warning:: | ||
When attempting to use Modin logging in user defined functions that execute in workers for logging lower-level operators |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Check indents here. There should be 4 spaces like we have in modin/docs/flow/modin/core/dataframe/algebra.rst:L62.
Co-authored-by: Iaroslav Igoshev <Poolliver868@mail.ru> Signed-off-by: arunjose696 <arunjose696@gmail.com>
6bd05b6
to
ad3f011
Compare
What do these changes do?
flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py
black --check modin/ asv_bench/benchmarks scripts/doc_checker.py
git commit -s
docs/development/architecture.rst
is up-to-date