From e4cdf018a964a226f18e8b87e31ee449de578392 Mon Sep 17 00:00:00 2001 From: Alexander Clausen Date: Thu, 30 Nov 2023 10:25:21 +0100 Subject: [PATCH 1/3] WIP: documentation updates * Clarify `LiveContext` lifetime * Add section on `run_udf_iter` --- docs/source/usage.rst | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/docs/source/usage.rst b/docs/source/usage.rst index 467587e3..2082969e 100644 --- a/docs/source/usage.rst +++ b/docs/source/usage.rst @@ -64,7 +64,9 @@ which is the :class:`~libertem_live.api.LiveContext`: ctx = LiveContext() This creates the appropriate resources for computation, in other words, it -starts worker processes and prepares them for receiving data. +starts worker processes and prepares them for receiving data. As it may take a +while for the :class:`~libertem_live.api.LiveContext` to spawn the worker processes, +it's best to keep the instance around as long as possible. The next step is to prepare a connection to the detector system; in most cases you'll specify network hostnames, IP addresses and/or ports here. @@ -405,6 +407,34 @@ In some cases, updating the plot can become a bottleneck - one way to circumvent this is to use `bqplot` for visualization. Please see :ref:`the examples ` for usage. +Handling partial results +------------------------ + +If you'd like to access partial results while the acquisition is running, +you can use :meth:`libertem:libertem.api.Context.run_udf_iter`. It returns +a generator of :class:`libertem:libertem.udf.base.UDFResults`, which you +can use like this: + + +.. testcode:: + + import contextlib + + part_iter = ctx.run_udf_iter(dataset=aq, udf=SumUDF()) + + for part_result in contextlib.closing(part_iter): + sum_arr = part_result.buffers[0]['intensity'] + + +:code:`part_result.buffers` is a list that has an entry for each UDF that is +run, so in this case, it only has one entry for the :class:`~libertem.udf.sum.SumUDF`. +Each list entry is a :code:`dict` of buffer names to their values. + +.. note:: + + Note the use :func:`contextlib.closing` - this makes sure that resources + are reliably released in case the iterator is not fully consumed. + Included UDFs ------------- From ec2dd1ee0df604e79358f5f76221a44cb95d0ebb Mon Sep 17 00:00:00 2001 From: Alexander Clausen Date: Thu, 30 Nov 2023 10:48:14 +0100 Subject: [PATCH 2/3] Add more complete test code --- docs/source/usage.rst | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/docs/source/usage.rst b/docs/source/usage.rst index 2082969e..79627c9e 100644 --- a/docs/source/usage.rst +++ b/docs/source/usage.rst @@ -417,13 +417,30 @@ can use like this: .. testcode:: + :skipif: not HAVE_DECTRIS_TESTDATA import contextlib - part_iter = ctx.run_udf_iter(dataset=aq, udf=SumUDF()) + with ctx.make_connection('dectris').open( + api_host="127.0.0.1", + api_port=DCU_API_PORT, + data_host="127.0.0.1", + data_port=DCU_DATA_PORT, + ) as conn: + aq = ctx.make_acquisition( + conn=conn, + nav_shape=(128, 128), + controller=conn.get_active_controller( + # NOTE: parameters here are detector-specific + trigger_mode='exte', + frame_time=1e-3, + ), + ) + + part_iter = ctx.run_udf_iter(dataset=aq, udf=SumUDF()) - for part_result in contextlib.closing(part_iter): - sum_arr = part_result.buffers[0]['intensity'] + for part_result in contextlib.closing(part_iter): + sum_arr = part_result.buffers[0]['intensity'] :code:`part_result.buffers` is a list that has an entry for each UDF that is From 79226f550b73dabbb6c4095548cc1b5e12fa19e3 Mon Sep 17 00:00:00 2001 From: Alexander Clausen Date: Thu, 30 Nov 2023 19:41:30 +0100 Subject: [PATCH 3/3] Maybe that's how closing works? --- docs/source/usage.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/source/usage.rst b/docs/source/usage.rst index 79627c9e..52d9011c 100644 --- a/docs/source/usage.rst +++ b/docs/source/usage.rst @@ -439,8 +439,9 @@ can use like this: part_iter = ctx.run_udf_iter(dataset=aq, udf=SumUDF()) - for part_result in contextlib.closing(part_iter): - sum_arr = part_result.buffers[0]['intensity'] + with contextlib.closing(part_iter): + for part_result in part_iter: + sum_arr = part_result.buffers[0]['intensity'] :code:`part_result.buffers` is a list that has an entry for each UDF that is