Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Thread 'tokio-runtime-worker' panicked at 'Failed to allocate memory: "Allocator ran out of space" #3090

Closed
caetanix opened this issue May 24, 2021 · 12 comments

Comments

@caetanix
Copy link

caetanix commented May 24, 2021

====================
 
Version: 0.9.1-d2fb28838-x86_64-linux-gnu
 
  0: sp_panic_handler::set::{{closure}}
  1: std::panicking::rust_panic_with_hook
            at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/panicking.rs:595:17
  2: std::panicking::begin_panic_handler::{{closure}}
            at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/panicking.rs:497:13
  3: std::sys_common::backtrace::__rust_end_short_backtrace
            at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/sys_common/backtrace.rs:141:18
  4: rust_begin_unwind
            at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/panicking.rs:493:5
  5: core::panicking::panic_fmt
            at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/core/src/panicking.rs:92:14
  6: core::option::expect_none_failed
            at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/core/src/option.rs:1300:5
  7: tracing::span::Span::in_scope
  8: <<sp_io::allocator::HostFunctions as sp_wasm_interface::HostFunctions>::host_functions::ExtAllocatorMallocVersion1 as sp_wasm_interface::Function>::execute
  9: <sc_executor_wasmi::FunctionExecutor as wasmi::host::Externals>::invoke_index
 10: wasmi::func::FuncInstance::invoke
 11: wasmi::runner::Interpreter::start_execution
 12: wasmi::func::FuncInstance::invoke
 13: <sc_executor_wasmi::WasmiInstance as sc_executor_common::wasm_runtime::WasmInstance>::call
 14: sc_executor_common::wasm_runtime::WasmInstance::call_export
 15: std::panicking::try
 16: std::thread::local::LocalKey<T>::with
 17: sc_executor::native_executor::WasmExecutor::with_instance::{{closure}}
 18: sc_executor::wasm_runtime::RuntimeCache::with_instance
 19: <sc_executor::native_executor::NativeExecutor<D> as sp_core::traits::CodeExecutor>::call
 20: sp_state_machine::execution::StateMachine<B,H,N,Exec>::execute_aux
 21: sp_state_machine::execution::StateMachine<B,H,N,Exec>::execute_using_consensus_failure_handler
 22: <sc_service::client::call_executor::LocalCallExecutor<B,E> as sc_client_api::call_executor::CallExecutor<Block>>::contextual_call
 23: <sc_service::client::client::Client<B,E,Block,RA> as sp_api::CallApiAt<Block>>::call_api_at
 24: sp_api::runtime_decl_for_Core::execute_block_call_api_at
 25: <polkadot_runtime::RuntimeApiImpl<__SR_API_BLOCK__,RuntimeApiImplCall> as sp_api::Core<__SR_API_BLOCK__>>::Core_execute_block_runtime_api_impl
 26: sp_api::Core::execute_block_with_context
 27: sc_service::client::client::Client<B,E,Block,RA>::prepare_block_storage_changes
 28: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
 29: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
 30: <sc_consensus_babe::BabeBlockImport<Block,Client,Inner> as sp_consensus::block_import::BlockImport<Block>>::import_block::{{closure}}
 31: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
 32: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
 33: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
 34: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
 35: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
 36: <sc_service::task_manager::prometheus_future::PrometheusFuture<T> as core::future::future::Future>::poll
 37: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
 38: <tracing_futures::Instrumented<T> as core::future::future::Future>::poll
 39: std::thread::local::LocalKey<T>::with
 40: futures_executor::local_pool::block_on
 41: tokio::runtime::task::core::Core<T,S>::poll
 42: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
 43: tokio::runtime::task::harness::Harness<T,S>::poll
 44: tokio::runtime::blocking::pool::Inner::run
 45: tokio::runtime::context::enter
 46: std::sys_common::backtrace::__rust_begin_short_backtrace
 47: core::ops::function::FnOnce::call_once{{vtable.shim}}
 48: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
            at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/alloc/src/boxed.rs:1521:9
     <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
            at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/alloc/src/boxed.rs:1521:9
     std::sys::unix::thread::Thread::new::thread_start
            at /rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/sys/unix/thread.rs:71:17
 49: start_thread
 50: clone
 
 
Thread 'tokio-runtime-worker' panicked at 'Failed to allocate memory: "Allocator ran out of space"', /usr/local/cargo/git/checkouts/substrate-7e08433d4c370a21/3f11019/primitives/io/src/lib.rs:1057
 
This is a bug. Please report it at:
 
       https://github.com/paritytech/polkadot/issues/new
 
2021-05-24 14:40:53  Evicting failed runtime instance: RuntimePanicked("Failed to allocate memory: \"Allocator ran out of space\"")
2021-05-24 14:40:53  Block prepare storage changes error:
RuntimeApiError(Application(Execution(RuntimePanicked("Failed to allocate memory: \"Allocator ran out of space\""))))
2021-05-24 14:40:53  💔 Error importing block 0xd79f21595fb10471ab2b0d9427dffa8de4001c351a53f408995cf2dda309e43a: Err(Other(ClientImport("Import failed: Error at calling runtime api: Execution failed: RuntimePanicked(\"Failed to allocate memory: \\\"Allocator ran out of space\\\"\")")))
@vae520283995
Copy link

Downgrade to the v0.8.30

@vae520283995
Copy link

Getting the same issue. Running a node on 0.9.2

Thread 'tokio-runtime-worker' panicked at 'Failed to allocate memory: "Allocator ran out of space"', /usr/local/cargo/git/checkouts/substrate-7e08433d4c370a21/1d7f6e1/primitives │
│ /io/src/lib.rs:1068                                                                                                                                                               │
│ This is a bug. Please report it at:                                                                                                                                               │
│     https://github.com/paritytech/polkadot/issues/new                                                                                                                             │
│ 2021-05-24 16:05:57  Block prepare storage changes error:                                                                                                                         │
│ RuntimeApiError(Application(Execution(Other("Wasm execution trapped: Failed to allocate memory: \"Allocator ran out of space\"\nwasm backtrace:\n    0: 0x1601 - <unknown>!sp_io: │
│ :allocator::extern_host_function_impls::malloc::h3ba50d5b2de26c25\n    1: 0xcf604 - <unknown>!core::ops::function::impls::<impl core::ops::function::FnMut<A> for &mut F>::call_m │
│ ut::he3692847e41b46cc\n    2: 0xcf1c2 - <unknown>!sp_npos_elections::setup_inputs::hd89ca1e0feeaabd9\n    3: 0x90707 - <unknown>!sp_npos_elections::phragmen::seq_phragmen::hd129 │
│ 027c03c0249a\n    4: 0xa1161 - <unknown>!pallet_staking::Module<T>::new_era::hf7fe25e8949425a7\n    5: 0x9930e - <unknown>!<(TupleElement0,TupleElement1) as frame_support::trait │
│ s::OnInitialize<BlockNumber>>::on_initialize::h398bc3af78c339e8\n    6: 0x1576c3 - <unknown>!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPallets,COnRunt │
│ imeUpgrade>::initialize_block::h975d2a527ce51f82\n    7: 0x15082b - <unknown>!Core_execute_block\nnote: run with `WASMTIME_BACKTRACE_DETAILS=1` environment variable to display m │
│ ore information\n"))))                                                                                                                                                            │
│ 2021-05-24 16:05:57  💔 Error importing block 0xd79f21595fb10471ab2b0d9427dffa8de4001c351a53f408995cf2dda309e43a: Err(Other(ClientImport("Import failed: Error at calling runtime │
│  api: Execution failed: Other(\"Wasm execution trapped: Failed to allocate memory: \\\"Allocator ran out of space\\\"\\nwasm backtrace:\\n    0: 0x1601 - <unknown>!sp_io::alloca │
│ tor::extern_host_function_impls::malloc::h3ba50d5b2de26c25\\n    1: 0xcf604 - <unknown>!core::ops::function::impls::<impl core::ops::function::FnMut<A> for &mut F>::call_mut::he │
│ 3692847e41b46cc\\n    2: 0xcf1c2 - <unknown>!sp_npos_elections::setup_inputs::hd89ca1e0feeaabd9\\n    3: 0x90707 - <unknown>!sp_npos_elections::phragmen::seq_phragmen::hd129027c │
│ 03c0249a\\n    4: 0xa1161 - <unknown>!pallet_staking::Module<T>::new_era::hf7fe25e8949425a7\\n    5: 0x9930e - <unknown>!<(TupleElement0,TupleElement1) as frame_support::traits: │
│ :OnInitialize<BlockNumber>>::on_initialize::h398bc3af78c339e8\\n    6: 0x1576c3 - <unknown>!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPallets,COnRunti │
│ meUpgrade>::initialize_block::h975d2a527ce51f82\\n    7: 0x15082b - <unknown>!Core_execute_block\\nnote: run with `WASMTIME_BACKTRACE_DETAILS=1` environment variable to display  │
│ more information\\n\")")))                                                                                                                                                        │
│ 2021-05-24 16:05:57  💔 Error importing block 0xa1c3ac904935bfbae42d6402df189392efda3d89c59737741bd92d30f09f63d4: Err(UnknownParent)                                              │
│ 2021-05-24 16:05:57  💔 Error importing block 0xf120666f47cc30b6403d0154df21b58e9093977afbe9fe9e180feea74f70a5b8: Err(UnknownParent)                                              │
│ 2021-05-24 16:05:57  💔 Error importing block 0x0b7c21976d218d6634213e29ef14b9182715886455ad8283ce960fcb17c96500: Err(UnknownParent)                                              │

Downgrade to the v0.8.30

@caetanix
Copy link
Author

Works with v0.8.30! Thanks

@Neurone
Copy link
Contributor

Neurone commented May 24, 2021

Same here with 0.9.2. I worked well without problem until block 5202213, then it cannot go any further and it continue to give this error:

2021-05-24 23:22:21  ⚙️  Syncing  0.0 bps, target=#5206510 (24 peers), best: #5202215 (0xa1ef…ee60), finalized #5202213 (0x23a2…d92d), ⬇ 33.7kiB/s ⬆ 41.2kiB/s
====================
Version: 0.9.2-f36bb8201-x86_64-linux-gnu
   0: sp_panic_handler::set::{{closure}}
   1: std::panicking::rust_panic_with_hook
             at rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/panicking.rs:595:17
   2: std::panicking::begin_panic_handler::{{closure}}
             at rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/panicking.rs:497:13
   3: std::sys_common::backtrace::__rust_end_short_backtrace
             at rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/sys_common/backtrace.rs:141:18
   4: rust_begin_unwind
             at rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/panicking.rs:493:5
   5: core::panicking::panic_fmt
             at rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/core/src/panicking.rs:92:14
   6: core::option::expect_none_failed
             at rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/core/src/option.rs:1300:5
   7: tracing::span::Span::in_scope
   8: <<sp_io::allocator::HostFunctions as sp_wasm_interface::HostFunctions>::host_functions::ExtAllocatorMallocVersion1 as sp_wasm_interface::Function>::execute
   9: scoped_tls::ScopedKey<T>::with
  10: sc_executor_wasmtime::state_holder::with_context
  11: sc_executor_wasmtime::imports::call_static
  12: wasmtime::func::Func::new::{{closure}}
  13: wasmtime::trampoline::func::stub_fn
  14: <unknown>
  15: <unknown>
  16: <unknown>
  17: <unknown>
  18: <unknown>
  19: <unknown>
  20: <unknown>
  21: <unknown>
  22: <unknown>
  23: <unknown>
  24: RegisterSetjmp
  25: wasmtime_runtime::traphandlers::catch_traps
  26: wasmtime::func::Func::_call
  27: wasmtime::func::Func::call
  28: sc_executor_wasmtime::instance_wrapper::EntryPoint::call
  29: scoped_tls::ScopedKey<T>::set
  30: sc_executor_wasmtime::runtime::perform_call
  31: <sc_executor_wasmtime::runtime::WasmtimeInstance as sc_executor_common::wasm_runtime::WasmInstance>::call
  32: sc_executor_common::wasm_runtime::WasmInstance::call_export
  33: std::panicking::try
  34: std::thread::local::LocalKey<T>::with
  35: sc_executor::native_executor::WasmExecutor::with_instance::{{closure}}
  36: sc_executor::wasm_runtime::RuntimeCache::with_instance
  37: <sc_executor::native_executor::NativeExecutor<D> as sp_core::traits::CodeExecutor>::call
  38: sp_state_machine::execution::StateMachine<B,H,N,Exec>::execute_aux
  39: sp_state_machine::execution::StateMachine<B,H,N,Exec>::execute_using_consensus_failure_handler
  40: <sc_service::client::call_executor::LocalCallExecutor<B,E> as sc_client_api::call_executor::CallExecutor<Block>>::contextual_call
  41: <sc_service::client::client::Client<B,E,Block,RA> as sp_api::CallApiAt<Block>>::call_api_at
  42: sp_api::runtime_decl_for_Core::execute_block_call_api_at
  43: <polkadot_runtime::RuntimeApiImpl<__SR_API_BLOCK__,RuntimeApiImplCall> as sp_api::Core<__SR_API_BLOCK__>>::Core_execute_block_runtime_api_impl
  44: sp_api::Core::execute_block_with_context
  45: sc_service::client::client::Client<B,E,Block,RA>::prepare_block_storage_changes
  46: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
  47: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
  48: <sc_consensus_babe::BabeBlockImport<Block,Client,Inner> as sp_consensus::block_import::BlockImport<Block>>::import_block::{{closure}}
  49: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
  50: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
  51: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
  52: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
  53: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
  54: <sc_service::task_manager::prometheus_future::PrometheusFuture<T> as core::future::future::Future>::poll
  55: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
  56: <tracing_futures::Instrumented<T> as core::future::future::Future>::poll
  57: std::thread::local::LocalKey<T>::with
  58: futures_executor::local_pool::block_on
  59: tokio::runtime::task::core::Core<T,S>::poll
  60: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
  61: tokio::runtime::task::harness::Harness<T,S>::poll
  62: tokio::runtime::blocking::pool::Inner::run
  63: tokio::runtime::context::enter
  64: std::sys_common::backtrace::__rust_begin_short_backtrace
  65: core::ops::function::FnOnce::call_once{{vtable.shim}}
  66: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/alloc/src/boxed.rs:1521:9
      <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/alloc/src/boxed.rs:1521:9
      std::sys::unix::thread::Thread::new::thread_start
             at rustc/2fd73fabe469357a12c2c974c140f67e7cdd76d0/library/std/src/sys/unix/thread.rs:71:17
  67: start_thread
  68: clone
Thread 'tokio-runtime-worker' panicked at 'Failed to allocate memory: "Allocator ran out of space"', /usr/local/cargo/git/checkouts/substrate-7e08433d4c370a21/1d7f6e1/primitives/io/src/lib.rs:1068
This is a bug. Please report it at:
        https://github.com/paritytech/polkadot/issues/new
2021-05-24 23:22:23  Block prepare storage changes error:
RuntimeApiError(Application(Execution(Other("Wasm execution trapped: Failed to allocate memory: \"Allocator ran out of space\"\nwasm backtrace:\n    0: 0x1601 - <unknown>!sp_io::allocator::extern_host_function_impls::malloc::h3ba50d5b2de26c25\n    1: 0xcf604 - <unknown>!core::ops::function::impls::<impl core::ops::function::FnMut<A> for &mut F>::call_mut::he3692847e41b46cc\n    2: 0xcf1c2 - <unknown>!sp_npos_elections::setup_inputs::hd89ca1e0feeaabd9\n    3: 0x90707 - <unknown>!sp_npos_elections::phragmen::seq_phragmen::hd129027c03c0249a\n    4: 0xa1161 - <unknown>!pallet_staking::Module<T>::new_era::hf7fe25e8949425a7\n    5: 0x9930e - <unknown>!<(TupleElement0,TupleElement1) as frame_support::traits::OnInitialize<BlockNumber>>::on_initialize::h398bc3af78c339e8\n    6: 0x1576c3 - <unknown>!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPallets,COnRuntimeUpgrade>::initialize_block::h975d2a527ce51f82\n    7: 0x15082b - <unknown>!Core_execute_block\nnote: run with `WASMTIME_BACKTRACE_DETAILS=1` environment variable to display more information\n"))))
2021-05-24 23:22:23  💔 Error importing block 0xd79f21595fb10471ab2b0d9427dffa8de4001c351a53f408995cf2dda309e43a: Err(Other(ClientImport("Import failed: Error at calling runtime api: Execution failed: Other(\"Wasm execution trapped: Failed to allocate memory: \\\"Allocator ran out of space\\\"\\nwasm backtrace:\\n    0: 0x1601 - <unknown>!sp_io::allocator::extern_host_function_impls::malloc::h3ba50d5b2de26c25\\n    1: 0xcf604 - <unknown>!core::ops::function::impls::<impl core::ops::function::FnMut<A> for &mut F>::call_mut::he3692847e41b46cc\\n    2: 0xcf1c2 - <unknown>!sp_npos_elections::setup_inputs::hd89ca1e0feeaabd9\\n    3: 0x90707 - <unknown>!sp_npos_elections::phragmen::seq_phragmen::hd129027c03c0249a\\n    4: 0xa1161 - <unknown>!pallet_staking::Module<T>::new_era::hf7fe25e8949425a7\\n    5: 0x9930e - <unknown>!<(TupleElement0,TupleElement1) as frame_support::traits::OnInitialize<BlockNumber>>::on_initialize::h398bc3af78c339e8\\n    6: 0x1576c3 - <unknown>!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPallets,COnRuntimeUpgrade>::initialize_block::h975d2a527ce51f82\\n    7: 0x15082b - <unknown>!Core_execute_block\\nnote: run with `WASMTIME_BACKTRACE_DETAILS=1` environment variable to display more information\\n\")")))
2021-05-24 23:22:23  💔 Error importing block 0x5921b7b1b5269c96e91421f5b55d37c70db36d99fa3a207ce50429b3a20d7939: Err(UnknownParent)
2021-05-24 23:22:23  💔 Error importing block 0x4e2825636313cb034d0673571065869921a1aed0d088fe781134bce12a2c44df: Err(UnknownParent)
2021-05-24 23:22:23  💔 Error importing block 0x40a62de11bffe2af4a0ee6368554e133b9ffde3ef44b5e755b091fbf3185f776: Err(UnknownParent)
2021-05-24 23:22:23  💔 Error importing block 0x1eabcf8103e4e51e85c489478b71731aaec43b6865019b758133cea3619f1e0f: Err(UnknownParent)
2021-05-24 23:22:23  💔 Error importing block 0x0b7c21976d218d6634213e29ef14b9182715886455ad8283ce960fcb17c96500: Err(UnknownParent)
2021-05-24 23:22:23  💔 Error importing block 0x8b7bf83f23e2434f75a8f46322ebbf9f7165d444e0a170a895d64303c44e5434: Err(UnknownParent)
2021-05-24 23:22:23  💔 Error importing block 0x0b44b9869d4e7d7d423efc5825da6fcd43f779c2dfcf494537f082ae114305fa: Err(UnknownParent)
2021-05-24 23:22:23  💔 Error importing block 0x70507c7a65a929fd0eb661f3a10ba00ba5d70f8d25c2ab2fabac5de18e81ae9f: Err(UnknownParent)
2021-05-24 23:22:26  ⚙️  Syncing  0.0 bps, target=#5206510 (24 peers), best: #5202215 (0xa1ef…ee60), finalized #5202213 (0x23a2…d92d), ⬇ 572.4kiB/s ⬆ 5.6kiB/s

@Neurone
Copy link
Contributor

Neurone commented May 24, 2021

What's exactly inside block 0xd79f21595fb10471ab2b0d9427dffa8de4001c351a53f408995cf2dda309e43a that 0.9.x does not like?

  • 2021-05-24 11:36:12 (+UTC) Block #5202215 0xa1ef501e63732fa9b23538d7bac6320b63dc704d547a0211e915c5f46395ee60
  • 2021-05-24 12:47:00 (+UTC) Block #5202216 0xd79f21595fb10471ab2b0d9427dffa8de4001c351a53f408995cf2dda309e43a

More then one hour without valid blocks?

@AlexZhenWang
Copy link

AlexZhenWang commented May 24, 2021

Just in case there is someone who hasn't know, we need to add --execution=native for the validator node when downgrading to 0.8.30

@gituser
Copy link

gituser commented May 25, 2021

Same issue happens on v0.9.2 latest release. Had to rollback to v0.8.30 otherwise syncing is not working.

@dynasty2016
Copy link

Just in case there is someone who hasn't know, we need to add --execution=native for the validator node when downgrading to 0.8.30

To which command do we add this flag?

@tejon-melero
Copy link

tejon-melero commented May 25, 2021

Maybe this will be helpful to someone:

To fix this issue make sure that you run the node version v0.8.30 here: https://hub.docker.com/layers/parity/polkadot/v0.8.30/images/sha256-86e4b6d2ce6e520df66e858f2b3d8aa2903e101e1f0996da07f92529dd48a769?context=explore

and run the binary with the arguments --execution native

@gituser
Copy link

gituser commented May 25, 2021

@tejon-melero

is this flag --execution native required for regular node (non-validator) ?

as v0.8.30 works for me just fine without this flag

@tejon-melero
Copy link

@tejon-melero

is this flag --execution native required for regular node (non-validator) ?

as v0.8.30 works for me just fine without this flag

Interesting, then it seems like the the native execution type is the default execution type and this flag might not be mandatory.

@bkchr
Copy link
Member

bkchr commented May 31, 2021

Fixed by 0.9.3

@bkchr bkchr closed this as completed May 31, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants