Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experiment with fully virtual VirtualThreadPool #11501

Merged
merged 14 commits into from
May 7, 2024
Merged
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Jetty//Configure//EN" "https://www.eclipse.org/jetty/configure_10_0.dtd">

<Configure>
<New id="threadPool" class="org.eclipse.jetty.util.thread.VirtualThreadPool">
<Set name="name" property="jetty.threadPool.namePrefix" />
</New>

<Call class="org.slf4j.LoggerFactory" name="getLogger">
<Arg>org.eclipse.jetty</Arg>
<Call name="info">
<Arg>Virtual threads enabled. Using virtual threads for all Jetty tasks.</Arg>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd remove "Jetty" here; it is redundant and feels like "only the tasks of the Jetty implementation, so not those of the application".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sbordet I don't agree. We frequently have issues with users that use a mix of scheduling mechanisms. They might have an asynchronous Jetty Handler that uses some 3rd party async library that ends up using platform threads for its callback, then end up calling Jetty callbacks. It is important to be clear that we are not making all threads in the JVM virtual, just the ones that we dispatch.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gregw this is the VirtualThreadPool and "all tasks" refers to the ones submitted to this pool.
There is no need to say "Jetty", it's just the task submitted to this pool.

Copy link
Contributor

@sbordet sbordet May 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A server uses a platform thread pool, but have a proxy web application using HttpClient that uses VirtualThreadPool and now we have "all Jetty tasks" but which ones, the client's or the server's?

Perhaps let's add "Virtual thread available. Using virtual threads for all tasks submitted to VirtualThreadPool@1234."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still don't like that, as it is unclear what tasks are submitted to the VTP. It is kind of saying the obvious that the VTP will use virtual threads. We need to make the distinction between using threads for tasks calling the application vs tasks for all jetty tasks. I think the current text does that OK.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gregw I don't like it because there is no definition of what a "Jetty task" is, and I don't think we want to define that as it won't be useful to users.

We just need to make a difference between: everything is run by a virtual thread, or only web application code, so specifying "Jetty" is redundant because all tasks will be run by virtual threads, not only the "Jetty" tasks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, but you will get a great big "I told you so" the first time we get somebody confused that they have enabled this module, but that there are platform threads calling jetty callbacks and doing jetty things.

</Call>
</Call>
</Configure>
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
<Call class="org.slf4j.LoggerFactory" name="getLogger">
<Arg>org.eclipse.jetty</Arg>
<Call name="info">
<Arg>Virtual threads are enabled.</Arg>
<Arg>Virtual threads enabled. Using virtual threads for Jetty application tasks.</Arg>
</Call>
</Call>
</Configure>
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
[description]
Enables and configures the Server ThreadPool with support for virtual threads to be used for all threads.
There is some risk of CPU pinning with this configuration. Only supported in Java 21 or later.

[depends]
logging

[provides]
threadpool

[xml]
etc/jetty-threadpool-all-virtual.xml

[ini-template]
# tag::documentation[]
## Platform threads name prefix.
#jetty.threadPool.namePrefix=vtp<hashCode>

# end::documentation[]
gregw marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
[description]
Enables and configures the Server ThreadPool with support for virtual threads in Java 21 or later.
Enables and configures the Server ThreadPool with support for virtual threads to be used for blocking tasks.
Only supported in Java 21 or later.

[depends]
logging
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,21 +32,9 @@
public class VirtualThreads
{
private static final Logger LOG = LoggerFactory.getLogger(VirtualThreads.class);
private static final Executor executor = probeVirtualThreadExecutor();
private static final Executor executor = getNamedVirtualThreadsExecutor(null);
private static final Method isVirtualThread = probeIsVirtualThread();

private static Executor probeVirtualThreadExecutor()
{
try
{
return (Executor)Executors.class.getMethod("newVirtualThreadPerTaskExecutor").invoke(null);
}
catch (Throwable x)
{
return null;
}
}

private static Method probeIsVirtualThread()
{
try
Expand Down Expand Up @@ -131,7 +119,8 @@ public static Executor getNamedVirtualThreadsExecutor(String namePrefix)
{
Class<?> builderClass = Class.forName("java.lang.Thread$Builder");
Object threadBuilder = Thread.class.getMethod("ofVirtual").invoke(null);
threadBuilder = builderClass.getMethod("name", String.class, long.class).invoke(threadBuilder, namePrefix, 0L);
if (StringUtil.isNotBlank(namePrefix))
threadBuilder = builderClass.getMethod("name", String.class, long.class).invoke(threadBuilder, namePrefix, 0L);
ThreadFactory factory = (ThreadFactory)builderClass.getMethod("factory").invoke(threadBuilder);
return (Executor)Executors.class.getMethod("newThreadPerTaskExecutor", ThreadFactory.class).invoke(null, factory);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -449,7 +449,7 @@ public void setReservedThreads(int reservedThreads)
}

/**
* @return the name of the this thread pool
* @return the name of this thread pool
*/
@ManagedAttribute("name of the thread pool")
public String getName()
Expand All @@ -460,7 +460,7 @@ public String getName()
/**
* <p>Sets the name of this thread pool, used as a prefix for the thread names.</p>
*
* @param name the name of the this thread pool
* @param name the name of this thread pool
*/
public void setName(String name)
{
Expand Down Expand Up @@ -835,7 +835,7 @@ public void join() throws InterruptedException

while (isStopping())
{
Thread.sleep(1);
Thread.onSpinWait();
}
}

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
//
// ========================================================================
// Copyright (c) 1995 Mort Bay Consulting Pty Ltd and others.
//
// This program and the accompanying materials are made available under the
// terms of the Eclipse Public License v. 2.0 which is available at
// https://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
// which is available at https://www.apache.org/licenses/LICENSE-2.0.
//
// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
// ========================================================================
//

package org.eclipse.jetty.util.thread;

import java.io.IOException;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.Executor;

import org.eclipse.jetty.util.annotation.ManagedAttribute;
import org.eclipse.jetty.util.annotation.ManagedObject;
import org.eclipse.jetty.util.component.Dumpable;

@ManagedObject("Tracking Executor wrapper")
public class TrackingExecutor implements Executor, Dumpable
{
private final Executor _threadFactoryExecutor;
private final Set<Thread> _threads = ConcurrentHashMap.newKeySet();
private boolean _detailed;

public TrackingExecutor(Executor executor, boolean detailed)
{
_threadFactoryExecutor = executor;
_detailed = detailed;
}

@Override
public void execute(Runnable task)
{
_threadFactoryExecutor.execute(() ->
{
Thread thread = Thread.currentThread();
try
{
_threads.add(thread);
task.run();
}
finally
{
_threads.remove(thread);
}
});
}

@Override
public void dump(Appendable out, String indent) throws IOException
{
Object[] threads = _threads.stream().map(DumpableThread::new).toArray();
Dumpable.dumpObjects(out, indent, _threadFactoryExecutor.toString() + " size=" + threads.length, threads);
}

public void setDetailedDump(boolean detailedDump)
{
_detailed = detailedDump;
}

@ManagedAttribute("reports additional details in the dump")
public boolean isDetailedDump()
{
return _detailed;
}

public int size()
{
return _threads.size();
}

private class DumpableThread implements Dumpable
{
private final Thread _thread;

private DumpableThread(Thread thread)
{
_thread = thread;
}

@Override
public void dump(Appendable out, String indent) throws IOException
{
if (_detailed)
{
Object[] stack = _thread.getStackTrace();
Dumpable.dumpObjects(out, indent, _thread.toString(), stack);
}
else
{
Dumpable.dumpObject(out, _thread);
}
}
}
}
Loading
Loading