Skip to content

Bug:Deadlocks occur when both engine and host startStopThreads values are set and the latter value is larger than the former#515

Closed
aooohan wants to merge 2 commits into
apache:mainfrom
aooohan:fixbug_deadlock
Closed

Bug:Deadlocks occur when both engine and host startStopThreads values are set and the latter value is larger than the former#515
aooohan wants to merge 2 commits into
apache:mainfrom
aooohan:fixbug_deadlock

Conversation

@aooohan

@aooohan aooohan commented May 18, 2022

Copy link
Copy Markdown
Member

apache-tomcat-10.0.21 version has been reproduced, I have not tried other versions

Reproduce

<Engine name="Catalina" defaultHost="localhost"  startStopThreads="3">
      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true" startStopThreads="4">
      </Host>
</Engine>

When starting tomcat as configured above, the situation shown in the image below will occur.

image

been stuck here⬆️⬆️⬆️

Then i used the Arthas tool to look at the threads and the results are shown below.

image

image

image

When starting tomcat, the current StandardServer instance object has been locked in LifecycleBase#start. When we do not set the startStopThreads value of StandardEngine, startStopThreads defaults to 1.
At this time, startStopExecutor in StandardEngine is actually an instance of InlineExecutorService (in fact, the current thread), ⬇️

startStopExecutor = new InlineExecutorService();

so when StandardEngine starts its subcontainer StandardHost through the startStopExecutor thread pool, because it is the current thread, there is no problem when executing the StandardServer#reconfigureUtilityExecutor method.
private synchronized void reconfigureUtilityExecutor(int threads) {

However, when the startStopExecutor in StandardEngine is not InlineExecutorService (ie Server#getUtilityExecutor()),
startStopExecutor = server.getUtilityExecutor();

the StandardServer#reconfigureUtilityExecutor method is executed again(The host's startStopThreads value is greater than the engine's startStopThreads ), because the lock of the StandardServer instance has not been released at this time, resulting in a deadlock.

My English is limited, I don't know if I have explained it clearly, or I am thinking in the wrong direction

@rmaucher

Copy link
Copy Markdown
Contributor

Good idea, the sync is overloaded in that case so this is risky, and other operations already use a servicesLock item for this reason.

@aooohan

aooohan commented May 19, 2022

Copy link
Copy Markdown
Member Author

@rmaucher Thank u for reviewing my PR, it's an honour to contribute PR to Tomcat.

@rmaucher

Copy link
Copy Markdown
Contributor

Thanks a lot for your PR. Sorry, I forgot to close it after merging it manually. The fix will be in Tomcat 10.1.0-M16, 10.0.22, 9.0.64. Tomcat 8.5 is not affected since it doesn't have the utility executor feature.

@rmaucher rmaucher closed this May 19, 2022
@aooohan
aooohan deleted the fixbug_deadlock branch May 19, 2022 13:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants