Skip to content

Initialization blocked by multi-threaded event publishing [SPR-16357] #20904

Closed

Description

@spring-projects-issues

Mark Paluch opened SPR-16357 and commented

Publishing events can block the container initialization in conjunction with event publishing from a different thread.

This can happen if events are published from a bean constructor while its bean is instantiated and events get published in a different thread while the constructed bean awaits completion of event publishing.

Consider following code:

@Component
class Foo {

	Foo(ApplicationEventPublisher publisher) throws Exception {

		Thread t = new Thread(() -> publisher.publishEvent(new Object()));
		t.start();
		t.join();
	}
}

The code above publishes an event in an other thread than the constructing thread while awaiting completion before the constructor progresses.

It leads to a thread state like:

"Thread-19@5373" daemon prio=5 tid=0x22 nid=NA waiting for monitor entry
  java.lang.Thread.State: BLOCKED
	 waiting for main@1315 to release lock on <0x17bd> (a java.util.concurrent.ConcurrentHashMap)
	  at org.springframework.context.event.AbstractApplicationEventMulticaster.getApplicationListeners(AbstractApplicationEventMulticaster.java:189)
	  at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:133)
	  at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:399)
	  at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:353)
	  at com.example.demo.Foo$1.run()
"main@1315" prio=5 tid=0xf nid=NA waiting (Thread calling initialization code)
  java.lang.Thread.State: WAITING
	 blocks Thread-19@5373
	  at java.lang.Object.wait(Object.java:-1)
	  at java.lang.Thread.join(Thread.java:1252)
	  at java.lang.Thread.join(Thread.java:1326)
	  at com.example.demo.Foo.<init>
	  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(NativeConstructorAccessorImpl.java:-1)
	  at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	  at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	  at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:163)
	  at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:117)
	  at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:271)
	  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1270)
	  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1127)
	  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:545)
	  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:502)
	  at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:312)
	  at org.springframework.beans.factory.support.AbstractBeanFactory$$Lambda$87.1534495070.getObject(Unknown Source:-1)
	  at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
	  - locked <0x17bd> (a java.util.concurrent.ConcurrentHashMap)
	  at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:310)
	  at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200)
	  at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:758)

The example originates from reactive code in which a constructor is used to call initialization code using .block() for synchronization, see DATAMONGO-1841.


Affects: 5.0 GA

Issue Links:

0 votes, 8 watchers

Activity

  1. spring-projects-issues commented on Jan 8, 2018

    @spring-projects-issues
    CollaboratorAuthor

    Juergen Hoeller commented

    This is a tough one since it relates to the general challenges of parallel bean initialization. At this point, we generally don't support custom threads triggering (singleton) bean initialization while the main thread is waiting on them.

  2. spring-projects-issues commented on Jan 8, 2018

    @spring-projects-issues
    CollaboratorAuthor

    Brian Clozel commented

    Apart from the general challenge, I wanted to point out that the driver for this issue is the following: being able to call initialization code during application startup using Mono / Flux types. Maybe there is a way to solve that use case without tackling that hard issue.

  3. spring-projects-issues commented on Dec 7, 2018

    @spring-projects-issues
    CollaboratorAuthor

    Sébastien Deleuze commented

    I faced the issue described by Brian Clozel in my reference Kotlin + Spring Reactive application https://github.com/mixitconf/mixit, and I could not find a satisfying solution, so I would like to know if we could improve that. I need to initialize data in my MongoDB instance with ReactiveMongoTemplate and I need these data to be in the database before serving requests to users and before running my integration tests.

    @Component
    class DatabaseInitializer(private val userRepository: UserRepository, ...) {
    
        @PostConstruct
        fun init() {
            userRepository.initData()
            /...
        }
    }
    
    @Repository
    class UserRepository(private val template: ReactiveMongoTemplate, ...) {
    
        private val logger = LoggerFactory.getLogger(this.javaClass)
    
        fun initData() {
            if (count().block() == 0L) {
                val usersResource = ClassPathResource("data/users.json")
                val users: List<User> = objectMapper.readValue(usersResource.inputStream)
                users.forEach { save(it).block() }
                logger.info("Users data initialization complete")
            }
        }
    }
    

    With Boot 2.0.2 it was working (by luck I think), but if I upgrade to Boot 2.0.6 or 2.1.1, it is blocking indefinitely at users.forEach level. I tried to change this to chain reactive operators and use a single block() but same issue. CommandLineRunner is not usable here since it will not ensure the data are in the database before running the application.

    I quote mp911de explanation:

    So what happened there is, that we emit application events Emission is synchronous and while waiting on a Mono.block() to complete, events wait being dispatched. Dispatch is blocked by a synchronized block in the application context (synchronized on the event dispatcher, it was, I think)
    The synchronized block was entered by the bean initializer that is currently executing Mono.block() (that’s where the circle meets its start)

    The only solution we have here is to use a blocking MongoTemplate, but I have hard time requiring injecting a second bean with a different API and infrastructure just for data initialization.

    I don't know if we could find a way to avoid this deadlock or support Mono<Void> return type for this kind of Reactive use cases, but it seems to me we are missing something here. Any thoughts?

  4. spring-projects-issues commented on Dec 7, 2018

    @spring-projects-issues
    CollaboratorAuthor

    Dave Syer commented

    I think CommandLineRunner is a reasonable compromise - it certainly should work with integration tests, even if technically there is a window where the app is listening on port 8080 but not able to serve accurate data. If it doesn't work, then I consider that a bug as well.

  5. spring-projects-issues commented on Dec 7, 2018

    @spring-projects-issues
    CollaboratorAuthor

    Brian Clozel commented

    Looking at Juergen Hoeller's comments in #21025, I don't think we'll have a reactive way of publishing/processing events - for the same reason, supporting Mono<Void> is probably not an option?

  6. spring-projects-issues commented on Dec 7, 2018

    @spring-projects-issues
    CollaboratorAuthor

    Sébastien Deleuze commented

    Dave Syer I was mistaken by spring-projects/spring-boot#7656 where application and command line runners were initially moved after ready event, then it was rollbacked. I did some test with my MiXiT application, and I can confirm the data are created before my integration tests. That said using CommandLineRunner breaks a test so I need to dig into it more in detail to understand if that can be used as a reliable replacement or not.

    Also I still think that a lot of users will be spend lot of time stuck on the @PostConstruct issue, and it seems we don't have an answer for that pretty common use case at Spring Framework level, so maybe worth to discuss for Spring 5.2?

  7. spring-projects-issues commented on Dec 28, 2018

    @spring-projects-issues
    CollaboratorAuthor

    Sébastien Deleuze commented

    I confirm CommandLineRunner is not a good fit as a general solution for this. In addition to being a Boot construct, in my application the web router which specifies @DependsOn("databaseInitializer") needs the data to be created in the database to generate routes at startup based on these data. It was working with @PostConstruct but it does not with CommandLineRunner so that's another indication something like #19487 is needed.

  8. modified the milestones: 5.2 M1, 5.2 M2 on Mar 23, 2019
  9. modified the milestones: 5.2 M2, 5.2 M3 on May 1, 2019
  10. 2 remaining items

  11. modified the milestones: 5.2 RC1, 5.2 RC2 on Jul 17, 2019
  12. modified the milestones: 5.2 RC2, 5.2 GA on Sep 2, 2019
  13. modified the milestones: 5.2 GA, 5.x Backlog on Sep 9, 2019
  14. ganeshkanyal commented on Jun 9, 2021

    @ganeshkanyal

    @jhoeller - I am using spring 5.2.8 and still seeing this issue. Any updates on the fix?

    Our project is trying to load the application cache using multiple parallel threads during @PostConstruct of the Cacheloader bean. If any of the threads has some issue while loading the cache we publish the issue as an events . A listener listens to those events and log the issue to the external system.

    I am seeing deadlock while cache loader threads trying to publish the events using spring ApplicationEventpublisher.

  15. modified the milestones: 6.x Backlog, 6.2.x on Dec 18, 2023
  16. modified the milestones: 6.2.x, 6.2.0-M1 on Feb 19, 2024
  17. jhoeller commented on Feb 19, 2024

    @jhoeller
    Contributor

    With #25799 having been addressed years ago and #23501 revised for 6.2 now, there should not be any blocking for asynchronous event publication anymore. #25799 made it work fine for existing listeners, #23501 makes it work for listener beans that need to be initialized yet as well (unless there is a conflict among the dependencies created for that listener).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

in: coreIssues in core modules (aop, beans, core, context, expression)type: bugA general bug

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions