Skip to content
This repository was archived by the owner on Nov 20, 2024. It is now read-only.
This repository was archived by the owner on Nov 20, 2024. It is now read-only.

KafkaBinderHealthIndicator doesn't work with SSL #79

Closed

Description

@barrycommins

Hi,

interesting one this.
When Kafka is configured to use SSL, health checks do not appear to work.

In KafkaBinderHealthIndicator, a minimal set of properties is set on the KafkaConsumer:

Map<String, String> properties = new HashMap<>();
properties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, this.configurationProperties
		.getKafkaConnectionString());
properties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, ByteArrayDeserializer.class.getName());
properties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, ByteArrayDeserializer.class.getName());
KafkaConsumer metadataConsumer = new KafkaConsumer(properties);

When using something that polls the /health, this can then lead to OutOfMemory errors from Kafka:

lang.OutOfMemoryError: Direct buffer memory] with root cause
java.lang.OutOfMemoryError: Direct buffer memory
    at java.nio.Bits.reserveMemory(Bits.java:693)
    at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
    at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
    at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:241)
    at sun.nio.ch.IOUtil.read(IOUtil.java:195)
    at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
    at org.apache.kafka.common.network.PlaintextTransportLayer.read(PlaintextTransportLayer.java:108)
    at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:97)
    at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71)
    at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:153)
    at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:134)
    at org.apache.kafka.common.network.Selector.poll(Selector.java:286)
    at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:256)
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:320)
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:213)
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:178)
    at org.apache.kafka.clients.consumer.internals.Fetcher.getTopicMetadata(Fetcher.java:201)
    at org.apache.kafka.clients.consumer.KafkaConsumer.partitionsFor(KafkaConsumer.java:1182)
    at org.springframework.cloud.stream.binder.kafka.KafkaBinderHealthIndicator.health(KafkaBinderHealthIndicator.java:64)
    at org.springframework.boot.actuate.health.CompositeHealthIndicator.health(CompositeHealthIndicator.java:68)
    at org.springframework.boot.actuate.health.CompositeHealthIndicator.health(CompositeHealthIndicator.java:68)
    at org.springframework.boot.actuate.health.CompositeHealthIndicator.health(CompositeHealthIndicator.java:68)
    at org.springframework.boot.actuate.endpoint.HealthEndpoint.invoke(HealthEndpoint.java:81)
    at org.springframework.boot.actuate.endpoint.mvc.HealthMvcEndpoint.getHealth(HealthMvcEndpoint.java:166)
    at org.springframework.boot.actuate.endpoint.mvc.HealthMvcEndpoint.invoke(HealthMvcEndpoint.java:140)

The simplest workaround seems to be to just turn off the health indicator, with:
management.health.binders.enabled=false, which works fine, but it would be nice to be able to perform health checks over SSL.

Would it be possible to do something like setting the properties specified by spring.cloud.stream.kafka.binder.configuration on the KafkaConsumer created in KafkaBinderHealthIndicator?

Thanks

Activity

  1. barrycommins commented on Jan 11, 2017

    @barrycommins
    ContributorAuthor

    Hi,

    I created a pull request as a fix for this.
    I may well be completely barking up the wrong tree with the solution, but I guess it's worth a try :-)

  2. added this to the 1.1.2.RELEASE milestone on Jan 18, 2017
  3. added a commit that references this issue on Mar 7, 2017
    ca3b89b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions