Kafka客户端2.2消费时，partition中reassign非leader replica导致分区的consumer主动退出修订版本明细

源

采纳

编辑于 3年前

M.s ALl

Kafka客户端2.2消费时，partition中reassign非leader replica导致分区的consumer主动退出

kafka

集群是2.2.1
kafka 客户端配置如下

[Executor task launch worker for task 49] INFO consumer.ConsumerConfig: ConsumerConfig values: 
    auto.commit.interval.ms = 5000
    auto.offset.reset = none
    bootstrap.servers = [略]
    check.crcs = true
    client.dns.lookup = default
    client.id = 
    connections.max.idle.ms = 540000
    default.api.timeout.ms = 60000
    enable.auto.commit = false
    exclude.internal.topics = true
    fetch.max.bytes = 52428800
    fetch.max.wait.ms = 500
    fetch.min.bytes = 1
    group.id = 123456
    heartbeat.interval.ms = 3000
    interceptor.classes = []
    internal.leave.group.on.close = true
    isolation.level = read_uncommitted
    key.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
    max.partition.fetch.bytes = 1048576
    max.poll.interval.ms = 300000
    max.poll.records = 500
    metadata.max.age.ms = 300000
    metric.reporters = []
    metrics.num.samples = 2
    metrics.recording.level = INFO
    metrics.sample.window.ms = 30000
    partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
    receive.buffer.bytes = 65536
    reconnect.backoff.max.ms = 1000
    reconnect.backoff.ms = 50
    request.timeout.ms = 30000
    retry.backoff.ms = 100
    security.protocol = PLAINTEXT
    send.buffer.bytes = 131072
    session.timeout.ms = 10000

在消费时，迁移例如某topic partition30中非leader节点

报错

User class threw exception: org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms expired before the position for partition topic-30 could be determined

达到三次后，这个 consumer 会向 group send 离开的请求，然后出发group reblance

现在初步怀疑是迁移导致的客户端 session 断连，然后客户端等待一个 max.poll.interval.ms 的时间内发生了此问题，也希望大佬来帮忙定位一下。

想请问一下

为何迁移非leader replica 会发生kafka.common.errors.TimeoutException
同时这种请况下调小max.poll.interval.ms，是否能够在后期isr扩张时不会再触发这个问题
有没有什么配置和方式能够避免这个问题

预览