kafka使用kerberos报错，求助修订版本明细

源

采纳

编辑于

小蕊

kafka使用kerberos报错，求助

kafka

我在使用kerberos的时候，遇到了以下的错误，实在是解决不了，求助！

zookeeper log:

[2016-07-24 02:23:23,935] INFO Accepted socket connection from /10.211.55.5:41176 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2016-07-24 02:23:23,944] DEBUG Session establishment request from client /10.211.55.5:41176 client's lastZxid is 0x0 (org.apache.zookeeper.server.ZooKeeperServer)
[2016-07-24 02:23:23,944] INFO Client attempting to establish new session at /10.211.55.5:41176 (org.apache.zookeeper.server.ZooKeeperServer)
[2016-07-24 02:23:23,949] DEBUG Processing request:: sessionid:0x15618f30b890001 type:createSession cxid:0x0 zxid:0x424b txntype:-10 reqpath:n/a (org.apache.zookeeper.server.FinalRequestProcessor)
[2016-07-24 02:23:23,950] DEBUG sessionid:0x15618f30b890001 type:createSession cxid:0x0 zxid:0x424b txntype:-10 reqpath:n/a (org.apache.zookeeper.server.FinalRequestProcessor)
[2016-07-24 02:23:23,950] INFO Established session 0x15618f30b890001 with negotiated timeout 6000 for client /10.211.55.5:41176 (org.apache.zookeeper.server.ZooKeeperServer)
[2016-07-24 02:23:23,973] DEBUG Responding to client SASL token. (org.apache.zookeeper.server.ZooKeeperServer)
[2016-07-24 02:23:23,973] DEBUG Size of client SASL token: 573 (org.apache.zookeeper.server.ZooKeeperServer)
[2016-07-24 02:23:23,973] ERROR cnxn.saslServer is null: cnxn object did not initialize its saslServer properly. (org.apache.zookeeper.server.ZooKeeperServer)
[2016-07-24 02:23:24,318] WARN caught end of stream exception (org.apache.zookeeper.server.NIOServerCnxn)
EndOfStreamException: Unable to read additional data from client sessionid 0x15618f30b890001, likely client has closed socket
    at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
    at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
    at java.lang.Thread.run(Thread.java:745)
[2016-07-24 02:23:24,318] INFO Closed socket connection for client /10.211.55.5:41176 which had sessionid 0x15618f30b890001 (org.apache.zookeeper.server.NIOServerCnxn)
[2016-07-24 02:23:30,000] INFO Expiring session 0x15618f30b890001, timeout of 6000ms exceeded (org.apache.zookeeper.server.ZooKeeperServer)
[2016-07-24 02:23:30,001] INFO Processed session termination for sessionid: 0x15618f30b890001 (org.apache.zookeeper.server.PrepRequestProcessor)
[2016-07-24 02:23:30,004] DEBUG Processing request:: sessionid:0x15618f30b890001 type:closeSession cxid:0x0 zxid:0x424c txntype:-11 reqpath:n/a (org.apache.zookeeper.server.FinalRequestProcessor)

kafka log:

[2016-07-24 02:23:23,954] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
[2016-07-24 02:23:23,954] DEBUG Leaving process event (org.I0Itec.zkclient.ZkClient)
[2016-07-24 02:23:23,954] DEBUG ClientCnxn:sendSaslPacket:length=0 (org.apache.zookeeper.client.ZooKeeperSaslClient)
[2016-07-24 02:23:23,955] DEBUG saslClient.evaluateChallenge(len=0) (org.apache.zookeeper.client.ZooKeeperSaslClient)
[2016-07-24 02:23:23,973] ERROR SASL authentication failed using login context 'Client'. (org.apache.zookeeper.client.ZooKeeperSaslClient)
[2016-07-24 02:23:23,974] DEBUG Received event: WatchedEvent state:AuthFailed type:None path:null (org.I0Itec.zkclient.ZkClient)
[2016-07-24 02:23:23,974] INFO zookeeper state changed (AuthFailed) (org.I0Itec.zkclient.ZkClient)
[2016-07-24 02:23:23,974] DEBUG Leaving process event (org.I0Itec.zkclient.ZkClient)
[2016-07-24 02:23:23,974] DEBUG Closing ZkClient... (org.I0Itec.zkclient.ZkClient)
[2016-07-24 02:23:23,974] INFO Terminate ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2016-07-24 02:23:23,974] DEBUG Closing ZooKeeper connected to 10.211.55.5:2181 (org.I0Itec.zkclient.ZkConnection)
[2016-07-24 02:23:23,974] DEBUG Close called on already closed client (org.apache.zookeeper.ZooKeeper)
[2016-07-24 02:23:23,974] DEBUG Closing ZkClient...done (org.I0Itec.zkclient.ZkClient)
[2016-07-24 02:23:23,975] FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure
    at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:946)
    at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:923)
    at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1230)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:156)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:130)
    at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:75)
    at kafka.utils.ZkUtils$.apply(ZkUtils.scala:57)
    at kafka.server.KafkaServer.initZk(KafkaServer.scala:294)
    at kafka.server.KafkaServer.startup(KafkaServer.scala:180)
    at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
    at kafka.Kafka$.main(Kafka.scala:67)
    at kafka.Kafka.main(Kafka.scala)
[2016-07-24 02:23:23,978] INFO shutting down (kafka.server.KafkaServer)
[2016-07-24 02:23:23,979] DEBUG Shutting down task scheduler. (kafka.utils.KafkaScheduler)
[2016-07-24 02:23:23,981] INFO shut down completed (kafka.server.KafkaServer)
[2016-07-24 02:23:23,982] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure
    at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:946)
    at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:923)
    at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1230)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:156)
    at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:130)
    at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:75)
    at kafka.utils.ZkUtils$.apply(ZkUtils.scala:57)
    at kafka.server.KafkaServer.initZk(KafkaServer.scala:294)
    at kafka.server.KafkaServer.startup(KafkaServer.scala:180)
    at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
    at kafka.Kafka$.main(Kafka.scala:67)
    at kafka.Kafka.main(Kafka.scala)
[2016-07-24 02:23:23,985] INFO shutting down (kafka.server.KafkaServer)

krb5kdc log

Jul 24 02:23:23 weiwei krb5kdc[17652](info): AS_REQ (3 etypes {17 16 23}) 10.211.55.5: ISSUE: authtime 1469298203, etypes {rep=17 tkt=18 ses=17}, kafka/10.211.55.5@EXAMPLE.COM for krbtgt/EXAMPLE.COM@EXAMPLE.COM
Jul 24 02:23:23 weiwei krb5kdc[17652](info): TGS_REQ (3 etypes {17 16 23}) 10.211.55.5: ISSUE: authtime 1469298203, etypes {rep=17 tkt=18 ses=17}, kafka/10.211.55.5@EXAMPLE.COM for zookeeper/10.211.55.5@EXAMPLE.COM

My configuration is as follows:

/etc/krb5.conf

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 default_realm = EXAMPLE.COM
 dns_lookup_realm = false
 dns_lookup_kdc = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true

[realms]
 EXAMPLE.COM = {
  kdc = 10.211.55.5
  admin_server = 10.211.55.5
 }

[domain_realm]
10.211.55.5 = EXAMPLE.COM

/etc/kafka/kafka_server_jaas.conf

KafkaServer {
     com.sun.security.auth.module.Krb5LoginModule required
     useKeyTab=true
     storeKey=true
     keyTab="/var/kerberos/krb5kdc/kafka.keytab"
     principal="kafka/10.211.55.5@EXAMPLE.COM";
};

 // Zookeeper client authentication
Client {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    storeKey=true
    keyTab="/var/kerberos/krb5kdc/kafka.keytab"
    principal="kafka/10.211.55.5@EXAMPLE.COM";
};

kadmin.local

Authenticating as principal root/admin@EXAMPLE.COM with password.
kadmin.local:  listprincs
K/M@EXAMPLE.COM
kadmin/admin@EXAMPLE.COM
kadmin/changepw@EXAMPLE.COM
kadmin/weiwei@EXAMPLE.COM
kafka/10.211.55.5@EXAMPLE.COM
kafka/weiwei@EXAMPLE.COM
krbtgt/EXAMPLE.COM@EXAMPLE.COM
root/admin@EXAMPLE.COM
zookeeper/10.211.55.5@EXAMPLE.COM

JVM：

start the process:

zookeeper:

# ps -ef|grep zookeeper|grep --color=auto /etc/kafka/kafka_server_jaas.conf

root      6172 20094 39 03:02 pts/5    00:00:00 /usr/java/jdk1.8.0_60/bin/java -Xmx512M -Xms512M -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+DisableExplicitGC -Djava.security.krb5.conf=/etc/krb5.conf -Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf -Djava.awt.headless=true -Xloggc:/tools/kafka_2.11-0.10.0.0/bin/../logs/zookeeper-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dkafka.logs.dir=/tools/kafka_2.11-0.10.0.0/bin/../logs -Dlog4j.configuration=file:bin/../config/log4j.properties -cp .:/usr/java/jdk1.8.0_60/lib/dt.jar:/usr/java/jdk1.8.0_60/lib/tools.jar:/usr/java/jdk1.8.0_60/bin/java:/tools/kafka_2.11-0.10.0.0/bin/../libs/aopalliance-repackaged-2.4.0-b34.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/argparse4j-0.5.0.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/connect-api-0.10.0.0.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/connect-file-0.10.0.0.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/connect-json-0.10.0.0.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/connect-runtime-0.10.0.0.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/guava-18.0.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/hk2-api-2.4.0-b34.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/hk2-locator-2.4.0-b34.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/hk2-utils-2.4.0-b34.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jackson-annotations-2.6.0.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jackson-core-2.6.3.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jackson-databind-2.6.3.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jackson-jaxrs-base-2.6.3.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jackson-jaxrs-json-provider-2.6.3.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jackson-module-jaxb-annotations-2.6.3.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/javassist-3.18.2-GA.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/javax.annotation-api-1.2.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/javax.inject-1.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/javax.inject-2.4.0-b34.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/javax.servlet-api-3.1.0.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/javax.ws.rs-api-2.0.1.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jersey-client-2.22.2.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jersey-common-2.22.2.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jersey-container-servlet-2.22.2.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jersey-container-servlet-core-2.22.2.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jersey-guava-2.22.2.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jersey-media-jaxb-2.22.2.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jersey-server-2.22.2.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jetty-continuation-9.2.15.v20160210.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jetty-http-9.2.15.v20160210.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jetty-io-9.2.15.v20160210.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jetty-security-9.2.15.v20160210.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jetty-server-9.2.15.v20160210.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jetty-servlet-9.2.15.v20160210.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jetty-servlets-9.2.15.v20160210.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jetty-util-9.2.15.v20160210.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/jopt-simple-4.9.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/kafka_2.11-0.10.0.0.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/kafka_2.11-0.10.0.0-sources.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/kafka_2.11-0.10.0.0-test-sources.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/kafka-clients-0.10.0.0.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/kafka-log4j-appender-0.10.0.0.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/kafka-streams-0.10.0.0.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/kafka-streams-examples-0.10.0.0.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/kafka-tools-0.10.0.0.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/log4j-1.2.17.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/lz4-1.3.0.jar:/tools/kafka_2.11-0.10.0.0/bin/../libs/metrics-core-2.2.0.jar:/tools/kafka_2.11-0.10.

我的环境:

jdk

java -version
java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)

系统:

[root@weiwei kafka_2.11-0.10.0.0]# uname -a
Linux weiwei 2.6.32-358.el6.x86_64 #1 SMP Fri Feb 22 00:31:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

地址

[root@weiwei kafka_2.11-0.10.0.0]# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:1C:42:E4:B6:1E
          inet addr:10.211.55.5  Bcast:10.211.55.255  Mask:255.255.255.0
          inet6 addr: fdb2:2c26:f4e4:0:21c:42ff:fee4:b61e/64 Scope:Global
          inet6 addr: fe80::21c:42ff:fee4:b61e/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:80270 errors:0 dropped:0 overruns:0 frame:0
          TX packets:45714 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:37037138 (35.3 MiB)  TX bytes:7155183 (6.8 MiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:40532 errors:0 dropped:0 overruns:0 frame:0
          TX packets:40532 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:2811173 (2.6 MiB)  TX bytes:2811173 (2.6 MiB)

virbr0    Link encap:Ethernet  HWaddr 52:54:00:56:6D:C8
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

预览