返回到文章

采纳

编辑于 4年前

kafka消费者控制台不停打印一条消息(bin/kafka-console-consumer.sh)

kafka

1、kafka消费者控制台不停打印一条消息
2、kafka版本 kafka_2.13-2.4.0.tgz
3、现象:只修改一次数据,kafka一直打印这条消息

 {"data":[{"id":"45320","user_id":"524341983699795968","task_id":"101","task_name":"完成语境中学习15分钟","kind":"1","kind_name":"everyday","number":"1","total_number":"1","score":"0","insert_time":"1600071504","update_time":"1600071504","date_time":"1600012800","is_had":"1"}],"database":"world","es":1600071626000,"id":247,"isDdl":false,"mysqlType":{"id":"int(11)","user_id":"varchar(32)","task_id":"int(11)","task_name":"varchar(32)","kind":"int(11)","kind_name":"varchar(32)","number":"int(11)","total_number":"int(11)","score":"int(11)","insert_time":"int(10) unsigned","update_time":"int(10) unsigned","date_time":"int(10) unsigned","is_had":"tinyint(3)"},"old":[{"update_time":"1600071500"}],"pkNames":["id"],"sql":"","sqlType":{"id":4,"user_id":12,"task_id":4,"task_name":12,"kind":4,"kind_name":12,"number":4,"total_number":4,"score":4,"insert_time":4,"update_time":4,"date_time":4,"is_had":-6},"table":"tt_user_task","ts":1600071627111,"type":"UPDATE"}
{"data":[{"id":"45320","user_id":"524341983699795968","task_id":"101","task_name":"完成语境中学习15分钟","kind":"1","kind_name":"everyday","number":"1","total_number":"1","score":"0","insert_time":"1600071504","update_time":"1600071504","date_time":"1600012800","is_had":"1"}],"database":"world","es":1600071626000,"id":247,"isDdl":false,"mysqlType":{"id":"int(11)","user_id":"varchar(32)","task_id":"int(11)","task_name":"varchar(32)","kind":"int(11)","kind_name":"varchar(32)","number":"int(11)","total_number":"int(11)","score":"int(11)","insert_time":"int(10) unsigned","update_time":"int(10) unsigned","date_time":"int(10) unsigned","is_had":"tinyint(3)"},"old":[{"update_time":"1600071500"}],"pkNames":["id"],"sql":"","sqlType":{"id":4,"user_id":12,"task_id":4,"task_name":12,"kind":4,"kind_name":12,"number":4,"total_number":4,"score":4,"insert_time":4,"update_time":4,"date_time":4,"is_had":-6},"table":"tt_user_task","ts":1600071627111,"type":"UPDATE"}
{"data":[{"id":"45320","user_id":"524341983699795968","task_id":"101","task_name":"完成语境中学习15分钟","kind":"1","kind_name":"everyday","number":"1","total_number":"1","score":"0","insert_time":"1600071504","update_time":"1600071504","date_time":"1600012800","is_had":"1"}],"database":"world","es":1600071626000,"id":247,"isDdl":false,"mysqlType":{"id":"int(11)","user_id":"varchar(32)","task_id":"int(11)","task_name":"varchar(32)","kind":"int(11)","kind_name":"varchar(32)","number":"int(11)","total_number":"int(11)","score":"int(11)","insert_time":"int(10) unsigned","update_time":"int(10) unsigned","date_time":"int(10) unsigned","is_had":"tinyint(3)"},"old":[{"update_time":"1600071500"}],"pkNames":["id"],"sql":"","sqlType":{"id":4,"user_id":12,"task_id":4,"task_name":12,"kind":4,"kind_name":12,"number":4,"total_number":4,"score":4,"insert_time":4,"update_time":4,"date_time":4,"is_had":-6},"table":"tt_user_task","ts":1600071627111,"type":"UPDATE"}
{"data":[{"id":"45320","user_id":"524341983699795968","task_id":"101","task_name":"完成语境中学习15分钟","kind":"1","kind_name":"everyday","number":"1","total_number":"1","score":"0","insert_time":"1600071504","update_time":"1600071504","date_time":"1600012800","is_had":"1"}],"database":"world","es":1600071626000,"id":247,"isDdl":false,"mysqlType":{"id":"int(11)","user_id":"varchar(32)","task_id":"int(11)","task_name":"varchar(32)","kind":"int(11)","kind_name":"varchar(32)","number":"int(11)","total_number":"int(11)","score":"int(11)","insert_time":"int(10) unsigned","update_time":"int(10) unsigned","date_time":"int(10) unsigned","is_had":"tinyint(3)"},"old":[{"update_time":"1600071500"}],"pkNames":["id"],"sql":"","sqlType":{"id":4,"user_id":12,"task_id":4,"task_name":12,"kind":4,"kind_name":12,"number":4,"total_number":4,"score":4,"insert_time":4,"update_time":4,"date_time":4,"is_had":-6},"table":"tt_user_task","ts":1600071627111,"type":"UPDATE"}
{"data":[{"id":"45320","user_id":"524341983699795968","task_id":"101","task_name":"完成语境中学习15分钟","kind":"1","kind_name":"everyday","number":"1","total_number":"1","score":"0","insert_time":"1600071504","update_time":"1600071504","date_time":"1600012800","is_had":"1"}],"database":"world","es":1600071626000,"id":247,"isDdl":false,"mysqlType":{"id":"int(11)","user_id":"varchar(32)","task_id":"int(11)","task_name":"varchar(32)","kind":"int(11)","kind_name":"varchar(32)","number":"int(11)","total_number":"int(11)","score":"int(11)","insert_time":"int(10) unsigned","update_time":"int(10) unsigned","date_time":"int(10) unsigned","is_had":"tinyint(3)"},"old":[{"update_time":"1600071500"}],"pkNames":["id"],"sql":"","sqlType":{"id":4,"user_id":12,"task_id":4,"task_name":12,"kind":4,"kind_name":12,"number":4,"total_number":4,"score":4,"insert_time":4,"update_time":4,"date_time":4,"is_had":-6},"table":"tt_user_task","ts":1600071627111,"type":"UPDATE"}
{"data":[{"id":"45320","user_id":"524341983699795968","task_id":"101","task_name":"完成语境中学习15分钟","kind":"1","kind_name":"everyday","number":"1","total_number":"1","score":"0","insert_time":"1600071504","update_time":"1600071504","date_time":"1600012800","is_had":"1"}],"database":"world","es":1600071626000,"id":247,"isDdl":false,"mysqlType":{"id":"int(11)","user_id":"varchar(32)","task_id":"int(11)","task_name":"varchar(32)","kind":"int(11)","kind_name":"varchar(32)","number":"int(11)","total_number":"int(11)","score":"int(11)","insert_time":"int(10) unsigned","update_time":"int(10) unsigned","date_time":"int(10) unsigned","is_had":"tinyint(3)"},"old":[{"update_time":"1600071500"}],"pkNames":["id"],"sql":"","sqlType":{"id":4,"user_id":12,"task_id":4,"task_name":12,"kind":4,"kind_name":12,"number":4,"total_number":4,"score":4,"insert_time":4,"update_time":4,"date_time":4,"is_had":-6},"table":"tt_user_task","ts":1600071627111,"type":"UPDATE"}
{"data":[{"id":"45320","user_id":"524341983699795968","task_id":"101","task_name":"完成语境中学习15分钟","kind":"1","kind_name":"everyday","number":"1","total_number":"1","score":"0","insert_time":"1600071504","update_time":"1600071504","date_time":"1600012800","is_had":"1"}],"database":"world","es":1600071626000,"id":247,"isDdl":false,"mysqlType":{"id":"int(11)","user_id":"varchar(32)","task_id":"int(11)","task_name":"varchar(32)","kind":"int(11)","kind_name":"varchar(32)","number":"int(11)","total_number":"int(11)","score":"int(11)","insert_time":"int(10) unsigned","update_time":"int(10) unsigned","date_time":"int(10) unsigned","is_had":"tinyint(3)"},"old":[{"update_time":"1600071500"}],"pkNames":["id"],"sql":"","sqlType":{"id":4,"user_id":12,"task_id":4,"task_name":12,"kind":4,"kind_name":12,"number":4,"total_number":4,"score":4,"insert_time":4,"update_time":4,"date_time":4,"is_had":-6},"table":"tt_user_task","ts":1600071627111,"type":"UPDATE"}
{"data":[{"id":"45320","user_id":"524341983699795968","task_id":"101","task_name":"完成语境中学习15分钟","kind":"1","kind_name":"everyday","number":"1","total_number":"1","score":"0","insert_time":"1600071504","update_time":"1600071504","date_time":"1600012800","is_had":"1"}],"database":"world","es":1600071626000,"id":247,"isDdl":false,"mysqlType":{"id":"int(11)","user_id":"varchar(32)","task_id":"int(11)","task_name":"varchar(32)","kind":"int(11)","kind_name":"varchar(32)","number":"int(11)","total_number":"int(11)","score":"int(11)","insert_time":"int(10) unsigned","update_time":"int(10) unsigned","date_time":"int(10) unsigned","is_had":"tinyint(3)"},"old":[{"update_time":"1600071500"}],"pkNames":["id"],"sql":"","sqlType":{"id":4,"user_id":12,"task_id":4,"task_name":12,"kind":4,"kind_name":12,"number":4,"total_number":4,"score":4,"insert_time":4,"update_time":4,"date_time":4,"is_had":-6},"table":"tt_user_task","ts":1600071627111,"type":"UPDATE"}
{"data":[{"id":"45320","user_id":"524341983699795968","task_id":"101","task_name":"完成语境中学习15分钟","kind":"1","kind_name":"everyday","number":"1","total_number":"1","score":"0","insert_time":"1600071504","update_time":"1600071504","date_time":"1600012800","is_had":"1"}],"database":"world","es":1600071626000,"id":247,"isDdl":false,"mysqlType":{"id":"int(11)","user_id":"varchar(32)","task_id":"int(11)","task_name":"varchar(32)","kind":"int(11)","kind_name":"varchar(32)","number":"int(11)","total_number":"int(11)","score":"int(11)","insert_time":"int(10) unsigned","update_time":"int(10) unsigned","date_time":"int(10) unsigned","is_had":"tinyint(3)"},"old":[{"update_time":"1600071500"}],"pkNames":["id"],"sql":"","sqlType":{"id":4,"user_id":12,"task_id":4,"task_name":12,"kind":4,"kind_name":12,"number":4,"total_number":4,"score":4,"insert_time":4,"update_time":4,"date_time":4,"is_had":-6},"table":"tt_user_task","ts":1600071627111,"type":"UPDATE"}
{"data":[{"id":"45320","user_id":"524341983699795968","task_id":"101","task_name":"完成语境中学习15分钟","kind":"1","kind_name":"everyday","number":"1","total_number":"1","score":"0","insert_time":"1600071504","update_time":"1600071504","date_time":"1600012800","is_had":"1"}],"database":"world","es":1600071626000,"id":247,"isDdl":false,"mysqlType":{"id":"int(11)","user_id":"varchar(32)","task_id":"int(11)","task_name":"varchar(32)","kind":"int(11)","kind_name":"varchar(32)","number":"int(11)","total_number":"int(11)","score":"int(11)","insert_time":"int(10) unsigned","update_time":"int(10) unsigned","date_time":"int(10) unsigned","is_had":"tinyint(3)"},"old":[{"update_time":"1600071500"}],"pkNames":["id"],"sql":"","sqlType":{"id":4,"user_id":12,"task_id":4,"task_name":12,"kind":4,"kind_name":12,"number":4,"total_number":4,"score":4,"insert_time":4,"update_time":4,"date_time":4,"is_had":-6},"table":"tt_user_task","ts":1600071627111,"type":"UPDATE"}
{"data":[{"id":"45320","user_id":"524341983699795968","task_id":"101","task_name":"完成语境中学习15分钟","kind":"1","kind_name":"everyday","number":"1","total_number":"1","score":"0","insert_time":"1600071504","update_time":"1600071504","date_time":"1600012800","is_had":"1"}],"database":"world","es":1600071626000,"id":247,"isDdl":false,"mysqlType":{"id":"int(11)","user_id":"varchar(32)","task_id":"int(11)","task_name":"varchar(32)","kind":"int(11)","kind_name":"varchar(32)","number":"int(11)","total_number":"int(11)","score":"int(11)","insert_time":"int(10) unsigned","update_time":"int(10) unsigned","date_time":"int(10) unsigned","is_had":"tinyint(3)"},"old":[{"update_time":"1600071500"}],"pkNames":["id"],"sql":"","sqlType":{"id":4,"user_id":12,"task_id":4,"task_name":12,"kind":4,"kind_name":12,"number":4,"total_number":4,"score":4,"insert_time":4,"update_time":4,"date_time":4,"is_had":-6},"table":"tt_user_task","ts":1600071627111,"type":"UPDATE"}
{"data":[{"id":"45320","user_id":"524341983699795968","task_id":"101","task_name":"完成语境中学习15分钟","kind":"1","kind_name":"everyday","number":"1","total_number":"1","score":"0","insert_time":"1600071504","update_time":"1600071504","date_time":"1600012800","is_had":"1"}],"database":"world","es":1600071626000,"id":247,"isDdl":false,"mysqlType":{"id":"int(11)","user_id":"varchar(32)","task_id":"int(11)","task_name":"varchar(32)","kind":"int(11)","kind_name":"varchar(32)","number":"int(11)","total_number":"int(11)","score":"int(11)","insert_time":"int(10) unsigned","update_time":"int(10) unsigned","date_time":"int(10) unsigned","is_had":"tinyint(3)"},"old":[{"update_time":"1600071500"}],"pkNames":["id"],"sql":"","sqlType":{"id":4,"user_id":12,"task_id":4,"task_name":12,"kind":4,"kind_name":12,"number":4,"total_number":4,"score":4,"insert_time":4,"update_time":4,"date_time":4,"is_had":-6},"table":"tt_user_task","ts":1600071627111,"type":"UPDATE"}^C

4、kafka为集群环境(3台服务器三个实例),具体业务为:canal同步mysql,==》 kafka ==》logstash ==》es
5、已尝试删除topic,新建topic再写时依然出现这种状况
6、又遇见过的小伙伴吗,急需解决办法,邮箱:15661685094@163.com