Elasticsearch删除历史日志

ElasticSearch
运维

环境

• elasticsearch 5.1

操作

首先，贴出我从kafka采集日志到es的相关logstash的配置

input{
  kafka{
  topics => ["logs-normal","logs-error","logs-point"]
     bootstrap_servers => "192.168.x.x:9092,192.168.x.x:9092:9092,192.168.x.x:9092"
     codec => json
     group_id=> "logstash"
     codec => multiline {
        pattern => "\s"
        negate=>true
        what => "previous"
    }
  }
}
filter{
   grok{
        match => {"message" => "\[(?<datetime>\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3})"}
   }
   date{
        match => ["datetime", "yyyy-MM-dd HH:mm:ss,SSS"]
        target => "@timestamp"
   }
   mutate {
    remove_field => ["datetime"]
  }
}
output{
    elasticsearch {
            action => "index"
            hosts  => ["192.168.x.x:9200","192.168.x.x:9200","192.168.x.x:9200"]
            index  => "applog-%{+YYYY.MM.dd}"
    }
}

大家注意 index => "applog-%{+YYYY.MM.dd}",这会根据timestamp的时间来生成每天的日志块，而我删除日志，也是根据索引+日期
来删除的。这样清楚多少天以前的就很简单了。

删除代码

然后根据索引+日期删除日志

curl -XDELETE  "192.168.101.123:9200/applog-2016.12.26"

查看日志存储的位置，磁盘已经释放了。