Logstash를 통한 csv import 오류

안녕하세요. 제가 geo_point 데이터를 2개 쓰고 싶어서 아래와 같이 작성하였는데요. logstash를 실행하면 elasticsearch에 잘 전달한 것 같은데 kibana를 보면 데이터가 들어와있지 않습니다. 어떤 오류인지 모르겠습니다.

스크린샷 2020-06-13 오후 5.46.50 zpRQtoUy8muYIGIyxpu5Fh.png)

input {
  file {
    path => "/Users/user/Downloads/coronavirusdataset_20200601/PatientRoute.csv"
    start_position => "beginning"
    sincedb_path => "/dev/null"    
  }
}
filter {
  csv {
      separator => ","
      columns => ["patient_id","global_num","date","province","city","type","lat","lon","city_lat","city_lon"]
  }
  mutate {convert => ["lat", "float"]}
  mutate {convert => ["lon", "float"]}
  mutate {convert => ["city_lat", "float"]}
  mutate {convert => ["city_lon", "float"]}
  mutate {convert => ["global_num", "float"]}
  mutate {
    add_field => {"[location-specific][lat]" => "%{lat}"}
    add_field => {"[location-specific][lon]" => "%{lon}"}
    add_field => {"[location-city][city_lat]" => "%{city_lat}"}
    add_field => {"[location-city][city_lon]" => "%{city_lon}"}
  }
  mutate {
    convert => {"[location-specific][lat]" => "float"}
    convert => {"[location-specific][lon]" => "float"}
    convert => {"[location-city][city_lat]" => "float"}
    convert => {"[location-city][city_lon]" => "float"}
  }
  
output {  
    elasticsearch {
        hosts => ["http://localhost:9200"]
        index => "map_test"
        user => "elastic"
        password => "비밀번호"
    }
    stdout { codec => rubydebug {metadata => true}}
    stdout { codec => dots }
}

kibana dev tools로 미리 index 설정해두고 logstash 실행시켰습니다.

PUT map_test/_mapping
{
  "properties": {
    "location-specific": {
      "type": "geo_point"
    },
    "location-city": {
      "type": "geo_point"
    }
  }
}

logstash 실행시 여러 메세지가 빠르게 지나가서 겨우 잡았는데요. 다음과 같은 에러 메세지가 나오더군요.. 무엇을 의미하는지 잘 모르겠습니다.ㅠ

[2020-06-13T18:09:19,549][WARN ][logstash.outputs.elasticsearch][main][502f63f93cc9a9a288e0b68fa196048a37c905498f14c83e1504f515a21b5a43] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"map_test", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x2d202e20>], :response=>{"index"=>{"_index"=>"map_test", "_type"=>"_doc", "_id"=>"dHjwrHIBhdLkyYcqwj4c", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [location-city] of type [geo_point]", "caused_by"=>{"type"=>"parse_exception", "reason"=>"field must be either [lat], [lon] or [geohash]"}}}}}

location-city 필드가

location-city : { city_lat: 0.0, city_lon:0.0 }

이렇게 되어 있는데, geo_point 필드로 쓰시려면

location-city : { lat: 0.0, lon: 0.0 } 

이렇게 하셔야 합니다.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.