Need help Mutating a Date column from CSV

Elastic Stack Logstash
1

Hi everyone,

I am quite new to using the Stack so I apologize for what may be a quite easy question, I am having trouble figuring this one out!

I am loading in a large CSV file into ES and I am mutating some of the fields to load in properly. I can get all of the data loaded in when I am only mutating for integer and float, but when I try to do DATE & GEO it throws errors. I understand that date is not a supported field type so I know something else needs to be done but I am unsure what. I am trying to mutate a DATE field where it looks like YYYY-MM-DD. Here is my config file:

input {
  file {
    path => "/Users/Celtics2018/documents/data/crime.csv"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}

filter {
    csv{
      separator => ","
      columns =>     ["ID","CaseNumber","Date","Block","IUCR",
      "PrimaryType","Description","LocationDescription",
      "Arrest","Domestic","Beat","District","Ward",
      "CommunityArea","FBI","Code","xCoordinate",
      "yCoordinate","Year","UpdatedOn","Latitude","Longitude","Location"]

    }
    mutate {convert => ["ID", "integer"]}
    **mutate {convert => ["Date", "date"]}**
    mutate {convert => ["IUCR", "integer"]}
    mutate {convert => ["beat", "integer"]}
    mutate {convert => ["District", "integer"]}
    mutate {convert => ["Ward", "integer"]}
    mutate {convert => ["CommunityArea", "integer"]}
    mutate {convert => ["xCoordinate", "integer"]}
    mutate {convert => ["yCoordinate", "integer"]}
    mutate {convert => ["Year", "integer"]}




}

output {
  elasticsearch {
    hosts => "localhost"
    index => "crime"
    document_type => "LA"
  }
  stdout{}
}

Please let me know if I can provide any more information or if I am unclear. Also, I am trying to mutate Lat/Long fields to GEO, so any help on that would be great too :slight_smile:

Thank you!

2

Parsing dates require the use of the date filter, and can not be done through mutate.

This blog post describes how geo_point fields need to be formatted and mapped.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.