Joint database table and an array

Elastic Stack Logstash
#1

I have a database table and a document array

+---------+--------+
| m_id    | state  |
+---------+--------+
| m1      | s1     |
| m2      | s2     |
| m3      | s1     |
+---------+--------+

state_array = array(s1=>'tamilnadu',s2=>'kerala')

How to joint and create a single type in elastic search using logstash.

+---------+------------+
| m_id    | state      |
+---------+------------+
| m1      | tamilnadu  |
| m2      | kerala     |
| m3      | tamilnadu  |
+---------+------------+

Give me any suggestions for this kind of situation.
My core process is analyzing this kind of data by Kibana.

#2

I suspect the translate filter can help you.

#3

Would you please give me an example for the same

#4

Untested:

translate {
  dictionary => {
    "s1" => "tamilnadu"
    "s2" => "kerala"
  }
  field => "state"
  destination => "state"
}
1 Like
#5

Thank you Magnus.
I tested it, destination is not work.
so I try this for the following table and configuration.

+-----------+----------+---------+
| Name      |  gender  |  state  |
+-----------+----------+---------+
| croos     |  1       |  2      |
| priya     |  2       |  1      |
+-----------+----------+---------+  



filter {
  translate {
    dictionary => {
      "1" => "male"
      "2" => "female"
    }
    field => "gender"
  }

  mutate {  
    rename => { "translation" => "gender" }
  }
}

filter {
  translate {
    dictionary => {
      "1" => "tamilnadu"
      "2" => "karnadaka"
    }
    field => "state"
  }

  mutate {    
    rename => { "translation" => "state" }
  }
}

it is work fine. The output is :

{
            "id" => 1,
          "name" => "croos",
        "gender" => "male",
         "state" => "karnadaka",
      "@version" => "1",
    "@timestamp" => "2016-03-23T07:30:40.322Z"
}
{
            "id" => 2,
          "name" => "priya",
        "gender" => "female",
         "state" => "tamilnadu",
      "@version" => "1",
    "@timestamp" => "2016-03-23T07:30:40.323Z"
}

I think that is not a proper way to do this. Any suggestions for do the same with in a single filter.

#6

Now it is work fine. Thank you very much Magnus.

filter {
  translate {
    dictionary => {
      "1" => "tamilnadu"
      "2" => "karnadaka"
    }
    field => "state"
    destination=>"state1"
  }

  translate {
    dictionary => {
      "1" => "male"
      "2" => "female"
    }
    field => "gender"
    destination=>"gender1"
  }

mutate {    
      remove_field=>["gender","state"]
    }
}
#7