Storing JSON array field from jdbc input to elastic search

Elastic Stack Logstash
1

My configuration file has a filter which converts majors field

filter {
  json {
    source => "majors"
    remove_field => ["majors"]
  }
}

I get following debug event

{
          "user_id" => 1529067,
        "school_id" => 1458,
       "cluster_id" => 39,
      "degree_type" => "Bachelor's Degree",
       "@timestamp" => 2019-05-23T06:21:16.782Z,
       "updated_at" => 2018-10-18T05:27:57.000Z,
           "majors" => "[\"Accounting\"]",
           "hub_id" => 39,
       "deleted_at" => nil,
    "import_status" => "joined",
         "@version" => "1"
}

looks like majors is a parsed string and on saving to elastic search it saves as a string. I want it to save it as an array. How can I achieve it?? or is it needed to convert the field to array from parsed string?

Thanks in advance

2

What does the input event look like?

3

I am using jdbc-input and passing data from Postgres to elastic. My statement looks like.

statement => "select a.user_id, a.hub_id, a.import_status, a.updated_at, a.deleted_at, b.cluster_id, c.school_id, c.degree_type, c.majors::text
      from user_hubs a
      join user_clusters b on a.user_id = b.user_id
      join user_schools c on a.user_id = c.user_id
      where a.hub_id = 39 and b.cluster_id = 39 and a.updated_at > :sql_last_value;"

where major is a JSON array. I need to save it as an array casting it to text save it as a string in elastic.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.