Logstash how to mutate string of float array to denseVector

Elastic Stack Logstash
1

I am using Logstash to transfer data from MySQL to ES, one column in MySQL is

`vec` text NOT NULL

vec has value like [0.1, 0.2, 0.3] and is of string type. How to convert string of float array to ES dense_vector?

I tried with the following filter conf, vec value becomes 0.0.

filter {
  mutate {
    copy => { "id" => "[@metadata][_id]"}
    remove_field => ["@version"]
  }
  mutate {
    convert => {
      "vec" => "float"
    }
  }
}

{
    "create_time" => 2023-02-21T16:00:00.000Z,
     "@timestamp" => 2023-03-01T10:05:16.734329758Z,
             "id" => 349542,
            "vec" => 0.0
}
2

I finally solved this by using

filter {
  mutate {
    copy => { "id" => "[@metadata][_id]"}
    remove_field => ["@version"]
  }
  mutate {
    gsub => [ "vec", "[\[\]]", "" ]
    split => { "vec" => "," }
  }
  mutate {
    convert => {
      "vec" => "float"
    }
  }
}
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.