Add existing fields to my logstash

Elastic Stack Logstash
1

hello !

I would like to be able to add existing fields in an already stored index and to be able to work with them in logstash.

I understand that there are enrichment policies but I would like not to have to use kibana.

Is there some way to have 2 entries in logstash or some way to add all the values ​​of a field from an elastic index?

Thanks in advanced!

2

I think what you are looking for is the elasticsearch input plugin. This will allow you to query an index and retrieve the data you are looking for then you can do your filtering or data enchriment and finally output it back to elasticsearch (using the same ID).

3

i need something similar but not an input.

i'm keeping data with jdcb from an oracle database wich an id-machine and i have other index with all the names of the machines (Catalog) , i want to keep the field machine_name from the preindexed index and use it as a normal field in mi canvas,dashboard etc .

i see a elasticsearch filter plugin but i dont understand well how to add the field with all its values.

thank you!

4

Sound like the filter is what you need then. It will probably be something like this.

filter {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "name-of-index"
    query => "field-name-with-matching-id:%{[JDBC-FIELD-NAME-WITH-ID]}"
    fields => { "new-field-name" => "field-name-of-index-you-want-data-from" }
  }    
}
5

im not doing something well

i want to add the name_machine field to my index alone.

can't i do something like:

  elasticsearch {
    hosts => ["https://xxxxxxxx:9200"]
    index => "indice"
    user => elastic
    password => xxxxxxxx
    ssl => true
    cacert => 'xxxxxxx/elasticsearch-ca.pem'
    fields => { "new-field" => "%{machine_name}" }
  }

using the fields parameter as a add_field of a mutate keeping de names of the machines ??

6

I think you are missing the above part.

In your JDBC input are you getting a field that has an id that would match the id in the elasticsearch index? If so that's the reason for the query. To find that single record you want and then assign the fields.

7

i dont see clear that relation... im sorry.

im going to put a concrete case.

i have and jdbc input keeping from oracle country_name,backups_ok and backups_fail fields

and i have the catalog indexed in elk with a field named machine_name wich is the name of the server of the backups.

I would like to add the field with the names of the machines to create a canvas in which I can see the backups_ok and fail of each machine.

I apologize for my english !! :pray:

8

Does your JDBC input have a field that will match to a specific record in your elastic catalog index?

You need to have a common field between your JDBC input and your index if you are going to enrich the data. That is the field you use to do the query.

9

okey i was creating a common field and i have "country_name" and cod_country with same values in both of them.
Now i want to keep the field machine_name and mix it with my backups statistic from the other.

sorry for taking so long

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.