How to set _parent properly using Logstash 2.0

Elastic Stack Logstash
1

I'm trying to load some tables from a relational database into ES 2.0 using logstash. I've created the apropriate mappings in ES to reflect the relationships between the tables (basically one is the parent with the other two pointing to it using _parent)

Reading the tables with Logstash is no problem using JDBC, creating the the proper parent-child relationships, however, is. It appears that the Elasticsearch output plugin lacks the ability to properly pass along the parent-id field to the _parent field in ES. In analogy to the "document_id" or "routing" parameters, I would expect to be able to use a "parent" parameter, to which I can pass the value of the parent-id from the database table I'm reading in input.

I've been crawling up and down the web to find a workaround. Even though I found lots of suggestions, none of them seem to work with ES 2.0 anymore (including adding the _parent field using filter/mutate/add_field etc.).

Is there any hope for this parameter to be available anytime soon?

2

This may help a bit - https://www.elastic.co/guide/en/elasticsearch/reference/current/_mapping_changes.html#migration-meta-fields

But LS doesn't work with parent/child at all.

3

Thanks for the link. I understand the changes, but at the same time I see no solution unless logstash is enhanced with the ability to set the _parent meta-field in the output.

I reverted to ES 1.7.3 now, where I can pass the _parent field in the document body and in combination with routing the child to the right parent shard, it appears to work. Feels like a hack, however. :smile:

On a side note, is it possible to have logstash generate a nested field from the child tables while reading the parent table using JDBC?

4

I am facing the same problem. Is there a work around now?