Remove field from logstash configuration

My indices are using the parent-child relationship, therefore my sql query returns parent ID that I use in the logstash configuration to use as the parent ID. Here is my logstash configuration,

output {
  elasticsearch {
    hosts => "https://xxx.us-east-1.es.amazonaws.com" 
	document_type => "segment"
	document_id => "%{id}"
	index => "${LANG_INDEX:en-us_th-th}"
	parent => "%{upload_id}"
    ssl => "true"
    manage_template => false
  }
}

The issue I have is that currently the parent ID is getting indexed in the child document, which I don't need. If I remove the field in logstash [mutate] then the parent field is not able to get the parent ID field.

How do I avoid elasticsearch from indexing this field?

If you make upload_id a subfield of @metadata (i.e. use the mutate filter's rename option to rename upload_id to [@metadata][upload_id]) the field won't get included in the document sent to ES but you'll still be able to reference the field in the parent option).

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.