# How to pass temporary data from filter to output in logstash **URL:** \ **Category:** Logstash\ **Created:** [April 5, 2018, 11:13am UTC](https://discuss.elastic.co/t/how-to-pass-temporary-data-from-filter-to-output-in-logstash/126903 "2018-04-05T11:13:02Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![EloaCardoso](https://avatars.discourse-cdn.com/v4/letter/e/7cd45c/32.png) [@EloaCardoso](https://discuss.elastic.co/u/EloaCardoso)\ **Post date:** [April 5, 2018, 11:13am UTC](https://discuss.elastic.co/t/how-to-pass-temporary-data-from-filter-to-output-in-logstash/126903/1 "2018-04-05T11:13:02Z")
How can I pass temporary data from a filter to output in logstash? I did like this: ``` filter { mutate { add_field => { "[@metadata][TEMP_DATA]" => "%{custom_index}" } } mutate { remove_field => ["custom_index"] } } output { elasticsearch { index => "%{[@metadata][TEMP_DATA]}" } } ``` But it did not work. Does anyone have an idea how I could do that? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 5, 2018, 11:14am UTC](https://discuss.elastic.co/t/how-to-pass-temporary-data-from-filter-to-output-in-logstash/126903/2 "2018-04-05T11:14:15Z")
In what way did it not work? ---
**Author:** ![EloaCardoso](https://avatars.discourse-cdn.com/v4/letter/e/7cd45c/32.png) [@EloaCardoso](https://discuss.elastic.co/u/EloaCardoso)\ **Post date:** [April 5, 2018, 11:23am UTC](https://discuss.elastic.co/t/how-to-pass-temporary-data-from-filter-to-output-in-logstash/126903/3 "2018-04-05T11:23:28Z")
It is weird because I can see at the Monitoring page in Kibana that the Logstash Node is receiving data. But when I go into the Discover page in Kibana I do not see any data. I have created an index in Elasticsearch and when I check the index in Kibana I don't see any data. I am using Logstash 5.5.2. ---
**Author:** ![EloaCardoso](https://avatars.discourse-cdn.com/v4/letter/e/7cd45c/32.png) [@EloaCardoso](https://discuss.elastic.co/u/EloaCardoso)\ **Post date:** [April 5, 2018, 11:30am UTC](https://discuss.elastic.co/t/how-to-pass-temporary-data-from-filter-to-output-in-logstash/126903/4 "2018-04-05T11:30:40Z")
The output plugin looks as follows: ``` output { stdout { codec => rubydebug } elasticsearch { codec => json manage_template => false hosts => ["localhost:9200"] user => "elastic" password => "changeme" index => "%{[@metadata][TEMP_DATA]}" document_type => "ObjectEvent" document_id => "%{epc}" } } ``` ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 5, 2018, 11:48am UTC](https://discuss.elastic.co/t/how-to-pass-temporary-data-from-filter-to-output-in-logstash/126903/5 "2018-04-05T11:48:25Z")
Use Elasticsearch's cat indices API to check what indices you have. Do you by any chance have an index named, literally, %{[@metadata][TEMP\_DATA]}? Or %{custom\_index}? ---
**Author:** ![EloaCardoso](https://avatars.discourse-cdn.com/v4/letter/e/7cd45c/32.png) [@EloaCardoso](https://discuss.elastic.co/u/EloaCardoso)\ **Post date:** [April 5, 2018, 12:03pm UTC](https://discuss.elastic.co/t/how-to-pass-temporary-data-from-filter-to-output-in-logstash/126903/6 "2018-04-05T12:03:22Z")
I already have done that and nothing shows up. I have used the stdout output plugin to display what I actually want to send to elasticsearch and it displays everything perfectly. ---
**Author:** ![EloaCardoso](https://avatars.discourse-cdn.com/v4/letter/e/7cd45c/32.png) [@EloaCardoso](https://discuss.elastic.co/u/EloaCardoso)\ **Post date:** [April 5, 2018, 12:10pm UTC](https://discuss.elastic.co/t/how-to-pass-temporary-data-from-filter-to-output-in-logstash/126903/7 "2018-04-05T12:10:28Z")
If I have something like this: ``` output { stdout { codec => rubydebug } elasticsearch { codec => json manage_template => false hosts => ["localhost:9200"] user => "elastic" password => "changeme" index => "%{custom_index}" document_type => "ObjectEvent" document_id => "%{epc}" } } ``` And I don't remove the custom\_index field from the filter, it works. But I don't want to send the custom\_index field to elasticsearch. That's why I am using @metadata. ---
**Author:** ![EloaCardoso](https://avatars.discourse-cdn.com/v4/letter/e/7cd45c/32.png) [@EloaCardoso](https://discuss.elastic.co/u/EloaCardoso)\ **Post date:** [April 6, 2018, 8:37am UTC](https://discuss.elastic.co/t/how-to-pass-temporary-data-from-filter-to-output-in-logstash/126903/8 "2018-04-06T08:37:11Z")
Hi @magnusbaeck, I just tried something else. Instead of TEMP\_DATA, I put another used %{[@metadata][endpoint]}. ``` output { stdout { codec => rubydebug } elasticsearch { codec => json manage_template => false hosts => ["localhost:9200"] user => "elastic" password => "changeme" index => "%{[@metadata][endpoint]}" document_type => "ObjectEvent" document_id => "%{epc}" } } ``` and an index %{[@metadata][endpoint]} has been created. But with TEMP\_DATA nothing changes. Do you know an way to give a value to an index and not send this value as a field to elasticsearch? Is this actually possible? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 6, 2018, 10:02am UTC](https://discuss.elastic.co/t/how-to-pass-temporary-data-from-filter-to-output-in-logstash/126903/9 "2018-04-06T10:02:45Z")
Comment out your elasticsearch output and change your stdout output to `stdout { codec => rubydebug { metadata => true } }`. What does an example event look like? ---
**Author:** ![EloaCardoso](https://avatars.discourse-cdn.com/v4/letter/e/7cd45c/32.png) [@EloaCardoso](https://discuss.elastic.co/u/EloaCardoso)\ **Post date:** [April 6, 2018, 10:13am UTC](https://discuss.elastic.co/t/how-to-pass-temporary-data-from-filter-to-output-in-logstash/126903/10 "2018-04-06T10:13:26Z")
I also have done that. But this is not appearing. ![grafik](https://us1.discourse-cdn.com/elastic/original/3X/d/6/d6256b9d24463a3f04f9f2ce6baa5ea5ffe5c574.png) I have actually two @metadata fields. One is [@metadata][EPC\_t] and the another one is [@metadata][endpoint]. Just [@metadata][EPC\_t] is been displayed as you can see. ---
**Author:** ![EloaCardoso](https://avatars.discourse-cdn.com/v4/letter/e/7cd45c/32.png) [@EloaCardoso](https://discuss.elastic.co/u/EloaCardoso)\ **Post date:** [April 6, 2018, 10:29am UTC](https://discuss.elastic.co/t/how-to-pass-temporary-data-from-filter-to-output-in-logstash/126903/11 "2018-04-06T10:29:39Z")
It works! BUT... I don't know why. I changed the position where I write the following code: ``` filter { mutate { add_field => { "[@metadata][endpoint]" => "%{endpoint}" } } mutate { remove_field => ["endpoint"] } } ``` and placed this after the following code: ``` if [EPC_t] { mutate { add_field => { "[@metadata][EPC_t]" => "%{EPC_t}" } } mutate { remove_field => ["EPC_t"] } } ``` If I try to put the addition of [@metadata][endpoint] field somewhere else before the EPC\_t metadata field, it does not work. Maybe I am missing some logstash basics? Or metadata basics? I don't get it. Both fields have nothing to do with each other.... it works... ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 4, 2018, 10:29am UTC](https://discuss.elastic.co/t/how-to-pass-temporary-data-from-filter-to-output-in-logstash/126903/12 "2018-05-04T10:29:47Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.