# Metricbeat: upload output file to elasticsearch manually **URL:** \ **Category:** Beats\ **Tags:** metricbeat\ **Created:** [February 28, 2017, 4:33pm UTC](https://discuss.elastic.co/t/metricbeat-upload-output-file-to-elasticsearch-manually/76825 "2017-02-28T16:33:22Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![WiemDW](https://avatars.discourse-cdn.com/v4/letter/w/ba8739/32.png) [@WiemDW](https://discuss.elastic.co/u/WiemDW)\ **Post date:** [February 28, 2017, 4:33pm UTC](https://discuss.elastic.co/t/metricbeat-upload-output-file-to-elasticsearch-manually/76825/1 "2017-02-28T16:33:22Z")
Hi, I'd like to post-process the metricbeat output, so I am using the "File output" module. However, when uploading the data to elasticsearch, I am getting an "illegal\_argument\_exception": A single example message taken from metricbeat.1 file ```python msg = '{"@timestamp":"2017-02-21T21:33:35.659Z","beat":{"hostname":"elk1","name":"elk1","version":"5.2.1"},"metricset":{"module":"system","name":"cpu","rtt":53},"system":{"cpu":{"cores":1,"idle":{"pct":0.000000},"iowait":{"pct":0.000000},"irq":{"pct":0.000000},"nice":{"pct":0.000000},"softirq":{"pct":0.000000},"steal":{"pct":0.000000},"system":{"pct":0.000000},"user":{"pct":0.000000}}},"type":"metricsets"}' ``` PUT on a new index fails: ```auto In [1]: requests.put('http://elk:9200/metricbeat-1/', data=msg).text Out[1]: u'{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"unknown setting [index.@timestamp] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"}],"type":"illegal_argument_exception","reason":"unknown setting [index.@timestamp] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"},"status":400}' ``` Additional: - Using version 5.2.2 of elasticsearch and metricbeat - No indices, mappings, index templates exist in elasticsearch (clean installation) - Whether the default metricbeat.template.json is loaded or not, does not seem to make a difference. How can I manually upload the metricbeat output file to the elasticsearch ingest node, so that it will look like a regular metricbeat client? Kind regards, Wiem DW ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [March 1, 2017, 8:34am UTC](https://discuss.elastic.co/t/metricbeat-upload-output-file-to-elasticsearch-manually/76825/2 "2017-03-01T08:34:15Z")
TBH I never tried that and the file output was no designed with this in mind. I'm not sure why you get a plugin error above but one thing that is probably missing from the event is the type. What is the client you are using above? ---
**Author:** ![WiemDW](https://avatars.discourse-cdn.com/v4/letter/w/ba8739/32.png) [@WiemDW](https://discuss.elastic.co/u/WiemDW)\ **Post date:** [March 1, 2017, 8:55am UTC](https://discuss.elastic.co/t/metricbeat-upload-output-file-to-elasticsearch-manually/76825/3 "2017-03-01T08:55:56Z")
The type seems to be set to "metricsets" (see the end of the example 'msg='). I am using python-requests for testing, so basically just a plain HTTP PUT call. ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [March 3, 2017, 11:56am UTC](https://discuss.elastic.co/t/metricbeat-upload-output-file-to-elasticsearch-manually/76825/4 "2017-03-03T11:56:54Z")
Can you remove the @timestamp from the event again and see if that works? Are you using any ingest pipeline in ES? ---
**Author:** ![WiemDW](https://avatars.discourse-cdn.com/v4/letter/w/ba8739/32.png) [@WiemDW](https://discuss.elastic.co/u/WiemDW)\ **Post date:** [March 6, 2017, 10:34am UTC](https://discuss.elastic.co/t/metricbeat-upload-output-file-to-elasticsearch-manually/76825/5 "2017-03-06T10:34:26Z")
When I remove the '@timestamp' field, the error remains the same but about the 'unknown setting [index.beat.hostname]" field. I am not using any pipeline or mapping, just a auto-created index. ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [March 6, 2017, 3:34pm UTC](https://discuss.elastic.co/t/metricbeat-upload-output-file-to-elasticsearch-manually/76825/6 "2017-03-06T15:34:13Z")
Could you try to put the document into elasticsearch with just using `curl` and see what happens? Which ES version are you using? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 3, 2017, 3:34pm UTC](https://discuss.elastic.co/t/metricbeat-upload-output-file-to-elasticsearch-manually/76825/7 "2017-04-03T15:34:24Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.