# Not able to get expected output from elasticsearch through logstash pipeline
**URL:** https://discuss.elastic.co/t/not-able-to-get-expected-output-from-elasticsearch-through-logstash-pipeline/111696
**Category:** Logstash
**Created:** [December 14, 2017, 7:18am UTC](https://discuss.elastic.co/t/not-able-to-get-expected-output-from-elasticsearch-through-logstash-pipeline/111696 "2017-12-14T07:18:02Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@somkhil](https://discuss.elastic.co/u/somkhil)
#### Post date: [December 14, 2017, 7:18am UTC](https://discuss.elastic.co/t/not-able-to-get-expected-output-from-elasticsearch-through-logstash-pipeline/111696/1 "2017-12-14T07:18:02Z")
HI
I am running logstash pipeline to pull the data from elasticsearch and output it into csv file.
At the very first run it generated output properly. However during subsequent runs (after loading more data/fields into elasticsearch) it is not giving proper output.
Two questions :
1. Is there any setting to clear out earlier offsets/history so that logstash can consider every request as fresh one and pull all the data from elasticsearch based on the specified query.
2. How do I specify document type in the input section of logstash config file. Index name can be specified with "index" attribute. Is there any attribute for specifying document type within index.
input {
elasticsearch {
hosts =\> ["localhost:9200"]
index =\> "xyz"
Thanks.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [December 14, 2017, 7:32am UTC](https://discuss.elastic.co/t/not-able-to-get-expected-output-from-elasticsearch-through-logstash-pipeline/111696/2 "2017-12-14T07:32:19Z")
> Is there any setting to clear out earlier offsets/history so that logstash can consider every request as fresh one and pull all the data from elasticsearch based on the specified query.
The elasticsearch input doesn't store any state so it pulls all data each time.
> How do I specify document type in the input section of logstash config file. Index name can be specified with "index" attribute. Is there any attribute for specifying document type within index.
Use the `query` option?
---
### Author:  [@somkhil](https://discuss.elastic.co/u/somkhil)
#### Post date: [December 14, 2017, 8:53am UTC](https://discuss.elastic.co/t/not-able-to-get-expected-output-from-elasticsearch-through-logstash-pipeline/111696/3 "2017-12-14T08:53:31Z")
> The elasticsearch input doesn't store any state so it pulls all data each time.
Actually I am seeing different behavior. When I ran logstash pipeline initially with 10 fields in the elasticsearch doc. I got all the docs with those 10 fields populated properly in the output csv.
After adding around 30 new fields in the same elasticsearch documents, I am getting most of the earlier fields as empty. Does logstash tries to get the delta somehow.
> Use the `query` option?
> Thanks. I got this working by adding filter on "\_type" field.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [December 14, 2017, 9:47am UTC](https://discuss.elastic.co/t/not-able-to-get-expected-output-from-elasticsearch-through-logstash-pipeline/111696/4 "2017-12-14T09:47:56Z")
> Does logstash tries to get the delta somehow.
No, it doesn't.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 11, 2018, 9:48am UTC](https://discuss.elastic.co/t/not-able-to-get-expected-output-from-elasticsearch-through-logstash-pipeline/111696/5 "2018-01-11T09:48:10Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.