# Getting this error while ingesting data from Mongodb into Elasticsearch

**URL:** <https://discuss.elastic.co/t/getting-this-error-while-ingesting-data-from-mongodb-into-elasticsearch/279651>\
**Category:** Logstash\
**Created:** [July 26, 2021, 3:29pm UTC](https://discuss.elastic.co/t/getting-this-error-while-ingesting-data-from-mongodb-into-elasticsearch/279651 "2021-07-26T15:29:52Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Husnain](https://avatars.discourse-cdn.com/v4/letter/h/a9a28c/32.png) [@Husnain](https://discuss.elastic.co/u/Husnain)\
**Post date:** [July 26, 2021, 3:29pm UTC](https://discuss.elastic.co/t/getting-this-error-while-ingesting-data-from-mongodb-into-elasticsearch/279651/1 "2021-07-26T15:29:52Z")

</div>

Hii.I am trying to ingest data from Mongodb into Elasticsearch using logstash pipeline.I am using "logstash-input-mongodb" plugin to ingest data from Mongodb into Elasticsearch.This is my configuration pipeline.

```auto
input {

    mongodb {

        uri => 'mongodb://localhost:27017/mydata'

        placeholder_db_dir => '/root/logstash-mongodb'

        placeholder_db_name => 'logstash_sqlite.db'

        collection => 'mylappy'

        batch_size => 50

    }

}

filter {

    mutate {

        remove_field => ["_id"]

    }

}

output {

    stdout { 

        codec => rubydebug 

    }

    elasticsearch {

        hosts => ["http://localhost:9200"]

        user => "elastic"

        password => "password"

        index => "mongodocs"

    }

}

```

I am getting this Error after running Logstash Pipeline:-

> D, [2021-07-26T20:54:19.438557 #3607] DEBUG -- : MONGODB | QUERY | namespace=mydata.mylappy selector={:\_id=\>{:$gt=\>BSON::ObjectId('600599437bd4eecfee0db02a')}} flags=[:slave\_ok] limit=50 skip=0 project=nil | runtime: 2.2900ms  
> D, [2021-07-26T20:54:19.450360 #3607] DEBUG -- : MONGODB | COMMAND | namespace=mydata.$cmd selector={:listCollections=\>1, :cursor=\>{}, :filter=\>{:name=\>{"$not"=\>/system.|$/}}} flags=[:slave\_ok] limit=-1 skip=0 project=nil | runtime: 2.5320ms

> D, [2021-07-26T20:54:22.511500 #3607] DEBUG -- : MONGODB | COMMAND | namespace=admin.$cmd selector={:ismaster=\>1} flags= limit=-1 skip=0 project=nil | runtime: 2.8701ms

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 26, 2021, 5:13pm UTC](https://discuss.elastic.co/t/getting-this-error-while-ingesting-data-from-mongodb-into-elasticsearch/279651/2 "2021-07-26T17:13:00Z")

</div>

Where is the error? Those are just DEBUG messages.

---

<div class="post-metadata">

**Author:** ![Husnain](https://avatars.discourse-cdn.com/v4/letter/h/a9a28c/32.png) [@Husnain](https://discuss.elastic.co/u/Husnain)\
**Post date:** [July 26, 2021, 6:59pm UTC](https://discuss.elastic.co/t/getting-this-error-while-ingesting-data-from-mongodb-into-elasticsearch/279651/3 "2021-07-26T18:59:33Z")

</div>

@Badger Thanks for the reply.I am trying to put documents in elasticsearch from mongodb,but out of 3 documents only 1 document is getting stored in elasticsearch.Rest are not.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 26, 2021, 7:14pm UTC](https://discuss.elastic.co/t/getting-this-error-while-ingesting-data-from-mongodb-into-elasticsearch/279651/4 "2021-07-26T19:14:21Z")

</div>

If there are three documents in the MongoDB collection then you should only expect two of them to get indexed. When the [code initializes](https://github.com/phutchins/logstash-input-mongodb/blob/097cc9be8bb57754ec4006e174f0cb5cf3ff6b65/lib/logstash/inputs/mongodb.rb#L91) the placeholder that it uses to track progress it sorts the documents in the collection and saves the oldest value (of \_id by default). When it runs a query to fetch data it fetches anything greater than that value, so the first document is never indexed.

Showing all of the DEBUG and INFO messages from the input might tell us more.

---

<div class="post-metadata">

**Author:** ![Husnain](https://avatars.discourse-cdn.com/v4/letter/h/a9a28c/32.png) [@Husnain](https://discuss.elastic.co/u/Husnain)\
**Post date:** [July 27, 2021, 2:01pm UTC](https://discuss.elastic.co/t/getting-this-error-while-ingesting-data-from-mongodb-into-elasticsearch/279651/5 "2021-07-27T14:01:26Z")

</div>

Hii @Badger .Thanks Alot for explaining this in detail.Is there any way that we can retrieve all the documents from Mongodb?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 24, 2021, 2:01pm UTC](https://discuss.elastic.co/t/getting-this-error-while-ingesting-data-from-mongodb-into-elasticsearch/279651/6 "2021-08-24T14:01:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.