# Not able to import data in elastic through logstash

**URL:** https://discuss.elastic.co/t/not-able-to-import-data-in-elastic-through-logstash/271912
**Category:** Logstash
**Created:** [May 2, 2021, 5:41pm UTC](https://discuss.elastic.co/t/not-able-to-import-data-in-elastic-through-logstash/271912 "2021-05-02T17:41:59Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![ajay8310](https://avatars.discourse-cdn.com/v4/letter/a/51bf81/32.png) [@ajay8310](https://discuss.elastic.co/u/ajay8310)
#### Post date: [May 2, 2021, 5:41pm UTC](https://discuss.elastic.co/t/not-able-to-import-data-in-elastic-through-logstash/271912/1 "2021-05-02T17:41:59Z")

</div>

I am using logstash to import data in elastic search. I am using .conf file to import data in index in elastic. Every time I run the .conf file I get the following message repeatedly and index DOES NOT get created.  
\<  
[DEBUG] 2021-05-02 22:56:26.174 [[main]\<mongodb] mongodb - collection\_data is: {"institutions\_metadeta"=\>{:name=\>"institutions\_metadeta", :last\_id=\>"608ea384a1441c4d77453a12"}}  
[DEBUG] 2021-05-02 22:56:26.199 [[main]\<mongodb] mongodb - Updating watch collections  
[DEBUG] 2021-05-02 22:56:26.279 [[main]\<mongodb] mongodb - Added institutions\_metadeta to the collection list as it matches our collection search  
[DEBUG] 2021-05-02 22:56:26.283 [[main]\<mongodb] mongodb - since table already exists  
[DEBUG] 2021-05-02 22:56:26.290 [[main]\<mongodb] mongodb - placeholder already exists, it is {:table=\>"logstash\_since\_institutions\_metadeta", :place=\>"608ea384a1441c4d77453a12"}  
[DEBUG] 2021-05-02 22:56:26.291 [[main]\<mongodb] mongodb - No new rows. Sleeping. {:time=\>5}  
[DEBUG] 2021-05-02 22:56:27.376 [pool-3-thread-1] jvm - collector name {:name=\>"ParNew"}  
[DEBUG] 2021-05-02 22:56:27.377 [pool-3-thread-1] jvm - collector name {:name=\>"ConcurrentMarkSweep"}  
[DEBUG] 2021-05-02 22:56:29.023 [logstash-pipeline-flush] PeriodicFlush - Pushing flush onto pipeline.  
/\>

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [May 2, 2021, 9:38pm UTC](https://discuss.elastic.co/t/not-able-to-import-data-in-elastic-through-logstash/271912/2 "2021-05-02T21:38:26Z")

</div>

Welcome to our community! 😃

> [@ajay8310](#):
>
> [DEBUG] 2021-05-02 22:56:26.290 [[main]\<mongodb] mongodb - placeholder already exists, it is {:table=\>"logstash\_since\_institutions\_metadeta", :place=\>"608ea384a1441c4d77453a12"}  
> [DEBUG] 2021-05-02 22:56:26.291 [[main]\<mongodb] mongodb - No new rows. Sleeping. {:time=\>5}

That would suggest that it's not seeing new rows because you've run the config before.

Can you post your config?

---

<div class="post-metadata">

### Author: ![ajay8310](https://avatars.discourse-cdn.com/v4/letter/a/51bf81/32.png) [@ajay8310](https://discuss.elastic.co/u/ajay8310)
#### Post date: [May 12, 2021, 5:43pm UTC](https://discuss.elastic.co/t/not-able-to-import-data-in-elastic-through-logstash/271912/3 "2021-05-12T17:43:36Z")

</div>

Thanks for responding. I have figured out the issue.

---

<div class="post-metadata">

### Author: ![ajay8310](https://avatars.discourse-cdn.com/v4/letter/a/51bf81/32.png) [@ajay8310](https://discuss.elastic.co/u/ajay8310)
#### Post date: [May 12, 2021, 5:47pm UTC](https://discuss.elastic.co/t/not-able-to-import-data-in-elastic-through-logstash/271912/4 "2021-05-12T17:47:45Z")

</div>

But now I am running into another issue. I am importing data from an collection named " **nad\_partner\_api\_dump**" of mongodb into an index. But since in the source DB there are two more collections named "nad\_partner\_api\_dump\_09\_april\_21" and  
"nad\_partner\_api\_dump\_backup\_March\_17\_2021", data from those collections are also getting imported automatically. But I want data to be imported from only **nad\_partner\_api\_dump**. Please help.

---

<div class="post-metadata">

### Author: ![ajay8310](https://avatars.discourse-cdn.com/v4/letter/a/51bf81/32.png) [@ajay8310](https://discuss.elastic.co/u/ajay8310)
#### Post date: [May 12, 2021, 6:13pm UTC](https://discuss.elastic.co/t/not-able-to-import-data-in-elastic-through-logstash/271912/5 "2021-05-12T18:13:32Z")

</div>

The config file is as follows -  
input {  
mongodb {  
uri =\> \<connection\_string\>  
placeholder\_db\_dir =\> "/opt/logstash/"  
placeholder\_db\_name =\> "ufi\_logstash\_sqlite.db"  
collection =\> "upload\_file\_info"  
batch\_size =\> 200  
}  
}

filter {  
mutate {  
rename =\> { "\_id" =\> "mongo\_id" }  
}  
}

output {  
elasticsearch {  
hosts =\> ["[http://127.0.0.1:9200](http://127.0.0.1:9200)"]  
action =\> "index"  
index =\> "upload\_file\_info"  
}  
}

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 12, 2021, 7:12pm UTC](https://discuss.elastic.co/t/not-able-to-import-data-in-elastic-through-logstash/271912/6 "2021-05-12T19:12:16Z")

</div>

According to the [code](https://github.com/phutchins/logstash-input-mongodb/blob/097cc9be8bb57754ec4006e174f0cb5cf3ff6b65/lib/logstash/inputs/mongodb.rb#L41), the collection option is "The collection to use. Is turned into a regex so 'events' will match 'events\_20150227'".

I have no idea if it will work but you could try `collection => "upload_file_info$"`

---

<div class="post-metadata">

### Author: ![ajay8310](https://avatars.discourse-cdn.com/v4/letter/a/51bf81/32.png) [@ajay8310](https://discuss.elastic.co/u/ajay8310)
#### Post date: [May 13, 2021, 8:06am UTC](https://discuss.elastic.co/t/not-able-to-import-data-in-elastic-through-logstash/271912/7 "2021-05-13T08:06:34Z")

</div>

Thanks that have worked. But now I am facing another issue. I have first imported data from **upload\_file\_info** collections and then I have imported data from **institutions** collection into a separate index. The moment I started importing data in institutions collection, the same data also got inserted in upload\_file\_info collection automatically. The conf file is similar as I shared before. Please help.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 13, 2021, 3:19pm UTC](https://discuss.elastic.co/t/not-able-to-import-data-in-elastic-through-logstash/271912/8 "2021-05-13T15:19:16Z")

</div>

You may be hitting [this](https://discuss.elastic.co/t/duplicated-log/269931/4).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 10, 2021, 3:19pm UTC](https://discuss.elastic.co/t/not-able-to-import-data-in-elastic-through-logstash/271912/9 "2021-06-10T15:19:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.