# Logstash-input-mongodb configuration file

**URL:** <https://discuss.elastic.co/t/logstash-input-mongodb-configuration-file/129002>\
**Category:** Logstash\
**Created:** [April 22, 2018, 12:04pm UTC](https://discuss.elastic.co/t/logstash-input-mongodb-configuration-file/129002 "2018-04-22T12:04:01Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nfekih](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nfekih/32/30295_2.png) [@Nfekih](https://discuss.elastic.co/u/Nfekih)\
**Post date:** [April 22, 2018, 12:04pm UTC](https://discuss.elastic.co/t/logstash-input-mongodb-configuration-file/129002/1 "2018-04-22T12:04:01Z")

</div>

Hi all,

I am trying to pull data out of mongodb and to process it with Logstash.

Know that I want to pull all the data from a DB named "test" and so I started with the collection named "student". This is my configuration file.

```
input {
	mongodb {
		uri => 'mongodb://localhost:27017/test'
		placeholder_db_dir => '/opt/logstash-mongodb/'
		placeholder_db_name => 'logstash_sqlite.db'
		collection => 'student'
		batch_size => 202
	}
}

filter {

}

output {
	elasticsearch {
		host => "localhost:9200"
		user => elastic
		password => changeme
		index => "testStudent"
	}
	stdout { codec => rubydebug }
}

```

I am having an error saying:  
`Pipelines YAML file is empty.`

Is this because I left the filter part empty?

---

<div class="post-metadata">

**Author:** ![Jemli\_Fathi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jemli_fathi/32/10714_2.png) [@Jemli\_Fathi](https://discuss.elastic.co/u/Jemli_Fathi)\
**Post date:** [April 22, 2018, 3:46pm UTC](https://discuss.elastic.co/t/logstash-input-mongodb-configuration-file/129002/2 "2018-04-22T15:46:34Z")

</div>

Where did you put this configuration?  
You need to put it in a .conf file under this path (if you installed logstash as a service in Linux) /etc/logstash/conf.d/

---

<div class="post-metadata">

**Author:** ![Nfekih](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nfekih/32/30295_2.png) [@Nfekih](https://discuss.elastic.co/u/Nfekih)\
**Post date:** [April 22, 2018, 4:01pm UTC](https://discuss.elastic.co/t/logstash-input-mongodb-configuration-file/129002/3 "2018-04-22T16:01:58Z")

</div>

Logstash is installed from a TGZ  
I put the config file in `~/Downloads/logstash/config`

---

<div class="post-metadata">

**Author:** ![Jemli\_Fathi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jemli_fathi/32/10714_2.png) [@Jemli\_Fathi](https://discuss.elastic.co/u/Jemli_Fathi)\
**Post date:** [April 22, 2018, 5:04pm UTC](https://discuss.elastic.co/t/logstash-input-mongodb-configuration-file/129002/4 "2018-04-22T17:04:31Z")

</div>

And how you started Logstash?  
Can you please check the logs and see if there is any abnormal behavior

---

<div class="post-metadata">

**Author:** ![Nfekih](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nfekih/32/30295_2.png) [@Nfekih](https://discuss.elastic.co/u/Nfekih)\
**Post date:** [April 22, 2018, 6:20pm UTC](https://discuss.elastic.co/t/logstash-input-mongodb-configuration-file/129002/5 "2018-04-22T18:20:17Z")

</div>

started logstash with this cmd:  
`bin/logstash --path.data file_name.conf`

I actually tried to load csv data with this exact same process: the config file in `logstash/config` + the same cmd

---

<div class="post-metadata">

**Author:** ![Jemli\_Fathi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jemli_fathi/32/10714_2.png) [@Jemli\_Fathi](https://discuss.elastic.co/u/Jemli_Fathi)\
**Post date:** [April 22, 2018, 9:01pm UTC](https://discuss.elastic.co/t/logstash-input-mongodb-configuration-file/129002/6 "2018-04-22T21:01:15Z")

</div>

Apparently, you need to configure [Logstash pipelines](https://www.elastic.co/guide/en/logstash/current/multiple-pipelines.html) when you're configuring more than one pipeline

---

<div class="post-metadata">

**Author:** ![Nfekih](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nfekih/32/30295_2.png) [@Nfekih](https://discuss.elastic.co/u/Nfekih)\
**Post date:** [April 22, 2018, 10:42pm UTC](https://discuss.elastic.co/t/logstash-input-mongodb-configuration-file/129002/7 "2018-04-22T22:42:45Z")

</div>

within logstash/config there is the `piplines.yml` file and here is what it contains:

> ```
> # List of pipelines to be loaded by Logstash
> #
> # This document must be a list of dictionaries/hashes, where the keys/values are pipeline settings.
> # Default values for ommitted settings are read from the `logstash.yml` file.
> # When declaring multiple pipelines, each MUST have its own `pipeline.id`.
> #
> # Example of two pipelines:
> #
> # - pipeline.id: test
> # pipeline.workers: 1
> # pipeline.batch.size: 1
> # config.string: "input { generator {} } filter { sleep { time => 1 } } output { stdout { codec => dots } }"
> # - pipeline.id: another_test
> # queue.type: persisted
> # path.config: "/tmp/logstash/*.config"
> #
> # Available options:
> #
> # # name of the pipeline
> # pipeline.id: mylogs
> #
> # # The configuration string to be used by this pipeline
> # config.string: "input { generator {} } filter { sleep { time => 1 } } output { stdout { codec => dots } }"
> #
> # # The path from where to read the configuration text
> # path.config: "/etc/conf.d/logstash/myconfig.cfg"
> #
> # # How many worker threads execute the Filters+Outputs stage of the pipeline
> # pipeline.workers: 1 (actually defaults to number of CPUs)
> #
> # # How many events to retrieve from inputs before sending to filters+workers
> # pipeline.batch.size: 125
> #
> # # How long to wait in milliseconds while polling for the next event
> # # before dispatching an undersized batch to filters+outputs
> # pipeline.batch.delay: 50
> #
> # # How many workers should be used per output plugin instance
> # pipeline.output.workers: 1
> #
> # # Internal queuing model, "memory" for legacy in-memory based queuing and
> # # "persisted" for disk-based acked queueing. Defaults is memory
> # queue.type: memory
> #
> # # If using queue.type: persisted, the page data files size. The queue data consists of
> # # append-only data files separated into pages. Default is 64mb
> # queue.page_capacity: 64mb
> #
> # # If using queue.type: persisted, the maximum number of unread events in the queue.
> # # Default is 0 (unlimited)
> # queue.max_events: 0
> #
> # # If using queue.type: persisted, the total capacity of the queue in number of bytes.
> # # Default is 1024mb or 1gb
> # queue.max_bytes: 1024mb
> #
> # # If using queue.type: persisted, the maximum number of acked events before forcing a checkpoint
> # # Default is 1024, 0 for unlimited
> # queue.checkpoint.acks: 1024
> #
> # # If using queue.type: persisted, the maximum number of written events before forcing a checkpoint
> # # Default is 1024, 0 for unlimited
> # queue.checkpoint.writes: 1024
> #
> # # If using queue.type: persisted, the interval in milliseconds when a checkpoint is forced on the head page
> # # Default is 1000, 0 for no periodic checkpoint.
> # queue.checkpoint.interval: 1000
> #
> # # Enable Dead Letter Queueing for this pipeline.
> # dead_letter_queue.enable: false
> #
> # If using dead_letter_queue.enable: true, the maximum size of dead letter queue for this pipeline. Entries
> # will be dropped if they would increase the size of the dead letter queue beyond this setting.
> # Default is 1024mb
> # dead_letter_queue.max_bytes: 1024mb
> #
> # If using dead_letter_queue.enable: true, the directory path where the data files will be stored.
> # Default is path.data/dead_letter_queue
> #
> # path.dead_letter_queue:
> 
> ```

I did not make any changes to this pipelines.yml file because I am not running multiple pipelines (at least as for now).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 20, 2018, 10:53pm UTC](https://discuss.elastic.co/t/logstash-input-mongodb-configuration-file/129002/8 "2018-05-20T22:53:47Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.