# Logstash setup with Docker - \[exits after ~25s\]\[not able to find the problem\]

**URL:** <https://discuss.elastic.co/t/logstash-setup-with-docker-exits-after-25s-not-able-to-find-the-problem/281211>\
**Category:** Logstash\
**Tags:** docker\
**Created:** [August 12, 2021, 1:52pm UTC](https://discuss.elastic.co/t/logstash-setup-with-docker-exits-after-25s-not-able-to-find-the-problem/281211 "2021-08-12T13:52:52Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jno\_Zrc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jno_zrc/32/78919_2.png) [@Jno\_Zrc](https://discuss.elastic.co/u/Jno_Zrc)\
**Post date:** [August 12, 2021, 1:52pm UTC](https://discuss.elastic.co/t/logstash-setup-with-docker-exits-after-25s-not-able-to-find-the-problem/281211/1 "2021-08-12T13:52:52Z")

</div>

Hi all,

I'm trying to insert a large csv file in Elasticsearch using Logstash.  
I manage to make it work, but after a while testing some different mutate transformations it stopped working.  
I even tried a minimal config file (see below), what is not working either.  
The log file doesn't show any particular issue but a generic one (`"Expected one of [\\t\\r\\n], \"#\", \"input\", \"filter\", \"output\" at line 1, column 1 (byte 1)"`) which is shown completely in line 22.

Every time I change the config file, I removed the container and create it again. And when it was working, I was deleting the index in elasticsearch (`DELETE /g_a`)

I checked other threads about the Logstash config file and I verified if the config file is in UTF-8 enconding, if there is no wrong syntax, and I even rewrited everything to make sure there were no invisible characters introduced by a copy and paste from some webpage.

**Docker commands:**

```auto
docker network create elasticnet

docker run -d --name elasticsearch --net elasticnet -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" -v d:/elk/elasticsearch/data:/usr/share/elasticsearch/data elasticsearch:7.14.0

docker run -d --name kibana --net elasticnet -p 5601:5601 kibana:7.14.0

docker run -d --name logstash --net elasticnet -v d:/elk/logstash/pipeline:/usr/share/logstash/pipeline -v d:/si/gn:/home logstash:7.14.0

```

**Logstash config file (logstash.conf):**

```auto
input {
  file {
    path => "/home/all/all.txt"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}

filter {
  csv{
      separator => "	"
      skip_header => "false"
      columns => ["gd","name","asciiname","alternatenames","latitude","longitude","feature class","feature code","country code","cc2","admin1 code","admin2 code","admin3 code","admin4 code","population","elevation","dem","timezone","modification date"]
  }
}

output {
   elasticsearch {
     hosts => ["elasticsearch","http://elasticsearch:9200", "elasticsearch:9200", "http://localhost:9200"]
     index => "g_a"
  }
stdout {}
}

```

Logstash logs

> Using bundled JDK: /usr/share/logstash/jdk  
> OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.  
> /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/bundler-1.17.3/lib/bundler/rubygems\_integration.rb:200: warning: constant Gem::ConfigMap is deprecated  
> Sending Logstash logs to /usr/share/logstash/logs which is now configured via log4j2.properties  
> [2021-08-12T13:05:56,512][INFO][logstash.runner] Log4j configuration path used is: /usr/share/logstash/config/log4j2.properties  
> [2021-08-12T13:05:56,522][INFO][logstash.runner] Starting Logstash {"logstash.version"=\>"7.14.0", "jruby.version"=\>"jruby 9.2.19.0 (2.5.8) 2021-06-15 55810c552b OpenJDK 64-Bit Server VM 11.0.11+9 on 11.0.11+9 +indy +jit [linux-x86\_64]"}  
> [2021-08-12T13:05:56,548][INFO][logstash.setting.writabledirectory] Creating directory {:setting=\>"path.queue", :path=\>"/usr/share/logstash/data/queue"}  
> [2021-08-12T13:05:56,560][INFO][logstash.setting.writabledirectory] Creating directory {:setting=\>"path.dead\_letter\_queue", :path=\>"/usr/share/logstash/data/dead\_letter\_queue"}  
> [2021-08-12T13:05:56,835][INFO][logstash.agent] No persistent UUID file found. Generating new UUID {:uuid=\>"e71a12c4-be48-428d-be6c-7f579c34ccaf", :path=\>"/usr/share/logstash/data/uuid"}  
> [2021-08-12T13:05:57,522][WARN][logstash.monitoringextension.pipelineregisterhook] xpack.monitoring.enabled has not been defined, but found elasticsearch configuration. Please explicitly set `xpack.monitoring.enabled: true` in logstash.yml  
> [2021-08-12T13:05:57,524][WARN][deprecation.logstash.monitoringextension.pipelineregisterhook] Internal collectors option for Logstash monitoring is deprecated and targeted for removal in the next major version.  
> Please configure Metricbeat to monitor Logstash. Documentation can be found at:  
> [Collect Logstash monitoring data with Metricbeat | Logstash Reference [8.11] | Elastic](https://www.elastic.co/guide/en/logstash/current/monitoring-with-metricbeat.html)  
> [2021-08-12T13:05:57,805][WARN][deprecation.logstash.outputs.elasticsearch] Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.  
> [2021-08-12T13:05:58,086][INFO][logstash.licensechecker.licensereader] Elasticsearch pool URLs updated {:changes=\>{:removed=\>, :added=\>[[http://elasticsearch:9200/](http://elasticsearch:9200/)]}}  
> [2021-08-12T13:05:58,276][WARN][logstash.licensechecker.licensereader] Restored connection to ES instance {:url=\>"[http://elasticsearch:9200/](http://elasticsearch:9200/)"}  
> [2021-08-12T13:05:58,311][INFO][logstash.licensechecker.licensereader] Elasticsearch version determined (7.14.0) {:es\_version=\>7}  
> [2021-08-12T13:05:58,312][WARN][logstash.licensechecker.licensereader] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>7}  
> [2021-08-12T13:05:58,386][INFO][logstash.monitoring.internalpipelinesource] Monitoring License OK  
> [2021-08-12T13:05:58,387][INFO][logstash.monitoring.internalpipelinesource] Validated license for monitoring. Enabling monitoring pipeline.  
> [2021-08-12T13:05:58,666][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=\>9600}  
> [2021-08-12T13:05:59,079][ERROR][logstash.agent] Failed to execute action {:action=\>LogStash::PipelineAction::Create/pipeline\_id:main, :exception=\>"LogStash::ConfigurationError", :message=\>"Expected one of [\t\r\n], "#", "input", "filter", "output" at line 1, column 1 (byte 1)", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:187:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:72:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:47:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline\_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:391:in `block in converge\_state'"]}  
> [2021-08-12T13:05:59,152][INFO][org.reflections.Reflections] Reflections took 57 ms to scan 1 urls, producing 120 keys and 417 values  
> [2021-08-12T13:05:59,631][WARN][deprecation.logstash.outputs.elasticsearchmonitoring] Relying on default value of `pipeline.ecs_compatibility`, which may change in a future major release of Logstash. To avoid unexpected changes when upgrading Logstash, please explicitly declare your desired ECS Compatibility mode.  
> [2021-08-12T13:05:59,670][INFO][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] New Elasticsearch output {:class=\>"LogStash::Outputs::ElasticSearchMonitoring", :hosts=\>["[http://elasticsearch:9200](http://elasticsearch:9200)"]}  
> [2021-08-12T13:05:59,686][INFO][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Elasticsearch pool URLs updated {:changes=\>{:removed=\>, :added=\>[[http://elasticsearch:9200/](http://elasticsearch:9200/)]}}  
> [2021-08-12T13:05:59,694][WARN][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Restored connection to ES instance {:url=\>"[http://elasticsearch:9200/](http://elasticsearch:9200/)"}  
> [2021-08-12T13:05:59,699][INFO][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Elasticsearch version determined (7.14.0) {:es\_version=\>7}  
> [2021-08-12T13:05:59,700][WARN][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document \_type {:es\_version=\>7}  
> [2021-08-12T13:05:59,739][WARN][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Configuration is data stream compliant but due backwards compatibility Logstash 7.x will not assume writing to a data-stream, default behavior will change on Logstash 8.0 (set `data_stream => true/false` to disable this warning)  
> [2021-08-12T13:05:59,740][WARN][logstash.outputs.elasticsearchmonitoring][.monitoring-logstash] Configuration is data stream compliant but due backwards compatibility Logstash 7.x will not assume writing to a data-stream, default behavior will change on Logstash 8.0 (set `data_stream => true/false` to disable this warning)  
> [2021-08-12T13:05:59,747][WARN][logstash.javapipeline][.monitoring-logstash] 'pipeline.ordered' is enabled and is likely less efficient, consider disabling if preserving event order is not necessary  
> [2021-08-12T13:05:59,792][INFO][logstash.javapipeline][.monitoring-logstash] Starting pipeline {:pipeline\_id=\>".monitoring-logstash", "pipeline.workers"=\>1, "pipeline.batch.size"=\>2, "pipeline.batch.delay"=\>50, "pipeline.max\_inflight"=\>2, "pipeline.sources"=\>["monitoring pipeline"], :thread=\>"#\<Thread:0x5c245967 run\>"}  
> [2021-08-12T13:06:00,330][INFO][logstash.javapipeline][.monitoring-logstash] Pipeline Java execution initialization time {"seconds"=\>0.54}  
> [2021-08-12T13:06:00,350][INFO][logstash.javapipeline][.monitoring-logstash] Pipeline started {"pipeline.id"=\>".monitoring-logstash"}  
> [2021-08-12T13:06:01,732][INFO][logstash.javapipeline][.monitoring-logstash] Pipeline terminated {"pipeline.id"=\>".monitoring-logstash"}  
> [2021-08-12T13:06:02,500][INFO][logstash.runner] Logstash shut down.

Thanks!

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [August 12, 2021, 2:55pm UTC](https://discuss.elastic.co/t/logstash-setup-with-docker-exits-after-25s-not-able-to-find-the-problem/281211/2 "2021-08-12T14:55:44Z")

</div>

See [this](https://discuss.elastic.co/t/logstash-pipeline-error-at-line-1-column-1/280746) as an example of the many threads about this.

---

<div class="post-metadata">

**Author:** ![Jno\_Zrc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jno_zrc/32/78919_2.png) [@Jno\_Zrc](https://discuss.elastic.co/u/Jno_Zrc)\
**Post date:** [August 12, 2021, 5:01pm UTC](https://discuss.elastic.co/t/logstash-setup-with-docker-exits-after-25s-not-able-to-find-the-problem/281211/3 "2021-08-12T17:01:09Z")

</div>

Thank you @Badger  
I didn't know that a file with a completely different name could cause this error.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 9, 2021, 5:02pm UTC](https://discuss.elastic.co/t/logstash-setup-with-docker-exits-after-25s-not-able-to-find-the-problem/281211/4 "2021-09-09T17:02:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.