# Logstash in Docker Container to ES Cloud

**URL:** <https://discuss.elastic.co/t/logstash-in-docker-container-to-es-cloud/135060>\
**Category:** Logstash\
**Created:** [June 7, 2018, 10:27pm UTC](https://discuss.elastic.co/t/logstash-in-docker-container-to-es-cloud/135060 "2018-06-07T22:27:17Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![jenyphur](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenyphur/32/40735_2.png) [@jenyphur](https://discuss.elastic.co/u/jenyphur)\
**Post date:** [June 7, 2018, 10:27pm UTC](https://discuss.elastic.co/t/logstash-in-docker-container-to-es-cloud/135060/1 "2018-06-07T22:27:17Z")

</div>

Hi there, new so please go easy.

I have logstash deployed in docker container and need it to send logs to our ElasticSearch in the Cloud account.

In the logstash config, I see where you can specify the output as such:

output {  
elasticsearch { hosts =\> ["localhost:9200"] }  
stdout { codec =\> rubydebug }  
}

Do I just replace the host with the endpoint of the cloud ES for my account? If so, how do I specifiy the userid and pw as well?

Where I am getting confused is the documentation says to use the logstash.yml file and to set the Cloud ID settings there. Do you have to do it in both? Or just one or the other?

Thanks for any/all help.

Jen

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 8, 2018, 4:37am UTC](https://discuss.elastic.co/t/logstash-in-docker-container-to-es-cloud/135060/2 "2018-06-08T04:37:23Z")

</div>

The settings in the `logstash.yml` file specify where monitoring data is to be sent if you have installed X-Pack and enabled this. For data in a pipeline, you need to configure username and password in the Elasticsearch output plugin.

---

<div class="post-metadata">

**Author:** ![jenyphur](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenyphur/32/40735_2.png) [@jenyphur](https://discuss.elastic.co/u/jenyphur)\
**Post date:** [June 11, 2018, 5:45pm UTC](https://discuss.elastic.co/t/logstash-in-docker-container-to-es-cloud/135060/3 "2018-06-11T17:45:05Z")

</div>

Yes, sorry I do realize that. But we are deploying the container using docker-compose.yml. In that file, we are using a 'Secret' (instead of pointing to a config file ) and the contents of the secret are below. Please see both the docker-compose file contents as well as the 'secret' contents below:

**_Docker-compose.yml_**  
version: "3.3"  
services:  
logstash:  
image: [dtr.qcorpaa.aa.com/etds/logstash:1.0](http://dtr.qcorpaa.aa.com/etds/logstash:1.0)  
entrypoint: "logstash -f /etc/logstash/conf.d/inputjen.conf"  
ports:  
- 5000  
networks:  
- etds-logging-jen  
deploy:  
replicas: 1  
labels:  
com.docker.lb.hosts: [logstashet2.ecaas.qcorpaa.aa.com](http://logstashet2.ecaas.qcorpaa.aa.com)  
com.docker.lb.network: etds-logging-jen  
com.docker.lb.port: 8095  
com.docker.lb.ssl\_cert: ecaas-bundle-v2  
com.docker.lb.ssl\_key: ecaas-key-v1  
com.docker.ucp.access.label: /orgs/etds  
restart\_policy:  
condition: on-failure  
delay: 30s  
max\_attempts: 10  
window: 300s  
environment:  
METADATA: proxy-handles-tls  
secrets:  
- source: jen  
target: /etc/logstash/conf.d/inputjen.conf

secrets:  
jen:  
external: true

networks:  
etds-logging-jen:  
driver: overlay

**_Our secret called 'jen':_**

input {  
tcp {  
port =\> 5000  
type =\> syslog  
}  
udp {  
port =\> 5000  
type =\> syslog  
}  
}

filter  
{  
if [type] == "syslog" {  
grok  
{  
match =\> { "message" =\> "(?\<cf\_logid\>\d{3}) \<(?\<cf\_pri\>[0-9]{1,5})\>1 (?\<cf\_time\>[^]+) (?\<cf\_host\>[^]+) (?\<cf\_msgid\>[^]+) (?\<cf\_procid\>[^]+) - - %{TIMESTAMP\_ISO8601:app\_ti mestamp} (?\<cf\_offset\>[\d\s]{1,7}) %{DATA:app\_loglevel} %{DATA:app\_tranid} %{DATA:app\_client\_tranid} %{DATA:app\_servername} %{DATA:app\_version} %{DATA:app\_recordLoc} %{DATA:app\_className} - %{GREEDYDATA:app\_message}" }  
}  
date  
{  
locale =\> "en"  
match =\> ["logdate", "MMM dd yyyy HH:mm:ss", "MMM d yyyy HH:mm:ss", "ISO8601"]  
target =\> "@timestamp"  
}  
mutate  
{  
convert =\> { "cf\_offset" =\> "integer" }  
remove\_field =\> ["app\_timestamp"]  
}  
}  
}

output  
{  
elasticsearch  
{  
hosts =\> ["[1663e7cf9c704d7b8768cb050b816993.us-west-1.aws.found.io:9243](http://1663e7cf9c704d7b8768cb050b816993.us-west-1.aws.found.io:9243)"]  
manage\_template =\> false  
index =\> "filebeat-%{+YYYY.MM.dd}"  
user =\> "elastic"  
password =\> ""  
ssl =\> true  
}  
stdout { codec =\> rubydebug }  
}

The question I have is where can I (or CAN I?) also use a logstash.yml file as apparently I also need to specify that we are using x-pack such as:

xpack.monitoring.elasticsearch.username: logstash\_system  
xpack.monitoring.elasticsearch.password: logstashpassword

Hopefully this makes sense. ☹

Jen

---

<div class="post-metadata">

**Author:** ![jenyphur](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jenyphur/32/40735_2.png) [@jenyphur](https://discuss.elastic.co/u/jenyphur)\
**Post date:** [June 16, 2018, 10:15pm UTC](https://discuss.elastic.co/t/logstash-in-docker-container-to-es-cloud/135060/4 "2018-06-16T22:15:10Z")

</div>

I was able to finally figure it out and get connected. We just needed the proxy settings.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 14, 2018, 10:15pm UTC](https://discuss.elastic.co/t/logstash-in-docker-container-to-es-cloud/135060/5 "2018-07-14T22:15:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.