# When run logstash using the service command it fail to send data to aws elasticsearch

**URL:** <https://discuss.elastic.co/t/when-run-logstash-using-the-service-command-it-fail-to-send-data-to-aws-elasticsearch/202387>\
**Category:** Logstash\
**Created:** [October 4, 2019, 8:10pm UTC](https://discuss.elastic.co/t/when-run-logstash-using-the-service-command-it-fail-to-send-data-to-aws-elasticsearch/202387 "2019-10-04T20:10:14Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Marcos\_Silva](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marcos_silva/32/45160_2.png) [@Marcos\_Silva](https://discuss.elastic.co/u/Marcos_Silva)\
**Post date:** [October 4, 2019, 8:10pm UTC](https://discuss.elastic.co/t/when-run-logstash-using-the-service-command-it-fail-to-send-data-to-aws-elasticsearch/202387/1 "2019-10-04T20:10:14Z")

</div>

Hello, I'm having this issue:  
When I run: `sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/apache.conf` works perfectly and logstash send logs to AWS ES.  
When I run as a service using `sudo initctl logstash start` logstash starts correctly without errors but no data is sent to AWS ES.  
This is really frustrating because we choose AWS ES instead of Elastic Cloud..  
Currently we are running AWS ES v7.1, Kibana version 7.1.1 and Logstash version is 7.4 as the compatibility matrix says it supported [link](https://www.elastic.co/support/matrix#matrix_compatibility). Also we are sending data with Logstash from an Ubuntu 14.04 with the amazon\_es\_output plugin.  
Here are the two different outputs logs when running logstash from command and as a service:  
[logstash\_logs.log](https://github.com/awslabs/logstash-output-amazon_es/files/3692104/logstash_logs.log)

My logstash.yml file is by default and sits in /etc/logstash/

My apache.conf is as follows:

```
input {
  file {
    path => "/var/log/apache2/*.log"
  }
}

filter {
  if [path] =~ "access" {
    mutate { replace => { type => "apache_access" } }
    grok {
      match => { "message" => "%{COMBINEDAPACHELOG}" }
    }
    date {
      match => ["timestamp" , "dd/MMM/yyyy:HH:mm:ss Z"]
    }
  } else if [path] =~ "error" {
    mutate { replace => { type => "apache_error" } }
  } else {
    mutate { replace => { type => "random_logs" } }
  }
}

output {
  if [type] in ["apache_access","random_logs","apache_error"] {
    if [response] =~ /^2\d\d/ {
      amazon_es {
        hosts => ["vpc-xxxx.region.es.amazonaws.com"]
        region => "us-east-1"
        aws_access_key_id => ''
        aws_secret_access_key => ''
        index => "apache-access-logs-%{+YYYY.MM.dd}"
      }
    }
  }
}

```

Any help would be appreciated.  
Marcos.

---

<div class="post-metadata">

**Author:** ![Marcos\_Silva](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marcos_silva/32/45160_2.png) [@Marcos\_Silva](https://discuss.elastic.co/u/Marcos_Silva)\
**Post date:** [October 8, 2019, 1:17pm UTC](https://discuss.elastic.co/t/when-run-logstash-using-the-service-command-it-fail-to-send-data-to-aws-elasticsearch/202387/2 "2019-10-08T13:17:11Z")

</div>

> [@Marcos\_Silva](#):
>
> /var/log/apache2/\*.log

Hi, I have found the issue:

The service starts with sudo but it runs under "logstash" user. So I had to add logstash user to group "adm" to be able to read the logs generated at "/var/log/apache2/\*.log".

I used the command `sudo usermod -a -G adm logstash`

Then I restarted the service and everything is working fine.

Thanks.  
Marcos

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 5, 2019, 1:17pm UTC](https://discuss.elastic.co/t/when-run-logstash-using-the-service-command-it-fail-to-send-data-to-aws-elasticsearch/202387/3 "2019-11-05T13:17:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.