# Curator returns empty output for elasticsearch user

**URL:** <https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112>\
**Category:** Elasticsearch\
**Tags:** curator\
**Created:** [February 12, 2019, 8:24pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112 "2019-02-12T20:24:43Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![George\_Wainwright](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/george_wainwright/32/51054_2.png) [@George\_Wainwright](https://discuss.elastic.co/u/George_Wainwright)\
**Post date:** [February 12, 2019, 8:24pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/1 "2019-02-12T20:24:43Z")

</div>

On my Ubuntu Xenial host curator doesn't work at all under elasticsearch user:  
root@elastic1:~# su - elasticsearch -s /bin/bash  
elasticsearch@elastic1:~ export LC\_ALL=C.UTF-8; export LANG=C.UTF-8 elasticsearch@elastic1:~ curator --config /etc/elasticsearch/curator.yml /etc/elasticsearch/curator/actions/dns-retention.yaml  
elasticsearch@elastic1:~$

But it works under root user by exporting those locales or if I do the next:

root@elastic1:~# sudo -u elasticsearch export LC\_ALL=C.UTF-8; export LANG=C.UTF-8; curator --config /etc/elasticsearch/curator.yml /etc/elasticsearch/curator/actions/dns-retention.yaml

Curator version:  
root@elastic1:~# curator --version  
curator, version 5.6.0

How to make it work under elasticsearch user? What do I do wrong?

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [February 12, 2019, 8:40pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/2 "2019-02-12T20:40:15Z")

</div>

Does the Elasticsearch user have full read privileges to read from `/etc/elasticsearch/curator.yml` and `/etc/elasticsearch/curator/actions/dns-retention.yaml`?

Are there SELinux rules that would prevent that user from making outbound network calls?

---

<div class="post-metadata">

**Author:** ![George\_Wainwright](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/george_wainwright/32/51054_2.png) [@George\_Wainwright](https://discuss.elastic.co/u/George_Wainwright)\
**Post date:** [February 12, 2019, 9:24pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/3 "2019-02-12T21:24:48Z")

</div>

> Does the Elasticsearch user have full read privileges to read from `/etc/elasticsearch/curator.yml` and `/etc/elasticsearch/curator/actions/dns-retention.yaml` ?  
> Yes, it has.

> Are there SELinux rules that would prevent that user from making outbound network calls?

How to check it on Ubuntu?

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [February 12, 2019, 9:40pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/4 "2019-02-12T21:40:40Z")

</div>

Not an Ubuntu wizard, but in Ubuntu I believe they use app-armor instead of SELinux.

What happens if you log in as the Elasticsearch user, instead of using `su`?

---

<div class="post-metadata">

**Author:** ![George\_Wainwright](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/george_wainwright/32/51054_2.png) [@George\_Wainwright](https://discuss.elastic.co/u/George_Wainwright)\
**Post date:** [February 13, 2019, 7:35am UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/5 "2019-02-13T07:35:22Z")

</div>

elasticsearch user doesn't have login permission it is set /bin/false for it in env variables.

Like I said I can execute command via sudo -u elasticsearch export LC\_ALL=C.UTF-8; export LANG=C.UTF-8; curator --config /etc/elasticsearch/curator.yml /etc/elasticsearch/curator/actions/dns-retention.yaml

it works fine. But in fact it gives empty ouptput if I do su - elasticsearch -s /bin/bash or If I do it in crontab by setting those locale variables in cron as well.

Another fact is this is an AWS EC2 instance.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [February 13, 2019, 12:24pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/6 "2019-02-13T12:24:40Z")

</div>

That’s probably because cron expects a shell to be able to fork. You’d have to run it out of the root cron like you’ve been doing.

---

<div class="post-metadata">

**Author:** ![George\_Wainwright](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/george_wainwright/32/51054_2.png) [@George\_Wainwright](https://discuss.elastic.co/u/George_Wainwright)\
**Post date:** [February 13, 2019, 1:03pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/7 "2019-02-13T13:03:38Z")

</div>

Aaron,

the problem not in cron. When I run curator --help manually under elasticsearch user it doesn't give me an output as well. The root of issue lies in something else but not in cron.

---

<div class="post-metadata">

**Author:** ![George\_Wainwright](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/george_wainwright/32/51054_2.png) [@George\_Wainwright](https://discuss.elastic.co/u/George_Wainwright)\
**Post date:** [February 13, 2019, 1:25pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/8 "2019-02-13T13:25:11Z")

</div>

Hold on I was a bit fast with answer. Once I imported locales I am able to see --help output, but command:

curator --config /etc/elasticsearch/curator.yml /etc/elasticsearch/curator/actions/dns-retention.yaml

doesn't give any output.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [February 13, 2019, 1:42pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/9 "2019-02-13T13:42:56Z")

</div>

Try dialing up the `loglevel` to DEBUG and see what, if anything, comes of it.

---

<div class="post-metadata">

**Author:** ![George\_Wainwright](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/george_wainwright/32/51054_2.png) [@George\_Wainwright](https://discuss.elastic.co/u/George_Wainwright)\
**Post date:** [February 13, 2019, 1:58pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/10 "2019-02-13T13:58:43Z")

</div>

It doesn't output anything to logs as well, but looks like I found the root of issue:  
if I add elasticsearch into sudoers file with elasticsearch ALL=(ALL) NOPASSWD:ALL  
It works under that user. So the issue lay in sudo privileges for elasticsearch user.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [February 13, 2019, 2:07pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/11 "2019-02-13T14:07:45Z")

</div>

If you’re going to do that, you might as well run it as root. A user without an executable shell is probably the real culprit here. The command you are running simply executes the root user’s chosen shell as the Elasticsearch user. I will test and verify this at some point today, VPN permitting.

---

<div class="post-metadata">

**Author:** ![George\_Wainwright](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/george_wainwright/32/51054_2.png) [@George\_Wainwright](https://discuss.elastic.co/u/George_Wainwright)\
**Post date:** [February 13, 2019, 2:11pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/12 "2019-02-13T14:11:50Z")

</div>

Well, how to overcome it in order I do not add a user to sudoers and do not execute command with sudo?

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [February 13, 2019, 3:16pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/13 "2019-02-13T15:16:37Z")

</div>

I created 2 users on a test Ubuntu 16.04 VM I have:

```auto
has_login:x:2001:2001::/home/has_login:/bin/bash
nologin:x:2002:2002::/home/nologin:/bin/false

```

This is what I found.

When run as the root user with the user that has a bash shell:

```auto
# sudo -u has_login curator --config curator.yml open-test.yml
2019-02-13 14:54:45,098 INFO Preparing Action ID: 1, "open"
2019-02-13 14:54:45,154 INFO Trying Action ID: 1, "open": open all indices older than 0 days should only use with --dry-run as test for output
2019-02-13 14:54:45,214 INFO Opening selected indices: ['.triggered_watches', '.monitoring-alerts-6', '.monitoring-es-6-2019.02.12', '.tasks', '.monitoring-es-6-2019.02.13', '.kibana_2', 'netflow-000002', '.monitoring-kibana-6-2019.02.13', 'netflow-000001', 'netflow-000003', '.monitoring-kibana-6-2019.02.12', '.watcher-history-9-2019.02.12', '.watches', '.watcher-history-9-2019.02.13', '.kibana_1']
2019-02-13 14:54:45,223 INFO Action ID: 1, "open" completed.
2019-02-13 14:54:45,223 INFO Job completed.
# su - has_login -c 'curator --config curator.yml open-test.yml'
2019-02-13 14:55:38,720 INFO Preparing Action ID: 1, "open"
2019-02-13 14:55:38,776 INFO Trying Action ID: 1, "open": open all indices older than 0 days should only use with --dry-run as test for output
2019-02-13 14:55:38,854 INFO Opening selected indices: ['.kibana_2', 'netflow-000002', 'netflow-000003', '.tasks', '.watches', '.watcher-history-9-2019.02.12', '.triggered_watches', '.monitoring-alerts-6', '.monitoring-kibana-6-2019.02.13', '.monitoring-kibana-6-2019.02.12', 'netflow-000001', '.monitoring-es-6-2019.02.13', '.watcher-history-9-2019.02.13', '.monitoring-es-6-2019.02.12', '.kibana_1']
2019-02-13 14:55:38,861 INFO Action ID: 1, "open" completed.
2019-02-13 14:55:38,861 INFO Job completed.

```

When I run as the user with `/bin/false` as the shell:

```auto
# sudo -u nologin curator --config /bigdisk/buh/curator_packages/curator.yml /bigdisk/buh/curator_packages/open-test.yml
2019-02-13 14:57:00,627 INFO Preparing Action ID: 1, "open"
2019-02-13 14:57:00,683 INFO Trying Action ID: 1, "open": open all indices older than 0 days should only use with --dry-run as test for output
2019-02-13 14:57:00,743 INFO Opening selected indices: ['.kibana_1', '.monitoring-es-6-2019.02.13', '.monitoring-kibana-6-2019.02.13', '.watcher-history-9-2019.02.13', 'netflow-000002', '.monitoring-alerts-6', '.watches', '.triggered_watches', 'netflow-000003', '.watcher-history-9-2019.02.12', '.monitoring-es-6-2019.02.12', '.tasks', '.kibana_2', '.monitoring-kibana-6-2019.02.12', 'netflow-000001']
2019-02-13 14:57:00,751 INFO Action ID: 1, "open" completed.
2019-02-13 14:57:00,752 INFO Job completed.

# su - nologin -c 'curator --config /bigdisk/buh/curator_packages/curator.yml /bigdisk/buh/curator_packages/open-test.yml'
# (NO OUTPUT)

```

In other words, the user must have a shell if you plan on using `su -`. If you allow your sudoers file to have a blanket "Elasticsearch user can do anything", i.e., `elasticsearch ALL=(ALL) NOPASSWD:ALL`, this is a security risk. It's _much_ safer to have a plain user with an executing shell, and just let that user have a crontab entry:

```auto
# su - elasticsearch
$ crontab -l
no crontab for elasticsearch
$ crontab -e
< add 0 2 * * * /usr/bin/curator --config /etc/elasticsearch/curator.yml /etc/elasticsearch/curator/actions/dns-retention.yaml or something like it >
$ crontab -l
0 2 * * * /usr/bin/curator --config /etc/elasticsearch/curator.yml /etc/elasticsearch/curator/actions/dns-retention.yaml

```

---

<div class="post-metadata">

**Author:** ![George\_Wainwright](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/george_wainwright/32/51054_2.png) [@George\_Wainwright](https://discuss.elastic.co/u/George_Wainwright)\
**Post date:** [February 13, 2019, 3:41pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/14 "2019-02-13T15:41:35Z")

</div>

Aaron,

thank you for the investigation, but I actually set shell for that user:  
chsh --shell /bin/bash elasticsearch

but is still didn't work after it.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [February 13, 2019, 4:08pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/15 "2019-02-13T16:08:00Z")

</div>

So, now that the `elasticsearch` user has a shell, does it also have a home directory?

```auto
elasticsearch:x:1000:1000:Elasticsearch User,,,:/home/elasticsearch:/bin/bash

```

What happens when you `su - elasticsearch`?

```auto
# su - elasticsearch

$ curator --help
Usage: curator [OPTIONS] ACTION_FILE

  Curator for Elasticsearch indices.

  See http://elastic.co/guide/en/elasticsearch/client/curator/current

Options:
  --config PATH Path to configuration file. Default: ~/.curator/curator.yml
  --dry-run Do not perform any changes.
  --version Show the version and exit.
  --help Show this message and exit.
$

```

What happens if you switch to the `elasticsearch` user and then execute?

---

<div class="post-metadata">

**Author:** ![George\_Wainwright](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/george_wainwright/32/51054_2.png) [@George\_Wainwright](https://discuss.elastic.co/u/George_Wainwright)\
**Post date:** [February 13, 2019, 4:13pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/16 "2019-02-13T16:13:53Z")

</div>

elasticsearch:x:112:116::/home/elasticsearch:/bin/bash  
elasticsearch@elastic1:~ export LC\_ALL=C.UTF-8; export LANG=C.UTF-8 elasticsearch@elastic1:~ curator --help  
Usage: curator [OPTIONS] ACTION\_FILE

Curator for Elasticsearch indices.

See [http://elastic.co/guide/en/elasticsearch/client/curator/current](http://elastic.co/guide/en/elasticsearch/client/curator/current)

Options:  
--config PATH Path to configuration file. Default: ~/.curator/curator.yml  
--dry-run Do not perform any changes.  
--version Show the version and exit.  
--help Show this message and exit.

elasticsearch@elastic1:~ curator --config /etc/elasticsearch/curator.yml /etc/elasticsearch/curator/actions/dns-retention.yaml elasticsearch@elastic1:~

As you can see the last command didn't give an output.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [February 13, 2019, 4:23pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/17 "2019-02-13T16:23:24Z")

</div>

This suggests that the Elasticsearch user may not have read permissions to one of those files. This can be verified:

```auto
$ cat /etc/elasticsearch/curator.yml
... (output)
$ cat /etc/elasticsearch/curator/actions/dns-retention.yaml
... (output)

```

Barring this, as previously stated, it could be that the user cannot open a socket (also permissions).

---

<div class="post-metadata">

**Author:** ![George\_Wainwright](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/george_wainwright/32/51054_2.png) [@George\_Wainwright](https://discuss.elastic.co/u/George_Wainwright)\
**Post date:** [February 13, 2019, 4:24pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/18 "2019-02-13T16:24:01Z")

</div>

It has permissions, I am able to cat both of those files under elasticsearch user.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [February 13, 2019, 4:25pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/19 "2019-02-13T16:25:15Z")

</div>

Okay. Does this execute fully when executed as root? If so, it implies that root has permission to do something that the Elasticsearch user does not.

---

<div class="post-metadata">

**Author:** ![George\_Wainwright](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/george_wainwright/32/51054_2.png) [@George\_Wainwright](https://discuss.elastic.co/u/George_Wainwright)\
**Post date:** [February 13, 2019, 4:26pm UTC](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112/20 "2019-02-13T16:26:41Z")

</div>

> Does this execute fully when executed as root?  
> I removed elasticsearch from sudoers so it is not being executed as root now.

Being added to sudoers command is being executed properly.

[Next page](https://discuss.elastic.co/t/curator-returns-empty-output-for-elasticsearch-user/168112.md?page=2)