While using enviroment variable in Curator -crontab.Unable to get required output

Elastic Stack Elasticsearch
1

i set enviroment variable for elasticsearch-service in curator's config yml. and some env var in action_file.yml

i export variable in bash_profile file and other solution found in web but it does not work

there are two scenerio
1.when i run manually i.e.
curator --config /etc/config/config.yml /etc/config/action_file.yml >> /var/log/curator-(date +\%d\%b\%y_\%H\%M\%S).log then it works as expected 2.when i run with crontab i.e. 0 * * * * curator --config /etc/config/config.yml /etc/config/action_file.yml >> /var/log/curator-(date +%d%b%y_%H%M%S).log
error in logs

ERROR     Schema error: Configuration: filter: Location: Action ID "1", action "delete_indices", filter #1: {'filtertype': 'age', 'timestring': '%Y.%m.%d', 'direction': 'older'}: Bad Value: "(could not determine)", required key not provided @ data['source']. Check configuration file.

how to solve this issue of running crontab with using enviroment var in curator

2

This is typically because PATH and other environment variables that you normally have access to from the command-line are not automatically passed through to cron.

My personal recommendation is to make your one-liner into a simple shell script, and then have cron call that. Tweak it until all of the paths etc. are fully functional.

3

I did what you told me but the problem is still coming up.
i made a demo.sh file

#!/bin/bash
curator --config /etc/config/config.yml /etc/config/action_file.yml >> /var/log/curator-(date +%d%b%y_%H%M%S).log

in crontab file

* * * * * sh /demo.sh

in config.yml

config.yml: |
    # Remember, leave a key empty if there is no value.  None will be a string,
    # not a Python "NoneType"
    client:
        hosts:
        - ${ES_SVC}

means that crontab run the demo.sh file but unable to fetch the value of enviroment variable

4

Yes, which means you need to find a way to source those environment variables in your script. I'm sorry I wasn't more clear on that front.

cron cannot make use of arbitrary environment variables. They have to be declared in order to pass them to some script, or the script must source them.

Ways of doing this include a second file which has something like:

ENVVARNAME=VALUE; export ENVVARNAME

and then in your script, you would use

source /path/to/other_filename

and then you could use ${ENVVARNAME} in your script.

5

@theuntergeek
thanks for your solution
one more issue is there
if i make a enviroment variable for action file.

my action.yml

1:
        action: delete_indices
        description: >-
          Delete indices older than (based on index name), for
          prefixed indices. Ignore the error if the filter does not result in an
          actionable list of indices (ignore_empty_list) and exit cleanly.
        options:
          ignore_empty_list: True
          disable_action: False
        filters:
          - filtertype: pattern
            kind: regex
            value: '^(testing-).*$'
          - filtertype: age
            timestring: '%Y.%m.%d'
            source: name
            direction: older
            unit: days
            #unit_count: 4
            unit_count: ${RETENTION_PERIOD_FOR_TESTING}

my script where enviroment variable exported is - env-var.sh

#!/bin/bash
ES_SVC=${ES_SVC}; export ES_SVC
RETENTION_PERIOD_FOR_TESTING= ${RETENTION_PERIOD_FOR_TESTING}; export RETENTION_PERIOD_FOR_TESTING

with this configuration if i launch application following error throw by curator

2020-05-01 23:57:01,865 ERROR     Schema error: required key not provided @ data['unit_count']
2020-05-01 23:57:01,865 ERROR     Schema error: Configuration: filter: Location: Action ID "2", action "delete_indices", filter #2: {'filtertype': 'age', 'timestring': '%Y.%m.%d', 'source': 'name', 'direction': 'older', 'unit': 'days'}: Bad Value: "(could not determine)", required key not provided @ data['unit_count']. Check configuration file.

how to solve this please help

6

There appears to be a space after the =, which is not acceptable in shell scripting. Could it be that?

7

@theuntergeek
thanks for your reply
as you told me to do

RETENTION_PERIOD_FOR_TESTING=${RETENTION_PERIOD_FOR_TESTING}; export RETENTION_PERIOD_FOR_TESTING

but no success

8

So, that means that there is no value being passed—which, at closer look, is kind of not surprising:

ES_SVC=${ES_SVC}; export ES_SVC

How exactly can you pass the value of ES_SVC to itself? or RETENTION_PERIOD_FOR_TESTING to itself? You ought to be passing values to the environment variables. Here, you're passing the value of variables—which may or may not be set properly—to identically named variables. This would make it nearly impossible to cleanly troubleshoot.

At the very least, you should change the variable names to not be identical.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.