Need to define environment variable to calculate the previous date

[vsingh (kumar)]

Hello there,

I need to import the prior date data to create the index in elastic from mysql.
this is script:

input {
jdbc {
jdbc_connection_string => "jdbc:mysql://localhost:3306/testdb?zeroDateTimeBehavior=convertToNull"
jdbc_user => "root"
jdbc_password => "mysql"
jdbc_driver_library => "C:\Users\mysql-connector-java-8.0.13.jar"
jdbc_driver_class => "com.mysql.cj.jdbc.Driver"
Pre_DATE=date --date="-1 day" '+%Y.%m.%d'
statement => "select transaction_id from testdb.sample where process_date==Pre_DATE "
type => "response"
}
}
output {
if [type] == "response" {
elasticsearch {
index => "index_one"
document_type => "type"
document_id => "%{transaction_id}"
hosts => "localhost:9200"
}
stdout {
codec => rubydebug
}
}
}

this is log:

[2019-03-07T11:30:52,708][INFO ][logstash.runner ] Starting Logstash {"
logstash.version"=>"6.4.0"}
[2019-03-07T11:30:53,223][ERROR][logstash.agent ] Failed to execute ac
tion {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"L
ogStash::ConfigurationError", :message=>"Expected one of #, => at line 8, column
11 (byte 352) after input {\n jdbc {\n jdbc_connection_string => "jd
bc:mysql://localhost:3306/testdb?zeroDateTimeBehavior=convertToNull"\n j
dbc_user => "root"\n jdbc_password => "mysql"\n jdbc_driver_li
brary => "C:\Users\mysql-connector-java-8.0.13.j
ar"\n jdbc_driver_class => "com.mysql.cj.jdbc.Driver"\n\t\tPre_DATE",
:backtrace=>["C:/ELK/logstash/logstash-core/lib/logstash/compiler.rb:41:in comp ile_imperative'", "C:/ELK/logstash/logstash-core/lib/logstash/compiler.rb:49:incompile_graph'", "C:/ELK/logstash/logstash-core/lib/logstash/compiler.rb:11:in
block in compile_sources'", "org/jruby/RubyArray.java:2486:inmap'", "C:/ELK/l
ogstash/logstash-core/lib/logstash/compiler.rb:10:in compile_sources'", "org/lo gstash/execution/AbstractPipelineExt.java:157:ininitialize'", "C:/ELK/logstash
/logstash-core/lib/logstash/pipeline.rb:22:in initialize'", "C:/ELK/logstash/lo gstash-core/lib/logstash/pipeline.rb:90:ininitialize'", "C:/ELK/logstash/logst
ash-core/lib/logstash/pipeline_action/create.rb:38:in execute'", "C:/ELK/logsta sh/logstash-core/lib/logstash/agent.rb:309:inblock in converge_state'"]}
[2019-03-07T11:30:53,536][INFO ][logstash.agent ] Successfully started
Logstash API endpoint {:port=>9600}

please suggest where to define the environment variable.

Thanks.

[Badger]

@vsingh

Pre_DATE= `date --date="-1 day" '+%Y.%m.%d'`

You would have to run that in the shell that invokes logstash, then refer to it using ${Pre_DATE} in the statement option.

[vsingh (kumar)]

Thanks for reply.

It is working if i call it through shell and refer it in logstash.
i want to call logstash using cron to automate the process then where Pre_DATE will be define.

kindly suggest.
Thanks.

[Badger]

Have cron call a shell script that sets it and then calls logstash.

[vsingh (kumar)]

Can i define Pre_DATE environment variable instead of the shell so that i can refer it in logstash configuration file.

Thanks.