Sql query input to output csv plugin

Elastic Stack Elasticsearch
1

Hi,
I am generating csv files using logstash output csv plugin.I wanted to know if I can write SQL query as input to output csv plugin to generate csv files?I want to query the ES index and fetch the results and store in csv files using logstash.Is it possible?

Thanks,
Prashanth.S

2

You can may be use the jdbc input plugin with an elasticsearch jdbc driver (which requires a commercial license).
Or use the elasticsearch input plugin in which case you can't use sql but the query dsl.

I'd prefer the last option.

My 2 cents.

3

Hi...Thanks for the response.I am currently using query DSL.But I want to query on top of the index using SQL like query.I will look for getting JDBC client driver but how do I supply the SQL query?Can you please share an example of how my conf file should look like?

This is my current conf file

input {
elasticsearch {
hosts => "localhost:9200"
index => "condition"
query => '{
"query": {
"bool": {
"filter": [
{ "term": { "RankValue": "1" }},
{ "terms": { "ProbCode": ["I63.011", "I60.8"]}},
{ "term": { "ProbCodeSystem": "2.16.840.1.113883.6.90" }}

  ]
}

}
}
'
}
}
output {
csv {
fields => ["RecordId","Code", "codesystemuri","RankValue",""]
path => "/tmp/csv-export.csv"
}
}

What changes Do I make in the conf file to have the SQL query on top of the index?

4

Have a look at the jdbc input plugin then.

5

Hi David,
Yes I will.Thanks again for the response.Can you please let me know an example for what conf file chmages that I need to make to execute the SQL query on top of the ES index

6

Did you read the documentation? What did you try so far? What did not work?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.