[http-module] Collecting Data from Webserver: Problem with JSON-Array

[PeterPain]

Hey there, so i want to collect live Data from a webservice. The webservice is for compressor-monitoring and offers an interface in JSON-Format.

Demo:
http://www.airleader.biz/demo/index.jsp
Interface:
http://www.airleader.biz/airleader/interfaces/json.jsp

I am able to get the data in Elasticsearch, but the JSON-Format from the webservice puts the status (on, off, error, etc.) in an array for each compressor under modules.

-

So i want to check for example if Compressor 1,2,3... is on error. But elasticsearch isnt able to query correctly through arrays

I gave i quick look to nested datatype, but i dont know how to use it in my case.

thx in advance for any help

[ruflin]

I assume when you mention nested queries that is what you were looking at ? https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-nested-query.html

Can you share the queries that you tried and didn't work?

[PeterPain]

Hey Ruflin,

I think i need to explain my problem more detailed.
The JSON-Interface of the website (see link above) puts the Informations of each compressor under http.airleader.mm.modules in an array of objects.

"M" = compressor is on
"E" = compressor is on error

I am not able to modify this interface.
So i dont have the Nested Datatypes and Elasticsearch isn't able to query through an array of objects.

Example:

Because its not a nested datatype i get wrong results for the query.

To solve the problem i was looking at the option of pre-processing of documents using ingest-node. But i dont know how to handle arrays of object in painless