Query ElasticSearchData USING Ruby ElasticsearchGem

[prakash1243 (Mike)]

Hi -
I have a setup with Elastic Search, Kibana and MetricBeat running with 5.1.1 version on my Windows-7 machine.
Am trying to create a test case to check CPU rate at the API level; using Ruby elasticsearch-5.0.0 gem.

I have installed required gem on my machine and trying to connect to my cluster, which I couldn't proceed up.
Am getting the unauthorized error, may be because, my set-up is installed with x-pack.
Could you please let me know how do I connect to the Elastic Search cluster and query the CPU usage from API level (i.e. ealsticsearch.gem).

Thanks in advance.

Prakash.

[danielmitterdorfer]

Hi @prakash1243,

you need to pass the correct credentials when creating the Ruby client as stated in the README of elasticsearch-transport.

If you did not change the defaults, the user is elastic and the password changeme.

Daniel

[prakash1243 (Mike)]

Hi @danielmitterdorfer

Thanks so much for your reply:

I tried with this statement: Elasticsearch::Client.new url: 'http://elastic:changeme@localhost:9200'
I've attached the screenshot of the response:

I believe, I've got connected to the Elastic Search. Could you please let me know how do I query for the Metric Beat CPU usage ? Thanks in advance.

[danielmitterdorfer]

Hi @prakash1243,

this doesn't look like a (HTTP) response to me but rather a debugger view of the client object.

You could verify it by simply retrieving the version (my Ruby is a bit rusty ;)):

puts client.info()["version"]["number"]

You should see 5.1.1. in your terminal.

@prakash1243

Could you please let me know how do I query for the Metric Beat CPU usage ? Thanks in advance.

I'm not familiar with the document schema. I think you should ask this question in a Beats forum rather than the Elasticsearch forum. Can you please open a new topic there?

Daniel

[prakash1243 (Mike)]

Hi @danielmitterdorfer:
Yes, You are absolutely right, I've got 5.1.1 as version on my IRB console.
So, I believe, I've got connected to the cluster. I'd like to know, how do we query for the cpu usage of a node.
Thank you very much for your help.

Regards,
Prakash

[prakash1243 (Mike)]

Hi @andrewkroh-

Could you please help getting the Metric beats data using Elastic Search Gem with Ruby.

Thanks in advance !!

[ruflin]

Not sure I can follow: How is Metricbeat and Ruby client connected? Do you have a specific issue with querying the data?

[prakash1243 (Mike)]

Hi @ruflin

My Scenario is: I've to check the CPU Usage of a windows server, to make sure continue the Test cases execution
If it goes beyond the threshold, I'd abort the Test case execution.

I would be able to check CPU only via API (Not with Kibana). So I tried using Ruby gem (elasticsearch-5.1.1), As I mentioned in my previous posts, Am able to connect to the cluster and able to get the version.

I'd need a help in querying the data to get the CPU Usage. Please let me know if am unclear. Thanks !

Regards,
Prakash

[ruflin]

Ok, this sounds more like you have problems with the ruby client then metricbeat itself I understand it correctly. Did you check the docs for the ruby client? https://www.elastic.co/guide/en/elasticsearch/client/ruby-api/current/_the_ruby_client.html

There it also shows you how you can do some simple queries. What did you try so far that didn't work?

[prakash1243 (Mike)]

Hi @ruflin-
I have this query so far (GET metricbeat-2017.01.09/metricsets/_search?q=cpu) .
I'd like to know how to run the GET queries with the Ruby Client Gem. Could you please help me out. Thanks !

Prakash.

[ruflin]

Please check the docs. Here you see some example queries (https://www.elastic.co/guide/en/elasticsearch/client/ruby-api/current/_the_ruby_client.html#_example_usage) but there are much more details in the client api docs http://www.rubydoc.info/gems/elasticsearch

[prakash1243 (Mike)]

Thanks for your reponse @ruflin.

Am having hard time to execute the below query with the examples I had gone thru, Could you please help with executing the below query using Ruby Client

GET metricbeat-2017.01.20/metricsets/_search
{
"size": 0,
"query": {
"range": {
"@timestamp": {
"gte": "19/01/2017",
"lte": "23/01/2017",
"format": "dd/MM/yyyy||yyyy"
}
}
},
"aggs": {
"NAME": {
"avg": {
"field": "system.fsstat.total_size.total"
}
}
}
}

Thanks in advance.

[ruflin]

Can you share what you tried so far based on the examples and didn't work?

[prakash1243 (Mike)]

Hi @ruflin:

Here is my query:

res = client.search index: 'metricbeat-2017.01.23',
body: {
size: 0,
query: {
match:
#{ range: {
#date: { gte: "{{1484896203312}}", lte: "{{1484897124046}}" } },
{ "beat.hostname": "AAEINBLR03199L" }
},
aggregations: { tags: { terms: { field: 'system.fsstat.total_size.used' } } }
}
end

I wanted to put the range, to select the data between time stamps, which I couldn't able to do that. so I had commented it out for now.

Would you please let me know, how do I put the range inbetween and execute this query.

When I use the search_template method, I get the below error message:

Thanks !

[prakash1243 (Mike)]

Hi @ruflin-

I've to do the multiple filters and aggregate the final results.

Filters is like, host name and @timestamp and get the aggregated disk info of a node. Please let me know. Thanks !

Regards,
Prakash

[ruflin]

For the range, this one here should help: https://github.com/elastic/elasticsearch-ruby/blob/master/elasticsearch-dsl/lib/elasticsearch/dsl/search/queries/range.rb#L8

For the error: You must paste it here in text not screenshot, otherwise it is very hard to read.

I'm not really an expert on Ruby or the es client, but I strongly recommend you to either check some examples online or check the source code which has good docs inside.