Representing URL data in kibana dashboard


(Sourav Choudhary) #1

Hi,
I am completely new to Kibana ,elasticsearch etc.
I followed a tutorial and set up kibana , elastic search logtash etc.

My requirement is as following :
1.I have some json data which loads when i open a url. I want to add a dashboard and set it up in such a way that when i click on the dashboard it should call the URL, get the values and display it on dashboard.
2. If i am able to change the state of monitor to" red" , "green" or "up" , "down" it will be an add on if possible.

Hoping to get some answer here as i have the setup but i dont know which specific file i need to modify to get this done as i dont have any experience with these dashboards


(Joe Fleming) #2

when i click on the dashboard it should call the URL, get the values and display it on dashboard

Unfortunately, that's not how these tools work, and there's really no way to build that in Kibana. Whatever data you are using the generate the JSON at that URL, you'll have to index into Elasticsearch first, and then you can use Kibana to aggregate and visualize it.

You could index the JSON from the URL directly if you wanted, assuming it had all the information you needed, and just index that.

I'm assuming that the JSON is providing you with a snapshot of the state of something. As in, the JSON might contain load information, uptimes, system statuses at the moment, and things like that. Am I right about that?


(Sourav Choudhary) #3

Yes Joe I found i need to put data in elasticsearch but which file??,is it the yml i need to edit??
ALso yes JSON has the data but is there any way i can set a kind of threshold on basis of which i can change color etc ??


(Sourav Choudhary) #4

Also one more thing :
I tried following command :
curl -POST http://localhost:9200/etc/elasticsearch -d @abc.txt
Where etc/elasticsearch is the path where elasticsearch is installed and abc.txt is the file, but my problems are :

  1. This command gives error of "Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes"
  2. I want the data to be fetched from a URL so that i have live data so is there a way where i can add the url instead of static file as the url returns me JSON data only.

(Joe Fleming) #5

Depending on the version of the stack you're on, you probably also need to include the content type header in that curl command for it to work. I think that became a requirement in 5.0, or maybe an early point release. Try this:

curl -H "Content-Type:application/json" -POST http://localhost:9200/etc/elasticsearch -d @abc.txt

I want the data to be fetched from a URL so that i have live data so is there a way where i can add the url instead of static file as the url returns me JSON data only

I don't think you can do this with curl directly, but you can almost certainly do it by piping the curl output to another curl call that will index the data. Unfortunately I'm not super skilled in the art of command piping, but it would look something like this meta example:

curl -H "Content-Type:application/json" -XGET <path_to_json_url> | curl -H "Content-Type:application/json" -XPUT <path_to_elasticsearch>/<index_name>

You could also write a script in pretty much any language that would do this for you, using one of the existing Elasticsearch client libraries, or just making REST calls manually. Once you're there, just put that on a cron job and you're pretty much all set. You can get fancier with it, and probably should for a real production use case, but that would be good enough to start out.

is there any way i can set a kind of threshold on basis of which i can change color etc

Once the raw document data is in Elasticsearch, you can perform aggregations on that data and do thing like split on fields and roll up metrics into averages and the like. The visualization builder UI in Kibana is modeled after the Elasticsearch Query DSL, so it's probably good to have at least a basic understanding of how it functions (you don't really need to understand the syntax to use Kibana, just how it works).

Once you are indexing documents, you can use, for example, a range aggregation in your visualization to show data within various thresholds.


(Sourav Choudhary) #6

Thanks Joe most of the things worked here but now the problem is the data on basis of which i am supposed to set the color/threshold etc is in message i.e the json i received, i tried but not able to understand how to pick a value from my message and set the color on the basis of that, Here is a sample json which appears in my message also :

{"uptime":146000.582,"cpu":{"time":"2017-10-05 05:19:24","process":0,"system":0.998453}


(system) #7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.