Some help for a newbie

Elastic Stack Elasticsearch
1

Hi everyone !

I'm posting today to ask some help with ElasticSearch.

I would like use ELK for treat some data from a JSON file but I don't know how organize my ElasticSearch Structure.

Below the structure of my JSON datafile:

"dateoftheday":{
"countrieName":[
{
"serverName":"hisName",
"status":"hisStatus",
"users":[
{
"name":"hisName",
"computerName":"hisComputerName",
"idNumber":"hisIdNumber"
},
{
"name":"hisName",
"computerName":"hisComputerName",
"idNumber":"hisIdNumber"
},
{
"name":"hisName",
"computerName":"hisComputerName",
"idNumber":"hisIdNumber"
},
]
},

              {
                          "serverName":"hisName",
                          "status":"hisStatus",
                          "users":[
                                    {
                                              "name":"hisName",
                                              "computerName":"hisComputerName",
                                              "idNumber":"hisIdNumber"
                                    },
                                   {
                                              "name":"hisName",
                                              "computerName":"hisComputerName",
                                              "idNumber":"hisIdNumber"
                                    },
                                   {
                                              "name":"hisName",
                                              "computerName":"hisComputerName",
                                              "idNumber":"hisIdNumber"
                                    },
                          ]
              },
     ]

}

I don't detail more my file but i've several countries and several server for each countries and a lot of users for each server. (my file have about 3000 lines)

Moreover, il will receive a new 3000 lines file every 3 hours and I have to make statistic on these.

I would like make kibana chart on the number of users per countries, a general chart with the evolution of the number of users in terms of time with différents line representing different countries
and so one

But I don't know how organize my datafiles in elasticSearch to these make Kibana chart.

Can someone help me to define my structure ElasticSearch, I did lots of manipulation with ES and Kibana but i can't generate chart like I want.

I think my ES structure is wrong.

Thanks in advance for all eventual help

Regards,

Math

2

Any way you can create one json document per user per server?

Is this the whole structure? How consistent is it? You might be able to do something in Logstash if you can't ship individual documents per user...

Or you could have e.g. a python (or other language you know, I use python) script break up the 3000 line document into per user documents. That should be a pretty simple script.

I'm sure there's many other way to do it as well :slight_smile:

3

Hi,

I decided to try this out. It's good practise :slight_smile:

Output would be something like

{"Date": "08-11-2017", "Location": "GER", "Server": "Server1", "Status": "Running", "Name": "User001", "Computer": "Computer001", "Id": "555001"}
{"Date": "08-11-2017", "Location": "GER", "Server": "Server1", "Status": "Running", "Name": "User002", "Computer": "Computer002", "Id": "555002"}
{"Date": "08-11-2017", "Location": "GER", "Server": "Server1", "Status": "Running", "Name": "User003", "Computer": "Computer003", "Id": "555003"}
{"Date": "08-11-2017", "Location": "GER", "Server": "Server2", "Status": "Idle", "Name": "User101", "Computer": "Computer101", "Id": "555101"}
{"Date": "08-11-2017", "Location": "GER", "Server": "Server2", "Status": "Idle", "Name": "User102", "Computer": "Computer102", "Id": "555102"}
{"Date": "08-11-2017", "Location": "GER", "Server": "Server2", "Status": "Idle", "Name": "User103", "Computer": "Comuter103", "Id": "555103"}

Which filebeat could ship to Elasticsearch.

4

Hi A_B,
first of all thanks for your help and your attention !

I tried this out too this morning and i can do more thinhgs with my data !

So my new data structure is per users like you can see below:

{"date":"2017-10-04T08:15:03.079+XXXX","Location":"Angola","serverName":"XXXXXXX02-UUUUU2", "projectName":"XXBUU_HELLO_WORLD20XX","ID":"00K73760","computerName":"fhgzzid-globjzed.oo.local"}

Now i can make chart about users per Location like i would like to do !

I still have a question, i dont know how to do this kind of chart
image

i would like have date on the X-axis, the number of users on the y-axis and lines which represent my differents countries.

Is it possible with Kibana ?

5

It's ok ! I fixed my issue, i found how to make my multi linear chart !

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.