How to check that my log file data is transfered to elasticsearch 5.5.0


(Vivek Pandey) #1
input {
    file {
        add_field => [ "host", "my-dev-host" ]
        path => "D:\JHipster_Demo\logFile.2017-07-27.log"
        codec => "plain"
    }
}

filter {
  date {
    match => [ "timestamp" , "YYYY/MM/DD:HH:mm:ss Z" ]
  }
}

output {
    elasticsearch {
        index => "global2-%{+YYYY.MM.dd}"
        hosts => [ "localhost:9200" ]
    }
    stdout { codec => rubydebug }
}

J hipster enable and disable elastic search
(Joseph Johney) #2

What is the issue you are facing ?/


(Vivek Pandey) #3

Thanks For Reply am just curious to know how i should check that my log data is coming towards elasticsearch.


(David Pilato) #4

Please format your code using </> icon as explained in this guide. It will make your post more readable.

Or use markdown style like:

```
CODE
```

I edited your post.


(David Pilato) #5

Run something like:

GET global2-*/_search

(Vivek Pandey) #6

Thanks for replying.
where should i run this


(David Pilato) #7

In Kibana dev console. Or run as CURL like

curl localhost:9200/global2-*/_search?pretty

(Vivek Pandey) #8

Thanks !! At least i can see something like:
{
"took" : 346,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 10238,
"max_score" : 1.0,
"hits" : [
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yK",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=HikariPool-1.pool.IdleConnections, value=10\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yO",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Address.average-put-time, value=0.0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yQ",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Address.cache-evictions, value=0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yT",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Address.cache-hits, value=0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yc",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Authority.cache-gets, value=0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yi",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Authority.cache-removals, value=0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yj",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Borrower.addresses.average-get-time, value=0.0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yk",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Borrower.addresses.average-put-time, value=0.0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yv",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Borrower.average-put-time, value=0.0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yy",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.341Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Borrower.cache-gets, value=0\r"
}
}
]
}
}


(Vivek Pandey) #9

Do i need to do any setting to see the log in kibana?
I have installed cersion of 5.5.0 on my windows machine.
Home page is getting loaded but how can isee this in kibana.


(David Pilato) #10

Please format your code using </> icon as explained in this guide. It will make your post more readable.

Or use markdown style like:

```
CODE
```

(David Pilato) #11

It's a question for #kibana group now.
Anyway, you basically need to configure an index pattern first then you should see that in discover panel.

If you are just starting with Kibana, you might want to read: https://www.elastic.co/guide/en/kibana/current/index.html


(Vivek Pandey) #12

Thanks you so much!!


(system) #13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.