How to check that my log file data is transfered to elasticsearch 5.5.0

vivekpandey564 · July 27, 2017, 1:08pm

input {
    file {
        add_field => [ "host", "my-dev-host" ]
        path => "D:\JHipster_Demo\logFile.2017-07-27.log"
        codec => "plain"
    }
}

filter {
  date {
    match => [ "timestamp" , "YYYY/MM/DD:HH:mm:ss Z" ]
  }
}

output {
    elasticsearch {
        index => "global2-%{+YYYY.MM.dd}"
        hosts => [ "localhost:9200" ]
    }
    stdout { codec => rubydebug }
}

josephjohney · July 27, 2017, 1:09pm

What is the issue you are facing ?/

vivekpandey564 · July 27, 2017, 1:12pm

Thanks For Reply am just curious to know how i should check that my log data is coming towards elasticsearch.

dadoonet · July 27, 2017, 1:21pm

Please format your code using </> icon as explained in this guide. It will make your post more readable.

Or use markdown style like:

```
CODE
```

I edited your post.

dadoonet · July 27, 2017, 1:23pm

Run something like:

GET global2-*/_search

vivekpandey564 · July 27, 2017, 1:24pm

Thanks for replying.
where should i run this

dadoonet · July 27, 2017, 1:25pm

In Kibana dev console. Or run as CURL like

curl localhost:9200/global2-*/_search?pretty

vivekpandey564 · July 27, 2017, 1:28pm

Thanks !! At least i can see something like:
{
"took" : 346,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 10238,
"max_score" : 1.0,
"hits" : [
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yK",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=HikariPool-1.pool.IdleConnections, value=10\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yO",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Address.average-put-time, value=0.0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yQ",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Address.cache-evictions, value=0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yT",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Address.cache-hits, value=0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yc",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Authority.cache-gets, value=0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yi",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Authority.cache-removals, value=0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yj",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Borrower.addresses.average-get-time, value=0.0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yk",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Borrower.addresses.average-put-time, value=0.0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yv",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Borrower.average-put-time, value=0.0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yy",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.341Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Borrower.cache-gets, value=0\r"
}
}
]
}
}

vivekpandey564 · July 27, 2017, 1:29pm

Do i need to do any setting to see the log in kibana?
I have installed cersion of 5.5.0 on my windows machine.
Home page is getting loaded but how can isee this in kibana.

dadoonet · July 27, 2017, 1:45pm

Please format your code using </> icon as explained in this guide. It will make your post more readable.

Or use markdown style like:

```
CODE
```

dadoonet · July 27, 2017, 1:46pm

It's a question for #kibana group now.
Anyway, you basically need to configure an index pattern first then you should see that in discover panel.

If you are just starting with Kibana, you might want to read: https://www.elastic.co/guide/en/kibana/current/index.html

vivekpandey564 · July 27, 2017, 3:23pm

Thanks you so much!!

system · August 24, 2017, 3:24pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
_grokparsefailure inspite of working well with grok debugger Logstash	10	1766	May 4, 2019
Change @timestamp to time from actual log file Logstash	6	1724	October 9, 2017
Parsing Syslog to Logstash Logstash	6	475	March 10, 2020
Parsing the date and sending to ElasticSearch Logstash	13	7841	May 26, 2017
Error management in elasticsearch output plugin Logstash docker	5	218	February 13, 2024

How to check that my log file data is transfered to elasticsearch 5.5.0

Related Topics