How to check that my log file data is transfered to elasticsearch 5.5.0

vivekpandey564 · July 27, 2017, 1:08pm

input {
    file {
        add_field => [ "host", "my-dev-host" ]
        path => "D:\JHipster_Demo\logFile.2017-07-27.log"
        codec => "plain"
    }
}

filter {
  date {
    match => [ "timestamp" , "YYYY/MM/DD:HH:mm:ss Z" ]
  }
}

output {
    elasticsearch {
        index => "global2-%{+YYYY.MM.dd}"
        hosts => [ "localhost:9200" ]
    }
    stdout { codec => rubydebug }
}

josephjohney · July 27, 2017, 1:09pm

What is the issue you are facing ?/

vivekpandey564 · July 27, 2017, 1:12pm

Thanks For Reply am just curious to know how i should check that my log data is coming towards elasticsearch.

dadoonet · July 27, 2017, 1:21pm

Please format your code using </> icon as explained in this guide. It will make your post more readable.

Or use markdown style like:

```
CODE
```

I edited your post.

dadoonet · July 27, 2017, 1:23pm

Run something like:

GET global2-*/_search

vivekpandey564 · July 27, 2017, 1:24pm

Thanks for replying.
where should i run this

dadoonet · July 27, 2017, 1:25pm

In Kibana dev console. Or run as CURL like

curl localhost:9200/global2-*/_search?pretty

vivekpandey564 · July 27, 2017, 1:28pm

Thanks !! At least i can see something like:
{
"took" : 346,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 10238,
"max_score" : 1.0,
"hits" : [
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yK",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=HikariPool-1.pool.IdleConnections, value=10\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yO",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Address.average-put-time, value=0.0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yQ",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Address.cache-evictions, value=0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yT",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Address.cache-hits, value=0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yc",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Authority.cache-gets, value=0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yi",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Authority.cache-removals, value=0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yj",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Borrower.addresses.average-get-time, value=0.0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yk",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Borrower.addresses.average-put-time, value=0.0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yv",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.326Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Borrower.average-put-time, value=0.0\r"
}
},
{
"_index" : "global2-2017.07.27",
"_type" : "logs",
"_id" : "AV2EEITcJkknhZQBM5yy",
"_score" : 1.0,
"_source" : {
"path" : "D:\JHipster_Demo\logFile.2017-07-27.log",
"@timestamp" : "2017-07-27T12:41:01.341Z",
"@version" : "1",
"host" : [
"FS-WS195-D45",
"my-dev-host"
],
"message" : " 2017-07-27 18:11:01,061 INFO [metrics-logger-reporter-1-thread-1] metrics: type=GAUGE, name=jcache.statistics.com.hexaware.rad.domain.Borrower.cache-gets, value=0\r"
}
}
]
}
}

vivekpandey564 · July 27, 2017, 1:29pm

Do i need to do any setting to see the log in kibana?
I have installed cersion of 5.5.0 on my windows machine.
Home page is getting loaded but how can isee this in kibana.

dadoonet · July 27, 2017, 1:45pm

Please format your code using </> icon as explained in this guide. It will make your post more readable.

Or use markdown style like:

```
CODE
```

dadoonet · July 27, 2017, 1:46pm

It's a question for #kibana group now.
Anyway, you basically need to configure an index pattern first then you should see that in discover panel.

If you are just starting with Kibana, you might want to read: https://www.elastic.co/guide/en/kibana/current/index.html

vivekpandey564 · July 27, 2017, 3:23pm

Thanks you so much!!

system · August 24, 2017, 3:24pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
How to check if logstash is sending the logs to Elasticsearch Logstash	2	5905	December 24, 2019
Beginner question: Verifying that a file has successfully been entered in ES Logstash	6	1573	July 6, 2017
Is my Data being received by Elasticsearch? Elasticsearch	18	25689	June 2, 2017
General Help Logstash	3	243	May 21, 2020
I want to output log to elasticsearch index from only specific "host.ip" Logstash	5	1400	April 6, 2021

How to check that my log file data is transfered to elasticsearch 5.5.0

Related topics