Wrong mapping, need to change it \ lost in how to and documentation

Elastic Stack Elasticsearch
6

I totally understand. Below is an example which I tried. By default, elasticsearch will index everything as "string" unless specified. Try this, before going in deep with analyzers etc.

Step-1 Logstash
input {
file {
path => "/home/ubuntu/Logstash_input/access1.log" Path of the input file
type => "apache"  set sourcetype
start_position => beginning  followtail=0

}
}

filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" } Built-in type for apache
#pattern => ["%{COMBINEDAPACHELOG}"]
}
date
{
locale => "en"
match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z", "ISO8601"] Identifying timestamp from data, else current time is set
timezone => "Asia/Kolkata"
target => "@timestamp"
}
geoip { Setting geo coordinates
source => "[clientip]"  Source field
target => "geoip"  Target field name
}
}
output {
elasticsearch { Output to ES
hosts => ["172.30.0.73:9200"]
index => "apache_access" Index name
}
stdout { codec => rubydebug } Also print on sysout
}

Step-2 Elasticsearch Mapping

curl -XPOST 'http://localhost:9200/apache_access -d '{

"settings" : {
    "number_of_shards" : 1
},
"mappings" : {
    "apache" : {
        
        "properties" : {

"bytes": {"type": "long"},
"response":{ "type":"long"},
"clientip":{ "type": "ip"}
}
}
}
}’

Sample Log
182.236.164.11 - - [10/Apr/2015:18:20:50 +0530] "GET /category.screen?categoryId=STRATEGY&JSESSIONID=SD6SL8FF10ADFF53101 HTTP 1.1" 200 1200 "http://www.google.com" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.46 Safari/536.5" 490
182.236.164.11 - - [10/Apr/2015:18:20:52 +0530] "GET /product.screen?productId=MB-AG-G07&JSESSIONID=SD6SL8FF10ADFF53101 HTTP 1.1" 200 1035 "http://www.buttercupgames.com/category.screen?categoryId=ARCADE" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.46 Safari/536.5" 461
182.236.164.11 - - [10/Apr/2015:18:20:53 +0530] "POST /cart.do?action=addtocart&itemId=EST-6&productId=MB-AG-G07&JSESSIONID=SD6SL8FF10ADFF53101 HTTP 1.1" 200 533 "http://www.buttercupgames.com/product.screen?productId=MB-AG-G07" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.46 Safari/536.5" 470

I would suggest - Try a simple example first, then it is easy to do for other complex grok patterns.

Hope this helps!