I am running Logstash and one of my logs has various properties.
I have successfully applied this template:
curl -XPUT http://localhost:9200/_template/logstash_per_index -d '{
"template": "logstash*",
"settings": {
"index.query.default_field": "@message",
"index.cache.field.type": "soft",
"index.store.compress.stored": true
},
"mappings": {
"default": {
"_all": { "enabled": false },
"properties": {
"@message": { "type": "string", "index": "analyzed" },
"@source": { "type": "string", "index": "not_analyzed" },
"@source_host": { "type": "string", "index": "not_analyzed" },
"@source_path": { "type": "string", "index": "not_analyzed" },
"@tags": { "type": "string", "index": "not_analyzed" },
"@timestamp": { "type": "string", "index": "not_analyzed" },
"@type": { "type": "string", "index": "not_analyzed" }
}
}
}
}
'
However, one of the logs being sent to ES via Logstash includes field
properties. This is the grok filter from the Logstash conf:
pattern => "%{IP:vipIP} %{IP:queryingServerIP} - -
[%{HTTPDATE:timestamp}] "%{WORD:httpmethod} %{URIPATHPARAM:request}
%{WORD:httpprotocol}/%{NUMBER:httpversion:float}"
%{NUMBER:httpresponsecode:int} %{NUMBER:bytes:int}
%{NUMBER:responsetime:float}"
add_field => [ "teamname", "Team1" ]
add_field => [ "sourcetype", "access_log" ]
This translates into these mappings on the ES server:
"metadata" : {
"templates" : { },
"indices" : {
"logstash-2013.10.18" : {
"state" : "open",
"settings" : {
"index.number_of_shards" : "3",
"index.number_of_replicas" : "2",
"index.version.created" : "900599"
},
"mappings" : {
"access_log" : {
"properties" : {
"@fields" : {
"properties" : {
"vipIP" : {
"type" : "string"
},
"timestamp" : {
"type" : "string"
},
"httpversion" : {
"type" : "double"
},
"responsetime" : {
"type" : "double"
},
"bytes" : {
"type" : "long"
},
"teamname" : {
"type" : "string"
},
"request" : {
"type" : "string"
},
"queryingServerIP" : {
"type" : "string"
},
"httpmethod" : {
"type" : "string"
},
"httpprotocol" : {
"type" : "string"
},
"httpresponsecode" : {
"type" : "long"
},
"sourcetype" : {
"type" : "string"
}
}
},
"@timestamp" : {
"format" : "dateOptionalTime",
"type" : "date"
},
"@message" : {
"type" : "string"
},
"@source" : {
"type" : "string"
},
"@type" : {
"type" : "string"
},
"@tags" : {
"type" : "string"
},
"@source_host" : {
"type" : "string"
},
"@source_path" : {
"type" : "string"
}
}
},
[snip]
Do I need to handle the field properties in the template? Or will the
generic one I showed at the beginning of the post be adequate? If I do
need to do something with them, what do I need to add?
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.