Using a default mapping template {:es_version=>7, :ecs_compatibility=>:v8} gives me this error,
[2024-01-23T02:24:07,381][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch version determined (7.13.3) {:es_version=>7}
[2024-01-23T02:24:07,382][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
[2024-01-23T02:24:07,401][INFO ][logstash.outputs.elasticsearch][main] Not eligible for data streams because config contains one or more settings that are not compatible with data streams: {"index"=>"modsecurity"}
[2024-01-23T02:24:07,401][INFO ][logstash.outputs.elasticsearch][main] Data streams auto configuration (`data_stream => auto` or unset) resolved to `false`
[2024-01-23T02:24:07,408][WARN ][logstash.outputs.elasticsearch][main] `template_api => auto` resolved to `legacy` since we are connected to Elasticsearch 7, but will resolve to `composable` the first time it connects to Elasticsearch 8+. We recommend either setting `template_api => legacy` to continue providing legacy-style templates, or migrating your template to the composable style and setting `template_api => composable`. The legacy template API is slated for removal in Elasticsearch 9.
[2024-01-23T02:24:07,409][INFO ][logstash.outputs.elasticsearch][main] Using a default mapping template {:es_version=>7, :ecs_compatibility=>:v8}
[2024-01-23T02:24:07,415][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>12, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1500, "pipeline.sources"=>["/tmp/logstash.conf"], :thread=>"#<Thread:0x2e33326d /usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:134 run>"}
[2024-01-23T02:24:07,433][INFO ][logstash.outputs.elasticsearch][main] Installing Elasticsearch template {:name=>"ecs-logstash"}
[2024-01-23T02:24:07,467][ERROR][logstash.outputs.elasticsearch][main] Failed to install template {:message=>"Got response code '400' contacting Elasticsearch at URL 'http://x.x.x.x:9200/_template/ecs-logstash'",
What is ecs-logstash template ? Do I need to create it in Elasticsearch ?
Stack trace said no handler for type [match_only_text] declared on field [text]
"/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-output-elasticsearch-11.22.2-java/lib/logstash/plugin_mixins/elasticsearch/common.rb:172:in `block in after_successful_connection'"],
:body=>"{\"error\":{\"root_cause\":[{\"type\":\"mapper_parsing_exception\",\"reason\":\"no handler for type [match_only_text] declared on field [text]\"}],\"type\":\"mapper_parsing_exception\",\"reason\":\"Failed to parse mapping [_doc]: no handler for type [match_only_text] declared on field [text]\",\"caused_by\":{\"type\":\"mapper_parsing_exception\",\"reason\":\"no handler for type [match_only_text] declared on field [text]\"}},\"status\":400}"}
logstash.conf
input{
}
filter {
}
output {
file {
codec => json
path => "c:/temp/logstash_out.log"
}
elasticsearch {
hosts => ["x.x,x.x:9200"]
index => "modsecurity"
user => "elastic"
password => "**********"
}
}
c:/temp/logstash_out.log works perectly fine.
Note, no such index exists in Elasticsearch. And this is modsecurity json logs which has nested mapping. I don't know how to create a valid index for that. I only know to create a simple mapping from api/devtools
PUT /test_index1?pretty
{
"settings" : {
"number_of_shards" : 2,
"number_of_replicas" : 1
},
"mappings" : {
"properties" : {
"updated_at" : { "type" : "date" }
}
}
}
