Hello!
Since today I`m not enable to create a new index. I've deleted my old index so I could start over. But ever since then I'm not enable to get it working again.
I already got 3 indexes working (They are from some weeks back).
My logstash code:
input {
file {
path => "C:\Users\JOEY2\Desktop\Deelproblemen\Applicatie\Output\OuputURLS.csv"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
csv {
separator => ","
columns => [URL,timestamp,ScanID]
}
date {
match => ["timestamp", "YYYY-MM-dd;HH:mm:ss"]
target => "@timestamp"
}
}
output {
elasticsearch {
action => "index"
hosts => "http://localhost:9200"
index => "komop"
document_type => "csv"
}
}
When I run logstash I get the following output:
formatted text :\ELK-Stack\logstash\bin>logstash -f C:\ELK-Stack\logstash\biL-results.conf Sending Logstash's logs to C:/ELK-Stack/logstash/logs which is now configured via log4j2.properties [2017-11-03T10:59:29,222][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"C:/ELK-Stack/logstash/modules/fb_apache/configuration"} [2017-11-03T10:59:29,240][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"C:/ELK-Stack/logstash/modules/netflow/configuration"} [2017-11-03T10:59:30,111][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}} [2017-11-03T10:59:30,126][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"} [2017-11-03T10:59:30,245][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"} [2017-11-03T10:59:30,339][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil} [2017-11-03T10:59:30,354][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}} [2017-11-03T10:59:30,386][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["http://localhost:9200"]} [2017-11-03T10:59:30,407][INFO ][logstash.pipeline ] Starting pipeline {"id"=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>250} [2017-11-03T10:59:30,860][INFO ][logstash.pipeline ] Pipeline main started [2017-11-03T10:59:30,960][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
When I curl to my indices I get the following output:
C:\Users\JOEY2\Downloads\curl-7.56.0-win64-ming\curl-7.56.0-win64-mingw\bin>curl localhost:9200/_cat/indices?v health status index uuid pri rep docs.count docs.deleted store.size pri.store.size yellow open .watcher-history-6-2017.10.31 UoHu2fLWSrChT1rBpyrP5w 1 1 7200 0 5.7mb 5.7mb yellow open .monitoring-es-6-2017.11.02 xcwC48IVS_Kkbq8nhHKuqA 1 1 53894 650 35mb 35mb yellow open .monitoring-alerts-6 5TnyoQb8Q6yeHx2zAVsQYg 1 1 1 0 6.3kb 6.3kb yellow open .kibana 1L9k_KBnTSmUganIpsNGDQ 1 1 22 3 85.5kb 85.5kb yellow open .monitoring-es-6-2017.10.25 TXbxMVLmQveU_fyYTB_AIw 1 1 13763 156 7.5mb 7.5mb yellow open .monitoring-es-6-2017.10.30 DKiKpUF9Q2GVLZ0lDXToiQ 1 1 237914 2064 169.3mb 169.3mb yellow open .watcher-history-6-2017.10.25 lcy_PuDDTnq-umE_bOACzQ 1 1 290 0 468.9kb 468.9kb yellow open .monitoring-es-6-2017.11.01 Vqnz1Gm-TNe-02ojPNVqzQ 1 1 275386 3672 193.3mb 193.3mb yellow open .watcher-history-6-2017.10.27 bzyv3J41QeiuKEkCYTNJ4A 1 1 2320 0 1.9mb 1.9mb yellow open .watcher-history-6-2017.10.26 pxV97zz1T4C98Lr-VMd60g 1 1 2300 0 1.9mb 1.9mb yellow open .watcher-history-6-2017.11.01 ekmwzeZoRv6f7h5y8tNP5A 1 1 4600 0 3.8mb 3.8mb yellow open .monitoring-es-6-2017.10.26 PDnG-qeLRxGakSXNuJCOKw 1 1 82211 492000 131.8mb 131.8mb yellow open attachment 8vCBdNWvRVCxGG865w17lg 5 1 66 0 290.4kb 290.4kb yellow open headersattachmentsurls Z0chmsrlR4yAP9ui0KR_Iw 5 1 65 0 581.7kb 581.7kb yellow open .watcher-history-6-2017.10.30 X2unznAmTW61fwL81td7Lw 1 1 4905 0 3.9mb 3.9mb yellow open csv KqtxG9cNQYiTU33HRCw38g 5 1 1319 0 2.8mb 2.8mb yellow open .watches dVCPJ-03SMKzfqDr9l4ngg 1 1 4 0 19.8kb 19.8kb yellow open .triggered_watches iSfYF9F2QGyGFpppvq0xcw 1 1 0 0 192b 192b yellow open .monitoring-es-6-2017.10.27 22N9R0sORVqhyCsjcjcOyg 1 1 97918 582856 159.2mb 159.2mb yellow open .watcher-history-6-2017.11.02 vWAivfnxSZuUJ7sc25ACwA 1 1 769 0 836.1kb 836.1kb yellow open .monitoring-es-6-2017.10.31 VuO0vIRIRX6AzMHt7RjZBA 1 1 385983 3384 276.7mb 276.7mb yellow open winlogbeat-2017.11.03 rUlyislfTpWOg-KFiJqhTg 5 1 3 0 51.3kb 51.3kb
What am I missing? I just can't figure out whats going wrong for me.
Thanks for your time.