I have created an example I hope someone can assist with as previously versions of this worked, this is a cut down version with only a few fields as I cannot figure out what is different from this compared to past examples! Below are the steps I performed:
I created the mappings as follows, ran and no errors:
curl-XPUT http://localhost:9200/analyzeme-d '
{"mappings": { "default": {"date_detection":false,"properties": {"mytext": {"type": "string", "index" :"not_analyzed" },"mydate": {"type": "date","index" :"not_analyzed" },"mynumber": { "type" : "integer" }}}}}';
Checked the settings, no errors:
curl -XGET 'http://localhost:9200/analyzeme/_settings'
Created a GROK script, no errors (at least from what I can see, location: /etc/logstash/conf.d/tc_analyze.conf) - previous versions work fine
input
{file{
type=> "testcase"
path=> '/usr/local/analyzeme/*.log'
sincedb_path=> "/usr/local/analyzeme/sincedb"
start_position=> "beginning"
}
}
filter
{
grok
{
patterns_dir=> "/usr/local/sof-elk/grok-patterns"
match=> {"message"=> "%{ANALYZEMELOG}"}}
output
{
elasticsearch{
index=> "analyzeme"
}
}
Imported some data, no errors:
curl-XPOST 'localhost:9200/analyzeme/testcase/_bulk?pretty'--data-binary @testcase.json
contents
of the “testcase.json” file:
{"index":{"_id":"2"}}
{"mytext":"sometext","mydate":"2009-11-15T14:12:12","mynumber":0}
Started logstash using command, no apparent errors
/opt/logstash/bin/logstash -f /etc/logstash/conf.d/tc_analyze.conf -vv –-debug
I then ran, which showed one document:
curl 'localhost:9200/_cat/indices?v'
To confirm logstash is working I created the file /usr/local/analyzeme/test.log and typed a record, saved and re-ran:
curl 'localhost:9200/_cat/indices?v'
The test showed a new document.
When I goto elasticsearch no data appears
Almost forgot: the grok pattern, which works in previous examples
ANALYZEMELOG "%{GREEDYDATA:sslurl}",%{TIMESTAMP_ISO8601:sslscandate},%{NUMBER:ssldaysremaining}
The problem doesnt seem to be with logstash but with elasticsearch, thanks for any help!
