Elastic search not updating


#1

I have created an example I hope someone can assist with as previously versions of this worked, this is a cut down version with only a few fields as I cannot figure out what is different from this compared to past examples! Below are the steps I performed:

I created the mappings as follows, ran and no errors:

curl-XPUT http://localhost:9200/analyzeme-d '
{"mappings": { "default": {"date_detection":false,"properties": {"mytext": {"type": "string", "index" :"not_analyzed" },"mydate": {"type": "date","index" :"not_analyzed" },"mynumber": { "type" : "integer" }}}}}';

Checked the settings, no errors:
curl -XGET 'http://localhost:9200/analyzeme/_settings'

Created a GROK script, no errors (at least from what I can see, location: /etc/logstash/conf.d/tc_analyze.conf) - previous versions work fine

input
{file{
type=> "testcase"
path=> '/usr/local/analyzeme/*.log'
sincedb_path=> "/usr/local/analyzeme/sincedb"
start_position=> "beginning"
}
}

filter
{
grok
{
patterns_dir=> "/usr/local/sof-elk/grok-patterns"
match=> {"message"=> "%{ANALYZEMELOG}"}}
output
{
elasticsearch{
index=> "analyzeme"
}
}
Imported some data, no errors:
curl-XPOST 'localhost:9200/analyzeme/testcase/_bulk?pretty'--data-binary @testcase.json

contents
of the “testcase.json” file:

{"index":{"_id":"2"}}
{"mytext":"sometext","mydate":"2009-11-15T14:12:12","mynumber":0}

Started logstash using command, no apparent errors

/opt/logstash/bin/logstash -f /etc/logstash/conf.d/tc_analyze.conf -vv –-debug

I then ran, which showed one document:

curl 'localhost:9200/_cat/indices?v'

To confirm logstash is working I created the file /usr/local/analyzeme/test.log and typed a record, saved and re-ran:

curl 'localhost:9200/_cat/indices?v'

The test showed a new document.

When I goto elasticsearch no data appears

Almost forgot: the grok pattern, which works in previous examples

ANALYZEMELOG "%{GREEDYDATA:sslurl}",%{TIMESTAMP_ISO8601:sslscandate},%{NUMBER:ssldaysremaining}

The problem doesnt seem to be with logstash but with elasticsearch, thanks for any help!


(Mark Walkom) #2

What does the output of _cat/indices?v show?


#3

Thanks for the reply, this is the output:


(Mark Walkom) #4

Please don't post pictures of text, they are difficult to read and some people may not be even able to see them :slight_smile:


(system) #5