Timestamp on ES using Mongo River


(karnamonkster) #1

Hi,

Here are my version details:

ES 1.0.0, MongoDB 2.4.9, mongo-river 2.0.0(with the driver that is required
for the river plugin)
The cluster health is green, I have ES 2 nodes on the same machine to make sure the shards are indexed properly.
I have a river with mongo which fetches events from the remote mongodb.

Now i have everything up an running but the only issue is with the "time
format" which is still in UNIX epoch for:

1.event_time
2.timestamp
3.app_event_time

here is the mapping.

curl -XGET http://ESserverIP:9200/secana/_mapping?pretty=true
{
"secana" : {
"mappings" : {
"messages" : {
"properties" : {
"app_event_time" : {
"type" : "long"
},
"event_time" : {
"type" : "long"
},
"frecency" : {
"type" : "long"
},
"hostname_id" : {
"type" : "string"
},
"message" : {
"type" : "string"
},
"netloc" : {
"type" : "string"
},
"signum_id" : {
"type" : "string"
},
"source" : {
"type" : "string"
},
"source2" : {
"type" : "string"
},
"title" : {
"type" : "string"
},
"url" : {
"type" : "string"
},
"visit_count" : {
"type" : "long"
}
}
}
}
}
}


Now i am still not able to get the correct date format as mentioned in my mapping.

By default it is picking up " long".

I have tried to update the mapping properties using

PUT /_river/secana/messages/_mapping

{"secana":{"mappings":{"messages":{"properties":{"_timestamp":{"enabled":"true","store":"yes"},"app_event_time":{"type":"date","format":"YYYY-MM-dd HH:mm:ss"},"event_time":{"type":"date","format":"YYYY-MM-dd HH:mm:ss"},"firstName":{"type":"string"},"frecency":{"type":"long"},"hostname_id":{"type":"string"},"lastName":{"type":"string"},"message":{"type":"string"},"netloc":{"type":"string"},"signum_id":{"type":"string"},"source":{"type":"string"},"source2":{"type":"string"},"timestamp":{"type":"date","format":"YYYY-MM-dd HH:mm:ss"},"title":{"type":"string"},"url":{"type":"string"},"visit_count":{"type":"long"}}}}}}

But this is not updated.
Using the head plugin, I could see that the index metadata is still giving me the incorrect details for date fields.
1.event_time
2.timestamp
3.app_event_time
So i could sense this might be the issue.
I have tried several to DELETE the index and _river many times with the correct mapping and index fields.

{

state: open 
settings: { 
    index: { 
        uuid: 6UBnCIgFRt-77OexGcIdnQ 
        number_of_replicas: 1 
        number_of_shards: 5 
        version: { 
            created: 1000099 
        } 
    } 
} 
mappings: { 
    messages: { 
        properties: { 
            visit_count: { 
                type: long 
            } 
            message: { 
                type: string 
            } 
            title: { 
                type: string 
            } 
            signum_id: { 
                type: string 
            } 
            source: { 
                type: string 
            } 
            app_event_time: { 
                type: long 
            } 
            hostname_id: { 
                type: string 
            } 
            event_time: { 
                type: long 
            } 
            netloc: { 
                type: string 
            } 
            source2: { 
                type: string 
            } 
            frecency: { 
                type: long 
            } 
            url: { 
                type: string 
            } 
        } 
    } 
} 
aliases: [ ] 

}


(system) #2