Use case and explication for newbie

Hi all,

I'm french, excuse my poor english. :wink:

in few words this is my case and my configuration :

  • nxlog for eventslog and sending them to rsyslog in json.
  • syslog on linux and aix servers
  • rsyslog to receive and elasticsearch for indexation/save.

Elasticsearch :

  • 1 node
  • index.number_of_shards: 1
  • index.number_of_replicas: 0

With nxlog, we've sent a whole windows eventslog to elasticsearch near
6:30pm. Memory and CPU use grow up during the logs transfert (near
10minutes) then CPU and memory down to normal. Few minutes later (~30mn),
CPU/RAM occupation grew up again and stay to 100% on one CPU from 7pm to
9am this morning until i restart elasticsearch service.
This morning kibana won't respond and elasticsearch directory show indexes
from years 2011, 2012 and 2013. I think i'ts because of old messages in the
windows eventlog.

After restarting elasticsearch, everythings came to normal but i've lost
all logs message between yesterday 7pm and today 9am.

https://lh5.googleusercontent.com/-LkJcQF5fk5k/VFIaHXfBAUI/AAAAAAAALMQ/nx7vLKKEx7M/s1600/Capture.PNG

This is what i think, can someone say me if i'm wrong or right ?

The server took in 10minutes near 500MB of logs with only 1 node and 1
shard. The indexation start and took all the night. I stopped it too soon.
All the logs in cue were lost when i've restart the service.

Have you some tweaks or tips to optimize my confirguration ?

Regards

Bastien

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/96297107-01ea-445a-a205-03fa03a24cdd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.