Indexes based on different names


I use the combination of rsyslog and elasticsearch which seems to be pretty

My requirement is something in the lines of this:

  1. I will have as many as 16 processes running parallely and each process
    has 23 threads sending syslog information. (using syslog( ) system call)
    The message itself will look something like this "ip=x.x.x.x,
    name=abcd, loglevel=3, this is a test log"
  2. My current architecture for logs is file based where I separate each
    process log to a different file.
  3. I tried to log the above message with standalone mysql /sqlite (use SQL
    insert statements) where I had a single database and multiple tables for
    each process, however the performance was poor. ( I had high cpu usage on a
    4 core x86_64 server having 32gb memory)
  4. I start all processes one by one they run for a few hours and stop them
    all at once and collect/examine the logs.
  5. I wont need to store the logs permanently, so I can go ahead and delete
    them once the analysis is over.

Is it possible in rsyslog to

  1. create 1 index and multiple types for each of the process through
    rsyslog (or outside of it first using curl) based on some name that
    identifies the process
  2. Query indexes separately for values (e.g. 1 process may have logs from
    ip addresses to and other process may have logs from to etc)

So my index should look something like this curl -XGET

"ip" : "x.x.x.x"
"name": "abcd"
"log": "test log"

"ip" : "y.y.y.y"
"name": "wxyz"
"log": "test log"

thanks a lot in advance

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
For more options, visit