Organizing data for Elasticsearch and Kibana

I have inserted some data into elastic search using REST interface.An
example is shown below.

curl -XPOST "http://xxx.xx.xx.xx:9200/bits/metrics/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/aaaaaaa/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/bbbbbbb/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/ccccccc/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/ddddddd/" -d @$file

I basically insert 5 object types which are

metrics
aaaaaaa
bbbbbbb
ccccccc
ddddddd

All these object types are inserted into the same index bits. The data is
inserted at regular intervals of say 30 mins, Meaning I add a set of JSON
document every 30 mins to each of the above five types. However only on of
my object type's have timestamp as a part of JSON data. I am using
elasticsearch 1.4 and Kibana 3. I can't seem to sepcify the index for the
Kibana dashboard. It does expect a timestamp value.

Because of this I suspect I am limited in the kinds of graphs that I can
plot.I would like to plot dual axis graphs and some time series data.Is
this correct assumption?

Is the index that Kibana is looking for the same as the index in
elasticsearch. In other words Should I rather use the timestamp value (of
when I load the data into elasticsearch) as the elasticsearch rather than
some constant value like bits used in the above example.

As my data does not have the timestamp inherently, What format should my
timestamp (that I am supposed to use as the index have, If I should)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/84a65ccc-8428-4287-b9e8-49687bd3b199%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

KB reads data from Elasticsearch, so yeah an index is the same thing for
both.

Basically you either need a timestamp in your docs to use KB3, or move to
KB4.

On 18 March 2015 at 19:10, Karthik Sharma karthik.sharma@gmail.com wrote:

I have inserted some data into Elasticsearch using REST interface.An
example is shown below.

curl -XPOST "http://xxx.xx.xx.xx:9200/bits/metrics/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/aaaaaaa/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/bbbbbbb/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/ccccccc/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/ddddddd/" -d @$file

I basically insert 5 object types which are

metrics
aaaaaaa
bbbbbbb
ccccccc
ddddddd

All these object types are inserted into the same index bits. The data
is inserted at regular intervals of say 30 mins, Meaning I add a set of
JSON document every 30 mins to each of the above five types. However only
on of my object type's have timestamp as a part of JSON data. I am using
elasticsearch 1.4 and Kibana 3. I can't seem to sepcify the index for the
Kibana dashboard. It does expect a timestamp value.

Because of this I suspect I am limited in the kinds of graphs that I can
plot.I would like to plot dual axis graphs and some time series data.Is
this correct assumption?

Is the index that Kibana is looking for the same as the index in
elasticsearch. In other words Should I rather use the timestamp value (of
when I load the data into elasticsearch) as the elasticsearch rather than
some constant value like bits used in the above example.

As my data does not have the timestamp inherently, What format should my
timestamp (that I am supposed to use as the index have, If I should)

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/84a65ccc-8428-4287-b9e8-49687bd3b199%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/84a65ccc-8428-4287-b9e8-49687bd3b199%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_Dy6rV3Hs10HmreQJ7gpbPBF8kc7gqBzvXdDzCi0NdKA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

How will moving to KB4 help in this case? Will it unlock the option of
plotting dual axis graphs?

Regards
Karthik.

On Friday, 20 March 2015 05:52:46 UTC+13, Mark Walkom wrote:

KB reads data from Elasticsearch, so yeah an index is the same thing for
both.

Basically you either need a timestamp in your docs to use KB3, or move to
KB4.

On 18 March 2015 at 19:10, Karthik Sharma <karthik...@gmail.com
<javascript:>> wrote:

I have inserted some data into Elasticsearch using REST interface.An
example is shown below.

curl -XPOST "http://xxx.xx.xx.xx:9200/bits/metrics/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/aaaaaaa/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/bbbbbbb/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/ccccccc/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/ddddddd/" -d @$file

I basically insert 5 object types which are

metrics
aaaaaaa
bbbbbbb
ccccccc
ddddddd

All these object types are inserted into the same index bits. The data
is inserted at regular intervals of say 30 mins, Meaning I add a set of
JSON document every 30 mins to each of the above five types. However only
on of my object type's have timestamp as a part of JSON data. I am using
elasticsearch 1.4 and Kibana 3. I can't seem to sepcify the index for the
Kibana dashboard. It does expect a timestamp value.

Because of this I suspect I am limited in the kinds of graphs that I can
plot.I would like to plot dual axis graphs and some time series data.Is
this correct assumption?

Is the index that Kibana is looking for the same as the index in
elasticsearch. In other words Should I rather use the timestamp value (of
when I load the data into elasticsearch) as the elasticsearch rather than
some constant value like bits used in the above example.

As my data does not have the timestamp inherently, What format should my
timestamp (that I am supposed to use as the index have, If I should)

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/84a65ccc-8428-4287-b9e8-49687bd3b199%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/84a65ccc-8428-4287-b9e8-49687bd3b199%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e63b58fa-ce44-41fc-94b7-415c477a841b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

KB4 doesn't specifically require time series data.

On 21 March 2015 at 16:25, Karthik Sharma karthik.sharma@gmail.com wrote:

How will moving to KB4 help in this case? Will it unlock the option of
plotting dual axis graphs?

Regards
Karthik.

On Friday, 20 March 2015 05:52:46 UTC+13, Mark Walkom wrote:

KB reads data from Elasticsearch, so yeah an index is the same thing for
both.

Basically you either need a timestamp in your docs to use KB3, or move to
KB4.

On 18 March 2015 at 19:10, Karthik Sharma karthik...@gmail.com wrote:

I have inserted some data into Elasticsearch using REST interface.An
example is shown below.

curl -XPOST "http://xxx.xx.xx.xx:9200/bits/metrics/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/aaaaaaa/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/bbbbbbb/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/ccccccc/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/ddddddd/" -d @$file

I basically insert 5 object types which are

metrics
aaaaaaa
bbbbbbb
ccccccc
ddddddd

All these object types are inserted into the same index bits. The data
is inserted at regular intervals of say 30 mins, Meaning I add a set of
JSON document every 30 mins to each of the above five types. However only
on of my object type's have timestamp as a part of JSON data. I am using
elasticsearch 1.4 and Kibana 3. I can't seem to sepcify the index for the
Kibana dashboard. It does expect a timestamp value.

Because of this I suspect I am limited in the kinds of graphs that I can
plot.I would like to plot dual axis graphs and some time series data.Is
this correct assumption?

Is the index that Kibana is looking for the same as the index in
elasticsearch. In other words Should I rather use the timestamp value (of
when I load the data into elasticsearch) as the elasticsearch rather than
some constant value like bits used in the above example.

As my data does not have the timestamp inherently, What format should my
timestamp (that I am supposed to use as the index have, If I should)

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/84a65ccc-8428-4287-b9e8-49687bd3b199%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/84a65ccc-8428-4287-b9e8-49687bd3b199%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e63b58fa-ce44-41fc-94b7-415c477a841b%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/e63b58fa-ce44-41fc-94b7-415c477a841b%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9hg%2Bjky%3DFvA0WSk9YdMnJoWYJcfMFYHQDqX91C4%2BrctQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

When you mention timestamp in my docs, Do you mean that I actually replace
the Elasticsearch 'index' with timestamp. If yes is there a python library
perhaps that does that for me?

Regards,
Karthik.

On Friday, 20 March 2015 05:52:46 UTC+13, Mark Walkom wrote:

KB reads data from Elasticsearch, so yeah an index is the same thing for
both.

Basically you either need a timestamp in your docs to use KB3, or move to
KB4.

On 18 March 2015 at 19:10, Karthik Sharma <karthik...@gmail.com
<javascript:>> wrote:

I have inserted some data into Elasticsearch using REST interface.An
example is shown below.

curl -XPOST "http://xxx.xx.xx.xx:9200/bits/metrics/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/aaaaaaa/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/bbbbbbb/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/ccccccc/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/ddddddd/" -d @$file

I basically insert 5 object types which are

metrics
aaaaaaa
bbbbbbb
ccccccc
ddddddd

All these object types are inserted into the same index bits. The data
is inserted at regular intervals of say 30 mins, Meaning I add a set of
JSON document every 30 mins to each of the above five types. However only
on of my object type's have timestamp as a part of JSON data. I am using
elasticsearch 1.4 and Kibana 3. I can't seem to sepcify the index for the
Kibana dashboard. It does expect a timestamp value.

Because of this I suspect I am limited in the kinds of graphs that I can
plot.I would like to plot dual axis graphs and some time series data.Is
this correct assumption?

Is the index that Kibana is looking for the same as the index in
elasticsearch. In other words Should I rather use the timestamp value (of
when I load the data into elasticsearch) as the elasticsearch rather than
some constant value like bits used in the above example.

As my data does not have the timestamp inherently, What format should my
timestamp (that I am supposed to use as the index have, If I should)

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/84a65ccc-8428-4287-b9e8-49687bd3b199%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/84a65ccc-8428-4287-b9e8-49687bd3b199%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/de65e13e-9939-43e7-9317-ea8d3fe254a3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

I mean you need a field in your document containing a timestamp.

On 26 March 2015 at 07:15, Karthik Sharma karthik.sharma@gmail.com wrote:

When you mention timestamp in my docs, Do you mean that I actually replace
the Elasticsearch 'index' with timestamp. If yes is there a python library
perhaps that does that for me?

Regards,
Karthik.

On Friday, 20 March 2015 05:52:46 UTC+13, Mark Walkom wrote:

KB reads data from Elasticsearch, so yeah an index is the same thing for
both.

Basically you either need a timestamp in your docs to use KB3, or move to
KB4.

On 18 March 2015 at 19:10, Karthik Sharma karthik...@gmail.com wrote:

I have inserted some data into Elasticsearch using REST interface.An
example is shown below.

curl -XPOST "http://xxx.xx.xx.xx:9200/bits/metrics/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/aaaaaaa/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/bbbbbbb/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/ccccccc/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/ddddddd/" -d @$file

I basically insert 5 object types which are

metrics
aaaaaaa
bbbbbbb
ccccccc
ddddddd

All these object types are inserted into the same index bits. The data
is inserted at regular intervals of say 30 mins, Meaning I add a set of
JSON document every 30 mins to each of the above five types. However only
on of my object type's have timestamp as a part of JSON data. I am using
elasticsearch 1.4 and Kibana 3. I can't seem to sepcify the index for the
Kibana dashboard. It does expect a timestamp value.

Because of this I suspect I am limited in the kinds of graphs that I can
plot.I would like to plot dual axis graphs and some time series data.Is
this correct assumption?

Is the index that Kibana is looking for the same as the index in
elasticsearch. In other words Should I rather use the timestamp value (of
when I load the data into elasticsearch) as the elasticsearch rather than
some constant value like bits used in the above example.

As my data does not have the timestamp inherently, What format should my
timestamp (that I am supposed to use as the index have, If I should)

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/84a65ccc-8428-4287-b9e8-49687bd3b199%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/84a65ccc-8428-4287-b9e8-49687bd3b199%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/de65e13e-9939-43e7-9317-ea8d3fe254a3%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/de65e13e-9939-43e7-9317-ea8d3fe254a3%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X81PrqOxSA9-88WbBCNDMx0iTmeW9xUqnhrgLchbTKYDQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

I use the following script to enable timestamps for all of my document
types inside my id 'bits'. when I visualize it using Kibana3, I can't seem
to find timestamp field come up there. Am I doing somethign wrong? Do I
need to do something more?

curl -XPOST localhost:9200/bits -d '{
"mappings" : {
"default":{
"_timestamp" : {
"enabled" : true,
"store" : true
}
}
}
}'

On Thursday, 26 March 2015 09:49:19 UTC+13, Mark Walkom wrote:

I mean you need a field in your document containing a timestamp.

On 26 March 2015 at 07:15, Karthik Sharma <karthik...@gmail.com
<javascript:>> wrote:

When you mention timestamp in my docs, Do you mean that I actually
replace the Elasticsearch 'index' with timestamp. If yes is there a python
library perhaps that does that for me?

Regards,
Karthik.

On Friday, 20 March 2015 05:52:46 UTC+13, Mark Walkom wrote:

KB reads data from Elasticsearch, so yeah an index is the same thing for
both.

Basically you either need a timestamp in your docs to use KB3, or move
to KB4.

On 18 March 2015 at 19:10, Karthik Sharma karthik...@gmail.com wrote:

I have inserted some data into Elasticsearch using REST interface.An
example is shown below.

curl -XPOST "http://xxx.xx.xx.xx:9200/bits/metrics/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/aaaaaaa/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/bbbbbbb/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/ccccccc/" -d @$file
curl -XPOST "http://xxx.xx.xx.xx:9200/bits/ddddddd/" -d @$file

I basically insert 5 object types which are

metrics
aaaaaaa
bbbbbbb
ccccccc
ddddddd

All these object types are inserted into the same index bits. The
data is inserted at regular intervals of say 30 mins, Meaning I add a set
of JSON document every 30 mins to each of the above five types. However
only on of my object type's have timestamp as a part of JSON data. I am
using elasticsearch 1.4 and Kibana 3. I can't seem to sepcify the index
for the Kibana dashboard. It does expect a timestamp value.

Because of this I suspect I am limited in the kinds of graphs that I
can plot.I would like to plot dual axis graphs and some time series data.Is
this correct assumption?

Is the index that Kibana is looking for the same as the index in
elasticsearch. In other words Should I rather use the timestamp value (of
when I load the data into elasticsearch) as the elasticsearch rather than
some constant value like bits used in the above example.

As my data does not have the timestamp inherently, What format should
my timestamp (that I am supposed to use as the index have, If I should)

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/84a65ccc-8428-4287-b9e8-49687bd3b199%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/84a65ccc-8428-4287-b9e8-49687bd3b199%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/de65e13e-9939-43e7-9317-ea8d3fe254a3%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/de65e13e-9939-43e7-9317-ea8d3fe254a3%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c53ba4ce-d6e7-4356-a2cb-83e538d19799%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.