So, I am a bit (ok more than a bit) of a newbie. We are using ELK for
collecting storing logs, and recently we started seeing these errors:
[FIELDDATA] New used memory 2570680025 [2.3gb] from field [@timestamp]
would be larger than configured breaker
I am using (more or less) the standard logstash template:
So, I am a bit (ok more than a bit) of a newbie. We are using ELK for
collecting storing logs, and recently we started seeing these errors:
[FIELDDATA] New used memory 2570680025 [2.3gb] from field [@timestamp]
would be larger than configured breaker
I am using (more or less) the standard logstash template:
Yea, that's where I started with it. But, if I understand it, that looks
like how I can change the mapping for a specific property. But I would
think I need to make a similar change to my index template otherwise new
indexes that get created will no long have that mapping. Or am I
misunderstanding?
On Wednesday, April 15, 2015 at 7:20:22 PM UTC-4, Mark Walkom wrote:
Yes that is correct, you have to update your mappings and wait for new
indices to be created from it, it's not something that can be applied
retroactively without reindexing.
Yea, that's where I started with it. But, if I understand it, that looks
like how I can change the mapping for a specific property. But I would
think I need to make a similar change to my index template otherwise new
indexes that get created will no long have that mapping. Or am I
misunderstanding?
On Wednesday, April 15, 2015 at 7:20:22 PM UTC-4, Mark Walkom wrote:
So, please help me with my template I gave above. I am familiar with up to
update it, I am just not real sure on how to change it so that a specific
field uses Doc View. Or if it is easier to make it the default for all
fields I suppose that's fine too since it sounds like that will happen
eventually.
Just need help with what the structure of the template should be... Thanks!
On Wednesday, April 15, 2015 at 8:29:28 PM UTC-4, Mark Walkom wrote:
Yes that is correct, you have to update your mappings and wait for new
indices to be created from it, it's not something that can be applied
retroactively without reindexing.
On 16 April 2015 at 09:55, Scott Chapman <scotted...@gmail.com
<javascript:>> wrote:
Yea, that's where I started with it. But, if I understand it, that looks
like how I can change the mapping for a specific property. But I would
think I need to make a similar change to my index template otherwise new
indexes that get created will no long have that mapping. Or am I
misunderstanding?
On Wednesday, April 15, 2015 at 7:20:22 PM UTC-4, Mark Walkom wrote:
So, please help me with my template I gave above. I am familiar with up to
update it, I am just not real sure on how to change it so that a specific
field uses Doc View. Or if it is easier to make it the default for all
fields I suppose that's fine too since it sounds like that will happen
eventually.
Just need help with what the structure of the template should be... Thanks!
On Wednesday, April 15, 2015 at 8:29:28 PM UTC-4, Mark Walkom wrote:
Yes that is correct, you have to update your mappings and wait for new
indices to be created from it, it's not something that can be applied
retroactively without reindexing.
Yea, that's where I started with it. But, if I understand it, that looks
like how I can change the mapping for a specific property. But I would
think I need to make a similar change to my index template otherwise new
indexes that get created will no long have that mapping. Or am I
misunderstanding?
On Wednesday, April 15, 2015 at 7:20:22 PM UTC-4, Mark Walkom wrote:
Thanks. The field I wanted to map was @timestamp which isn't explicitly in
the template. What would it look like?
Also, once I have made the change to my template, what's the right way to
test it (validate that for a new index i is using Doc Value for the
specific field)?
On Thursday, April 16, 2015 at 7:57:41 PM UTC-4, Mark Walkom wrote:
On 17 April 2015 at 09:35, Scott Chapman <scotted...@gmail.com
<javascript:>> wrote:
Thanks, that's what I thought.
So, please help me with my template I gave above. I am familiar with up
to update it, I am just not real sure on how to change it so that a
specific field uses Doc View. Or if it is easier to make it the default for
all fields I suppose that's fine too since it sounds like that will happen
eventually.
Just need help with what the structure of the template should be...
Thanks!
On Wednesday, April 15, 2015 at 8:29:28 PM UTC-4, Mark Walkom wrote:
Yes that is correct, you have to update your mappings and wait for new
indices to be created from it, it's not something that can be applied
retroactively without reindexing.
Yea, that's where I started with it. But, if I understand it, that
looks like how I can change the mapping for a specific property. But I
would think I need to make a similar change to my index template otherwise
new indexes that get created will no long have that mapping. Or am I
misunderstanding?
On Wednesday, April 15, 2015 at 7:20:22 PM UTC-4, Mark Walkom wrote:
Thanks. The field I wanted to map was @timestamp which isn't explicitly in
the template. What would it look like?
Also, once I have made the change to my template, what's the right way to
test it (validate that for a new index i is using Doc Value for the
specific field)?
On Thursday, April 16, 2015 at 7:57:41 PM UTC-4, Mark Walkom wrote:
So, please help me with my template I gave above. I am familiar with up
to update it, I am just not real sure on how to change it so that a
specific field uses Doc View. Or if it is easier to make it the default for
all fields I suppose that's fine too since it sounds like that will happen
eventually.
Just need help with what the structure of the template should be...
Thanks!
On Wednesday, April 15, 2015 at 8:29:28 PM UTC-4, Mark Walkom wrote:
Yes that is correct, you have to update your mappings and wait for new
indices to be created from it, it's not something that can be applied
retroactively without reindexing.
Yea, that's where I started with it. But, if I understand it, that
looks like how I can change the mapping for a specific property. But I
would think I need to make a similar change to my index template otherwise
new indexes that get created will no long have that mapping. Or am I
misunderstanding?
On Wednesday, April 15, 2015 at 7:20:22 PM UTC-4, Mark Walkom wrote:
On 17 April 2015 at 10:14, Scott Chapman <scotted...@gmail.com
<javascript:>> wrote:
Thanks. The field I wanted to map was @timestamp which isn't explicitly
in the template. What would it look like?
Also, once I have made the change to my template, what's the right way to
test it (validate that for a new index i is using Doc Value for the
specific field)?
On Thursday, April 16, 2015 at 7:57:41 PM UTC-4, Mark Walkom wrote:
So, please help me with my template I gave above. I am familiar with up
to update it, I am just not real sure on how to change it so that a
specific field uses Doc View. Or if it is easier to make it the default for
all fields I suppose that's fine too since it sounds like that will happen
eventually.
Just need help with what the structure of the template should be...
Thanks!
On Wednesday, April 15, 2015 at 8:29:28 PM UTC-4, Mark Walkom wrote:
Yes that is correct, you have to update your mappings and wait for new
indices to be created from it, it's not something that can be applied
retroactively without reindexing.
Yea, that's where I started with it. But, if I understand it, that
looks like how I can change the mapping for a specific property. But I
would think I need to make a similar change to my index template otherwise
new indexes that get created will no long have that mapping. Or am I
misunderstanding?
On Wednesday, April 15, 2015 at 7:20:22 PM UTC-4, Mark Walkom wrote:
It's a little difficult to see what is currently in field data to check
(you need a heap dump). You could probably keep an eye on existing field
data and see if it increases slower than before but that's a little
abstract.
Really, as long as it doesn't complain about the mapping you're good.
Thanks. The field I wanted to map was @timestamp which isn't explicitly
in the template. What would it look like?
Also, once I have made the change to my template, what's the right way
to test it (validate that for a new index i is using Doc Value for the
specific field)?
On Thursday, April 16, 2015 at 7:57:41 PM UTC-4, Mark Walkom wrote:
So, please help me with my template I gave above. I am familiar with
up to update it, I am just not real sure on how to change it so that a
specific field uses Doc View. Or if it is easier to make it the default for
all fields I suppose that's fine too since it sounds like that will happen
eventually.
Just need help with what the structure of the template should be...
Thanks!
On Wednesday, April 15, 2015 at 8:29:28 PM UTC-4, Mark Walkom wrote:
Yes that is correct, you have to update your mappings and wait for
new indices to be created from it, it's not something that can be applied
retroactively without reindexing.
Yea, that's where I started with it. But, if I understand it, that
looks like how I can change the mapping for a specific property. But I
would think I need to make a similar change to my index template otherwise
new indexes that get created will no long have that mapping. Or am I
misunderstanding?
On Wednesday, April 15, 2015 at 7:20:22 PM UTC-4, Mark Walkom wrote:
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.