Static data / metadata in an index

Elastic Stack Elasticsearch
1

Hi!

Is there a way to add something like static data or metadata to an index?
I have some data (some fields) in my indices that ALWAYS get the same values. Is there a way to set this fields to some static / metadata field in the index, so it is only once in the index and then it is retrieved in every search. If I set it to every document I guess it will take a lot of space so it makes no sense for me. It would be something like the "_index" value (https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-index-field.html)
I do not need this fields for searching, I just need them to be retrieved in the searches.
I know Elasticsearch have a "_meta" field (https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-meta-field.html) I can set in the mapping of an index, however, to retrieve it I have to perform a get indexName/_mapping. This also does not make much sense in my case because I would need two queries and I would need to know the exact index name, when I sometimes search in several indices at once.

Thanks in advance!

2

You can add a meta object in your documents and in the mapping disable this object so it's content will be ignored but stored in _source.
See https://www.elastic.co/guide/en/elasticsearch/reference/current/enabled.html

3

Hi!

Thanks for a really fast answer!!
Setting enabled to false on these fields I bet is faster (since it does not have to index it) and gets less disk space, so it is much better setting it, however I see it writes on EVERY document the values so it is taking a lot of disk space as document count grows.
I am sure that if you told me just to set the enabled to false, there is no way to set these fields to be part of the index and not part of the documents (like the _index one, that is a virtual field saved ONLY to index and saved only ONCE spending just I little disk space). I am just confirming there is no way.
All documents with the same data in these fields will use much disk space.

Thanks again!!

4

The only way to make it part of the index and not the document is exactly what you wrote early, using _meta field in the mapping. That'd be my advice.
But I tried to answer your question :slight_smile:

What is wrong with using _meta? As this is a fixed field for an index, when your application is starting you can totally load the mapping from elasticsearch and cache in memory the _meta values you have. Then you're basically done.

That will save disk space.

note though that _source field is compressed. So may be having this part of the source is not a super big deal, but you have to measure that.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.