# Index metadata **URL:** \ **Category:** Elasticsearch\ **Created:** [December 29, 2011, 1:59pm UTC](https://discuss.elastic.co/t/index-metadata/6251 "2011-12-29T13:59:15Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![MC1](https://avatars.discourse-cdn.com/v4/letter/m/b3f665/32.png) [@MC1](https://discuss.elastic.co/u/MC1)\ **Post date:** [December 29, 2011, 1:59pm UTC](https://discuss.elastic.co/t/index-metadata/6251/1 "2011-12-29T13:59:15Z")
Is it possible to retrieve index metadata via the API? In terms of metadata, I am looking to be able to dynamically retrieve things like: 1. The date/time at which an index was last updated 2. The field names (currently) within an index 3. The field types (string/boolean/date/numeric) currently in the index 4. The field storage scheme (i.e. stored, indexed, etc...) and the like. Thanks, Mayer ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [December 29, 2011, 5:38pm UTC](https://discuss.elastic.co/t/index-metadata/6251/2 "2011-12-29T17:38:18Z")
Mayer, I do not think it is possible to get the date/time of the last time an index was updated. Perhaps someone can prove otherwise. I would be a bit tough in a distributed environment. For info on the fields, you can use the mapping API: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. Fields that are not explicitly defined in your initial mapping will be added dynamically to the mapping. The GET mapping call will contain all fields. -- Ivan On Thu, Dec 29, 2011 at 5:59 AM, MC [mayer.crystal@gmail.com](mailto:mayer.crystal@gmail.com) wrote: > Is it possible to retrieve index metadata via the API? In terms of > metadata, I am looking to be able to dynamically retrieve things like: > > 1. The date/time at which an index was last updated > 2. The field names (currently) within an index > 3. The field types (string/boolean/date/numeric) currently in the > index > 4. The field storage scheme (i.e. stored, indexed, etc...) > > and the like. > > Thanks, > Mayer ---
**Author:** ![plaflamme](https://avatars.discourse-cdn.com/v4/letter/p/a88e57/32.png) [@plaflamme](https://discuss.elastic.co/u/plaflamme)\ **Post date:** [December 30, 2011, 2:56pm UTC](https://discuss.elastic.co/t/index-metadata/6251/3 "2011-12-30T14:56:28Z")
We manually store timestamps for creation and last update of our indices within the "\_meta" tag of a mapping. Then we use "Get Mapping" to fetch these timestamps. An alternative is to use the new timestamps[1] of the documents in the index and search on that field, maybe using the term\_stats facet to obtain the max... Hope it helps, Philippe [1] > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. On Thu, Dec 29, 2011 at 12:38, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > Mayer, > > I do not think it is possible to get the date/time of the last time an > index was updated. Perhaps someone can prove otherwise. I would be a > bit tough in a distributed environment. > > For info on the fields, you can use the mapping API: > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-get-mapping.html) > > Fields that are not explicitly defined in your initial mapping will be > added dynamically to the mapping. The GET mapping call will contain > all fields. > > -- > Ivan > > On Thu, Dec 29, 2011 at 5:59 AM, MC [mayer.crystal@gmail.com](mailto:mayer.crystal@gmail.com) wrote: > > > Is it possible to retrieve index metadata via the API? In terms of > > metadata, I am looking to be able to dynamically retrieve things like: > > > > 1. The date/time at which an index was last updated > > 2. The field names (currently) within an index > > 3. The field types (string/boolean/date/numeric) currently in the > > index > > 4. The field storage scheme (i.e. stored, indexed, etc...) > > > > and the like. > > > > Thanks, > > Mayer ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 30, 2011, 3:06pm UTC](https://discuss.elastic.co/t/index-metadata/6251/4 "2011-12-30T15:06:17Z")
Storing the last update in the \_meta part of the mapping? So does that mean that for each document indexed, you update the mapping? Thats expensive... On Fri, Dec 30, 2011 at 4:56 PM, Philippe Laflamme \< [philippe.laflamme@obiba.org](mailto:philippe.laflamme@obiba.org)\> wrote: > We manually store timestamps for creation and last update of our indices > within the "\_meta" tag of a mapping. Then we use "Get Mapping" to fetch > these timestamps. > > An alternative is to use the new timestamps[1] of the documents in the > index and search on that field, maybe using the term\_stats facet to obtain > the max... > > Hope it helps, > Philippe > > [1] > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/timestamp-field.html) > > On Thu, Dec 29, 2011 at 12:38, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > Mayer, > > > > I do not think it is possible to get the date/time of the last time an > > index was updated. Perhaps someone can prove otherwise. I would be a > > bit tough in a distributed environment. > > > > For info on the fields, you can use the mapping API: > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-get-mapping.html) > > > > Fields that are not explicitly defined in your initial mapping will be > > added dynamically to the mapping. The GET mapping call will contain > > all fields. > > > > -- > > Ivan > > > > On Thu, Dec 29, 2011 at 5:59 AM, MC [mayer.crystal@gmail.com](mailto:mayer.crystal@gmail.com) wrote: > > > > > Is it possible to retrieve index metadata via the API? In terms of > > > metadata, I am looking to be able to dynamically retrieve things like: > > > > > > 1. The date/time at which an index was last updated > > > 2. The field names (currently) within an index > > > 3. The field types (string/boolean/date/numeric) currently in the > > > index > > > 4. The field storage scheme (i.e. stored, indexed, etc...) > > > > > > and the like. > > > > > > Thanks, > > > Mayer ---
**Author:** ![plaflamme](https://avatars.discourse-cdn.com/v4/letter/p/a88e57/32.png) [@plaflamme](https://discuss.elastic.co/u/plaflamme)\ **Post date:** [December 30, 2011, 4:02pm UTC](https://discuss.elastic.co/t/index-metadata/6251/5 "2011-12-30T16:02:39Z")
Yes, that's what we ended up with. At the time, the document timestamps didn't exist. I wasn't aware that updating the mapping was expensive. Is this the case even when it's not chagning at all (except under \_meta)? Philippe On Fri, Dec 30, 2011 at 10:06, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > Storing the last update in the \_meta part of the mapping? So does that > mean that for each document indexed, you update the mapping? Thats > expensive... > > On Fri, Dec 30, 2011 at 4:56 PM, Philippe Laflamme \< > [philippe.laflamme@obiba.org](mailto:philippe.laflamme@obiba.org)\> wrote: > > > We manually store timestamps for creation and last update of our indices > > within the "\_meta" tag of a mapping. Then we use "Get Mapping" to fetch > > these timestamps. > > > > An alternative is to use the new timestamps[1] of the documents in the > > index and search on that field, maybe using the term\_stats facet to obtain > > the max... > > > > Hope it helps, > > Philippe > > > > [1] > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/timestamp-field.html) > > > > On Thu, Dec 29, 2011 at 12:38, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > > > Mayer, > > > > > > I do not think it is possible to get the date/time of the last time an > > > index was updated. Perhaps someone can prove otherwise. I would be a > > > bit tough in a distributed environment. > > > > > > For info on the fields, you can use the mapping API: > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-get-mapping.html) > > > > > > Fields that are not explicitly defined in your initial mapping will be > > > added dynamically to the mapping. The GET mapping call will contain > > > all fields. > > > > > > -- > > > Ivan > > > > > > On Thu, Dec 29, 2011 at 5:59 AM, MC [mayer.crystal@gmail.com](mailto:mayer.crystal@gmail.com) wrote: > > > > > > > Is it possible to retrieve index metadata via the API? In terms of > > > > metadata, I am looking to be able to dynamically retrieve things like: > > > > > > > > 1. The date/time at which an index was last updated > > > > 2. The field names (currently) within an index > > > > 3. The field types (string/boolean/date/numeric) currently in the > > > > index > > > > 4. The field storage scheme (i.e. stored, indexed, etc...) > > > > > > > > and the like. > > > > > > > > Thanks, > > > > Mayer ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 30, 2011, 4:15pm UTC](https://discuss.elastic.co/t/index-metadata/6251/6 "2011-12-30T16:15:52Z")
Well, expensive in the sense that it needs to be broadcasted to the cluster and persisted (the whole metadata). Even without the timestamp feature, you can add a timestamp to a document indexed, and find the latest one, you don't need the timestamp to have that ability. On Fri, Dec 30, 2011 at 6:02 PM, Philippe Laflamme \< [philippe.laflamme@obiba.org](mailto:philippe.laflamme@obiba.org)\> wrote: > Yes, that's what we ended up with. At the time, the document timestamps > didn't exist. I wasn't aware that updating the mapping was expensive. Is > this the case even when it's not chagning at all (except under \_meta)? > > Philippe > > On Fri, Dec 30, 2011 at 10:06, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > > > Storing the last update in the \_meta part of the mapping? So does that > > mean that for each document indexed, you update the mapping? Thats > > expensive... > > > > On Fri, Dec 30, 2011 at 4:56 PM, Philippe Laflamme \< > > [philippe.laflamme@obiba.org](mailto:philippe.laflamme@obiba.org)\> wrote: > > > > > We manually store timestamps for creation and last update of our indices > > > within the "\_meta" tag of a mapping. Then we use "Get Mapping" to fetch > > > these timestamps. > > > > > > An alternative is to use the new timestamps[1] of the documents in the > > > index and search on that field, maybe using the term\_stats facet to obtain > > > the max... > > > > > > Hope it helps, > > > Philippe > > > > > > [1] > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/timestamp-field.html) > > > > > > On Thu, Dec 29, 2011 at 12:38, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > > > > > Mayer, > > > > > > > > I do not think it is possible to get the date/time of the last time an > > > > index was updated. Perhaps someone can prove otherwise. I would be a > > > > bit tough in a distributed environment. > > > > > > > > For info on the fields, you can use the mapping API: > > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-get-mapping.html) > > > > > > > > Fields that are not explicitly defined in your initial mapping will be > > > > added dynamically to the mapping. The GET mapping call will contain > > > > all fields. > > > > > > > > -- > > > > Ivan > > > > > > > > On Thu, Dec 29, 2011 at 5:59 AM, MC [mayer.crystal@gmail.com](mailto:mayer.crystal@gmail.com) wrote: > > > > > > > > > Is it possible to retrieve index metadata via the API? In terms of > > > > > metadata, I am looking to be able to dynamically retrieve things like: > > > > > > > > > > 1. The date/time at which an index was last updated > > > > > 2. The field names (currently) within an index > > > > > 3. The field types (string/boolean/date/numeric) currently in the > > > > > index > > > > > 4. The field storage scheme (i.e. stored, indexed, etc...) > > > > > > > > > > and the like. > > > > > > > > > > Thanks, > > > > > Mayer ---
**Author:** ![plaflamme](https://avatars.discourse-cdn.com/v4/letter/p/a88e57/32.png) [@plaflamme](https://discuss.elastic.co/u/plaflamme)\ **Post date:** [December 30, 2011, 4:25pm UTC](https://discuss.elastic.co/t/index-metadata/6251/7 "2011-12-30T16:25:36Z")
Yes, we did consider adding timestamps to documents at the time, but it seemed more appropriate to use the \_meta tag since it's an index-level property (in our case at least). Maybe the \_meta tag could be made accessible from outside the mapping: since it's not used by es, updating it shouldn't have any impact; just a thought. What would be the most efficient way to query the "latest" (maximum) value when using a timestamp field on the documents? Thanks, Philippe On Fri, Dec 30, 2011 at 11:15, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > Well, expensive in the sense that it needs to be broadcasted to the > cluster and persisted (the whole metadata). Even without the timestamp > feature, you can add a timestamp to a document indexed, and find the latest > one, you don't need the timestamp to have that ability. > > On Fri, Dec 30, 2011 at 6:02 PM, Philippe Laflamme \< > [philippe.laflamme@obiba.org](mailto:philippe.laflamme@obiba.org)\> wrote: > > > Yes, that's what we ended up with. At the time, the document timestamps > > didn't exist. I wasn't aware that updating the mapping was expensive. Is > > this the case even when it's not chagning at all (except under \_meta)? > > > > Philippe > > > > On Fri, Dec 30, 2011 at 10:06, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > > > > > Storing the last update in the \_meta part of the mapping? So does that > > > mean that for each document indexed, you update the mapping? Thats > > > expensive... > > > > > > On Fri, Dec 30, 2011 at 4:56 PM, Philippe Laflamme \< > > > [philippe.laflamme@obiba.org](mailto:philippe.laflamme@obiba.org)\> wrote: > > > > > > > We manually store timestamps for creation and last update of our > > > > indices within the "\_meta" tag of a mapping. Then we use "Get Mapping" to > > > > fetch these timestamps. > > > > > > > > An alternative is to use the new timestamps[1] of the documents in the > > > > index and search on that field, maybe using the term\_stats facet to obtain > > > > the max... > > > > > > > > Hope it helps, > > > > Philippe > > > > > > > > [1] > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/timestamp-field.html) > > > > > > > > On Thu, Dec 29, 2011 at 12:38, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > > > > > > > Mayer, > > > > > > > > > > I do not think it is possible to get the date/time of the last time an > > > > > index was updated. Perhaps someone can prove otherwise. I would be a > > > > > bit tough in a distributed environment. > > > > > > > > > > For info on the fields, you can use the mapping API: > > > > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-get-mapping.html) > > > > > > > > > > Fields that are not explicitly defined in your initial mapping will be > > > > > added dynamically to the mapping. The GET mapping call will contain > > > > > all fields. > > > > > > > > > > -- > > > > > Ivan > > > > > > > > > > On Thu, Dec 29, 2011 at 5:59 AM, MC [mayer.crystal@gmail.com](mailto:mayer.crystal@gmail.com) wrote: > > > > > > > > > > > Is it possible to retrieve index metadata via the API? In terms of > > > > > > metadata, I am looking to be able to dynamically retrieve things > > > > > > like: > > > > > > > > > > > > 1. The date/time at which an index was last updated > > > > > > 2. The field names (currently) within an index > > > > > > 3. The field types (string/boolean/date/numeric) currently in the > > > > > > index > > > > > > 4. The field storage scheme (i.e. stored, indexed, etc...) > > > > > > > > > > > > and the like. > > > > > > > > > > > > Thanks, > > > > > > Mayer ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 30, 2011, 6:39pm UTC](https://discuss.elastic.co/t/index-metadata/6251/8 "2011-12-30T18:39:44Z")
It does not matter if \_meta is extracted or not, it still needs to be broadcasted and persisted. The best way to get the "latest" value is to sort base on the time field and ask for a single hit. On Fri, Dec 30, 2011 at 6:25 PM, Philippe Laflamme \< [philippe.laflamme@obiba.org](mailto:philippe.laflamme@obiba.org)\> wrote: > Yes, we did consider adding timestamps to documents at the time, but it > seemed more appropriate to use the \_meta tag since it's an index-level > property (in our case at least). Maybe the \_meta tag could be made > accessible from outside the mapping: since it's not used by es, updating it > shouldn't have any impact; just a thought. > > What would be the most efficient way to query the "latest" (maximum) value > when using a timestamp field on the documents? > > Thanks, > Philippe > > On Fri, Dec 30, 2011 at 11:15, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > > > Well, expensive in the sense that it needs to be broadcasted to the > > cluster and persisted (the whole metadata). Even without the timestamp > > feature, you can add a timestamp to a document indexed, and find the latest > > one, you don't need the timestamp to have that ability. > > > > On Fri, Dec 30, 2011 at 6:02 PM, Philippe Laflamme \< > > [philippe.laflamme@obiba.org](mailto:philippe.laflamme@obiba.org)\> wrote: > > > > > Yes, that's what we ended up with. At the time, the document timestamps > > > didn't exist. I wasn't aware that updating the mapping was expensive. Is > > > this the case even when it's not chagning at all (except under \_meta)? > > > > > > Philippe > > > > > > On Fri, Dec 30, 2011 at 10:06, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > > > > > > > Storing the last update in the \_meta part of the mapping? So does that > > > > mean that for each document indexed, you update the mapping? Thats > > > > expensive... > > > > > > > > On Fri, Dec 30, 2011 at 4:56 PM, Philippe Laflamme \< > > > > [philippe.laflamme@obiba.org](mailto:philippe.laflamme@obiba.org)\> wrote: > > > > > > > > > We manually store timestamps for creation and last update of our > > > > > indices within the "\_meta" tag of a mapping. Then we use "Get Mapping" to > > > > > fetch these timestamps. > > > > > > > > > > An alternative is to use the new timestamps[1] of the documents in the > > > > > index and search on that field, maybe using the term\_stats facet to obtain > > > > > the max... > > > > > > > > > > Hope it helps, > > > > > Philippe > > > > > > > > > > [1] > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/timestamp-field.html) > > > > > > > > > > On Thu, Dec 29, 2011 at 12:38, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > > > > > > > > > Mayer, > > > > > > > > > > > > I do not think it is possible to get the date/time of the last time an > > > > > > index was updated. Perhaps someone can prove otherwise. I would be a > > > > > > bit tough in a distributed environment. > > > > > > > > > > > > For info on the fields, you can use the mapping API: > > > > > > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/admin-indices-get-mapping.html) > > > > > > > > > > > > Fields that are not explicitly defined in your initial mapping will be > > > > > > added dynamically to the mapping. The GET mapping call will contain > > > > > > all fields. > > > > > > > > > > > > -- > > > > > > Ivan > > > > > > > > > > > > On Thu, Dec 29, 2011 at 5:59 AM, MC [mayer.crystal@gmail.com](mailto:mayer.crystal@gmail.com) wrote: > > > > > > > > > > > > > Is it possible to retrieve index metadata via the API? In terms of > > > > > > > metadata, I am looking to be able to dynamically retrieve things > > > > > > > like: > > > > > > > > > > > > > > 1. The date/time at which an index was last updated > > > > > > > 2. The field names (currently) within an index > > > > > > > 3. The field types (string/boolean/date/numeric) currently in the > > > > > > > index > > > > > > > 4. The field storage scheme (i.e. stored, indexed, etc...) > > > > > > > > > > > > > > and the like. > > > > > > > > > > > > > > Thanks, > > > > > > > Mayer ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:44am UTC](https://discuss.elastic.co/t/index-metadata/6251/9 "2017-07-06T03:44:07Z")