# Watcher to index all data from /cat/indices/\*,-.\* each indexname should be inserted as one doc **URL:** \ **Category:** Kibana\ **Tags:** elastic-stack-alerting\ **Created:** [January 9, 2024, 4:19pm UTC](https://discuss.elastic.co/t/watcher-to-index-all-data-from-cat-indices-each-indexname-should-be-inserted-as-one-doc/350672 "2024-01-09T16:19:21Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![mangeshmj1992](https://avatars.discourse-cdn.com/v4/letter/m/d9b06d/32.png) [@mangeshmj1992](https://discuss.elastic.co/u/mangeshmj1992)\ **Post date:** [January 9, 2024, 4:19pm UTC](https://discuss.elastic.co/t/watcher-to-index-all-data-from-cat-indices-each-indexname-should-be-inserted-as-one-doc/350672/1 "2024-01-09T16:19:21Z")
Hi team, Can you please help me on below requirement: 1. Get index name and size (in bytes) from GET /cat/indices/_,-._ 2. Insert this data into new index 3. IDs should be elastic generated. Add timestamp field in every doc Problem: I have created below watcher but it is inserting all index info as one document . I need each index name and size as one document. ```auto { "trigger": { "schedule": { "daily": { "at": [ "noon" ] } } }, "input": { "http": { "request": { "scheme": "http", "host": "localhost", "port": 9200, "method": "get", "path": "/_cat/indices/*,-.*", "params": { "format": "json", "h": "index,store.size", "bytes": "b" }, "headers": {}, "auth": { "basic": { "username": "abc", "password": "abc" } } } } }, "condition": { "always": {} }, "actions": { "email_admin": { "email": { "profile": "standard", "to": [ "abc@abc.com" ], "subject": "404 recently encountered", "body": { "text": "indices \n {{ctx.payload}}" } } }, "index_payload": { "index": { "index": "my-index", "doc_type": "_doc" } } } } ``` ---
**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\ **Post date:** [January 11, 2024, 1:47pm UTC](https://discuss.elastic.co/t/watcher-to-index-all-data-from-cat-indices-each-indexname-should-be-inserted-as-one-doc/350672/2 "2024-01-11T13:47:59Z")
Following [this example](https://www.elastic.co/guide/en/elasticsearch/reference/current/actions-index.html#anatomy-actions-index-multi-doc-support) in the docs, you could try ```auto { "trigger": { "schedule": { "interval": "1d" } }, "input": { "http": { "request": { "host": "localhost", "port": 9200, "scheme": "https", "path": "/_cat/indices/*,-.*", "params": { "format": "json", "h": "index,store.size", "bytes": "b" }, "headers": {}, "auth": { "basic": { "username": "elastic", "password": "xxx" } } } } }, "condition": { "always": {} }, "actions" : { "index_payload": { "transform": { "script": """ def documents = ctx.payload.data.stream() .map(hit -> [ "_index": "index_sizes", "index": hit.index, "store.size": hit['store.size'] ]) .collect(Collectors.toList()); return ["_doc" : documents]; """ }, "index": { "execution_time_field": "@timestamp" } } } } ``` ---
**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\ **Post date:** [January 11, 2024, 1:51pm UTC](https://discuss.elastic.co/t/watcher-to-index-all-data-from-cat-indices-each-indexname-should-be-inserted-as-one-doc/350672/3 "2024-01-11T13:51:10Z")
`GET index_sizes/_search` yields something like: ```auto { "took": 0, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 3, "relation": "eq" }, "max_score": 1, "hits": [ { "_index": "index_sizes", "_id": "y1jD-IwBRZWNq0rMQ4W9", "_score": 1, "_source": { "@timestamp": "2024-01-11T13:43:15.438Z", "index": "test-data", "store.size": "907197" } }, { "_index": "index_sizes", "_id": "zFjD-IwBRZWNq0rMQ4W9", "_score": 1, "_source": { "@timestamp": "2024-01-11T13:43:15.438Z", "index": "my-index", "store.size": "284852" } }, { "_index": "index_sizes", "_id": "0VjD-IwBRZWNq0rMQ4W9", "_score": 1, "_source": { "@timestamp": "2024-01-11T13:43:15.438Z", "index": "kibana_sample_data_ecommerce", "store.size": "8783478" } } ] ``` ---
**Author:** ![mangeshmj1992](https://avatars.discourse-cdn.com/v4/letter/m/d9b06d/32.png) [@mangeshmj1992](https://discuss.elastic.co/u/mangeshmj1992)\ **Post date:** [January 11, 2024, 4:40pm UTC](https://discuss.elastic.co/t/watcher-to-index-all-data-from-cat-indices-each-indexname-should-be-inserted-as-one-doc/350672/4 "2024-01-11T16:40:03Z")
Hi @richcollier , thanks for the help, i am using 6.8.23 version and getting below 1 error . can you please help me on this. NOTE: it is working in 8.6 version ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/6/26f3cafd05a21499be0fb57e6369d51eb4fa9b17.png) ---
**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\ **Post date:** [January 11, 2024, 11:41pm UTC](https://discuss.elastic.co/t/watcher-to-index-all-data-from-cat-indices-each-indexname-should-be-inserted-as-one-doc/350672/5 "2024-01-11T23:41:49Z")
Seems like it doesn't like the defining of a script using the triple double-quote notation (i.e `"""`) . This syntax means that you're able to use free-formatting until the closing `"""`, including new lines. If you look back at the [v6.8 documentation](https://www.elastic.co/guide/en/elasticsearch/reference/6.8/transform-script.html), it only shows defining a script on a single line deliniated with only a single double-quote (`"`) on either side of the code snippet. You could try to take what I wrote, remove the new lines, put it all on one contiguous line, and surround it with a single double-quote. Might work! But you should know 6.x is end-of-life and has been for years. I'm sorry that it doesn't work for you but if my above suggestion doesn't help I'm not sure I can help you further. ---
**Author:** ![mangeshmj1992](https://avatars.discourse-cdn.com/v4/letter/m/d9b06d/32.png) [@mangeshmj1992](https://discuss.elastic.co/u/mangeshmj1992)\ **Post date:** [January 12, 2024, 9:38am UTC](https://discuss.elastic.co/t/watcher-to-index-all-data-from-cat-indices-each-indexname-should-be-inserted-as-one-doc/350672/6 "2024-01-12T09:38:38Z")
its working now thank you so much ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 9, 2024, 9:39am UTC](https://discuss.elastic.co/t/watcher-to-index-all-data-from-cat-indices-each-indexname-should-be-inserted-as-one-doc/350672/7 "2024-02-09T09:39:08Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.