# Is it possible to query app-search documents through elastic search API? **URL:** \ **Category:** Elastic Search\ **Tags:** elastic-app-search\ **Created:** [August 26, 2020, 1:07am UTC](https://discuss.elastic.co/t/is-it-possible-to-query-app-search-documents-through-elastic-search-api/246370 "2020-08-26T01:07:22Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Ali\_Modern](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ali_modern/32/45312_2.png) [@Ali\_Modern](https://discuss.elastic.co/u/Ali_Modern)\ **Post date:** [August 26, 2020, 1:07am UTC](https://discuss.elastic.co/t/is-it-possible-to-query-app-search-documents-through-elastic-search-api/246370/1 "2020-08-26T01:07:22Z")
I have self-hosted app search instance with the Elastic stack (ELK+Appsearch). I can't see app-search index in Kibana. I'm wondering if there is any other way to manage the documents app-search is working on apart from app-search API? Can I use Logstash to keep app-search documents in sync with my SQL Server database? ---
**Author:** ![orhantoy](https://avatars.discourse-cdn.com/v4/letter/o/c5a1d2/32.png) [@orhantoy](https://discuss.elastic.co/u/orhantoy)\ **Post date:** [August 27, 2020, 1:44pm UTC](https://discuss.elastic.co/t/is-it-possible-to-query-app-search-documents-through-elastic-search-api/246370/2 "2020-08-27T13:44:00Z")
Hi @Ali_Modern 👋 Have you looked into the App Search output plugin: [https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elastic\_app\_search.html](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elastic_app_search.html)? ---
**Author:** ![Ali\_Modern](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ali_modern/32/45312_2.png) [@Ali\_Modern](https://discuss.elastic.co/u/Ali_Modern)\ **Post date:** [August 27, 2020, 5:47pm UTC](https://discuss.elastic.co/t/is-it-possible-to-query-app-search-documents-through-elastic-search-api/246370/3 "2020-08-27T17:47:45Z")
Hey @orhantoy! Thanks for providing that. That is a helpful plugin to feed new/updated documents into App Search. However, we're trying to keep it fully synced with a master data source. Documents updated/removed/created in master need to be reflected in app-search. So, we're looking for a full CRUD capability. Create/Update -\> DONE. API or Logstash plugin. We're able to create as many as we want. Read -\> Limited. You can only list up to 10k (not to mention you need 100 requests of getting 100 documents each time). In our use case, we have 14k. Delete -\> Blocked. We can't delete what we can't list in the first place. Once created, a document after 10k becomes a ghost that only appears in search results. That said, when we were looking into alternatives to the official App Search API which was obviously designed for front-end experience (explains paging), we were looking into alternatives that work better with machines. E.g. API that could return the entire dataset in one request somehow. That led us to think we could access the documents from the bottom; Elastic search instance that presumably stores the documents (and doesn't show it as an index in Kibana). So to summarize, the question is: Is it possible to manage (CRUD) app-search documents from anywhere else other than its official API? ---
**Author:** ![Todd\_Hiles](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/todd_hiles/32/75537_2.png) [@Todd\_Hiles](https://discuss.elastic.co/u/Todd_Hiles)\ **Post date:** [September 14, 2020, 1:09pm UTC](https://discuss.elastic.co/t/is-it-possible-to-query-app-search-documents-through-elastic-search-api/246370/4 "2020-09-14T13:09:18Z")
I'm having similar challenges using the App Search Document API. App search hides the underlying index 'somehow' with the 'system indices'. You can find it via [https://localhost:5601/app/kibana#/management/elasticsearch/index\_management/indices](https://localhost:5601/app/kibana#/management/elasticsearch/index_management/indices) and select the 'include system indices' option. To find it more quickly knowing how many documents it may have could help, but searching with the term 'search-engine' will give you a list of all App Search managed search engines/indices. This is a convention observed and not an API and has the caveats that come with such embedded knowledge/observation. Any manipulation of the underlying index MAY cause App Search to misbehave. Note: I'm just a user of this system and don't know the inner workings! It seems App Search doesn't take advantage of index 'alias' feature of Elasticsearch for index rebuilding, so it's probable that it creates a new index in Elasticsearch upon 'settings' change etc. This was gleaned from the dialog presented when poking around with Kibana 'lifecycle policy' feature: Add lifecycle policy to ".ent-search-engine-5f29af24d9eadc7a567c6d11" Lifecycle policy ``` kibana-event-log-policy Index has no aliases Policy kibana-event-log-policy is configured for rollover, but index .ent-search-engine-5f29af24d9eadc7a567c6d11 does not have an alias, which is required for rollover. ``` ---
**Author:** ![Todd\_Hiles](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/todd_hiles/32/75537_2.png) [@Todd\_Hiles](https://discuss.elastic.co/u/Todd_Hiles)\ **Post date:** [September 14, 2020, 2:02pm UTC](https://discuss.elastic.co/t/is-it-possible-to-query-app-search-documents-through-elastic-search-api/246370/5 "2020-09-14T14:02:23Z")
After going through a few Schema changes via the App Search UI at [http://localhost:3002/as#/engines/my-engine-name/schema/edit](http://localhost:3002/as#/engines/my-engine-name/schema/edit) I observed the addition of mappings but no removal. Also underlying Elasticsearch index was unchanged. Changing the `created_at` schema type from `text` to `date`and then from `date` to `text` produced the following mappings entries (from ui at [https://localhost:5601/app/kibana#/management/elasticsearch/index\_management/indices](https://localhost:5601/app/kibana#/management/elasticsearch/index_management/indices)) : ```json "created_at$date": { "type": "date" }, "created_at$string": { "type": "text", "index_options": "offsets", "fields": { "delimiter": { "type": "text", "term_vector": "with_positions_offsets", "index_options": "offsets", "analyzer": "iq_text_delimiter", "position_increment_gap": 100 }, "enum": { "type": "keyword", "ignore_above": 2048 }, "intragram": { "type": "text", "index_options": "docs", "analyzer": "iq_intragram" }, "joined": { "type": "text", "term_vector": "with_positions_offsets", "index_options": "offsets", "analyzer": "iq_text_bigram", "position_increment_gap": 100 }, "prefix": { "type": "text", "term_vector": "with_positions_offsets", "index_options": "docs", "analyzer": "i_prefix", "search_analyzer": "q_prefix" }, "stem": { "type": "text", "term_vector": "with_positions_offsets", "index_options": "offsets", "analyzer": "iq_text_stem", "position_increment_gap": 100 } }, "analyzer": "iq_text_base", "position_increment_gap": 100 } ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 12, 2020, 2:02pm UTC](https://discuss.elastic.co/t/is-it-possible-to-query-app-search-documents-through-elastic-search-api/246370/6 "2020-10-12T14:02:24Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed. ---
**Author:** ![JonasLavoie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jonaslavoie/32/56961_2.png) [@JonasLavoie](https://discuss.elastic.co/u/JonasLavoie)\ **Post date:** [June 16, 2022, 2:32pm UTC](https://discuss.elastic.co/t/is-it-possible-to-query-app-search-documents-through-elastic-search-api/246370/7 "2022-06-16T14:32:04Z")
Hey all - we've recently introduced new functionality on two fronts that will certainly help with the use case at hand: 1. [A way to use existing Elasticsearch indices with App Search.](https://www.elastic.co/guide/en/app-search/current/elasticsearch-engines.html) You can still create and managed indices (via engines) with App Search, but this gives you more flexibility on where and how the data is indexed. If some restaurants are _bring your own wine_, App Search is now _bring your own Elasticsearch index_ 🙂 2. [A way to use the Elasticsearch Query DSL for any App Search engine.](https://www.elastic.co/guide/en/app-search/current/elasticsearch-search-guide.html) This means even the most demanding of queries can be issued against App Search engines, with all the automatic goodness of Analytics capture, for example. Hope this helps you all on this thread, and perhaps any other peer that stumbles upon this thread! -- Jonas