# Logstash has fetched the data but not shipped into Elasticsearch **URL:** \ **Category:** Logstash\ **Created:** [June 1, 2018, 7:07am UTC](https://discuss.elastic.co/t/logstash-has-fetched-the-data-but-not-shipped-into-elasticsearch/134154 "2018-06-01T07:07:04Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Lee\_Jack](https://avatars.discourse-cdn.com/v4/letter/l/839c29/32.png) [@Lee\_Jack](https://discuss.elastic.co/u/Lee_Jack)\ **Post date:** [June 1, 2018, 7:07am UTC](https://discuss.elastic.co/t/logstash-has-fetched-the-data-but-not-shipped-into-elasticsearch/134154/1 "2018-06-01T07:07:05Z")
I am using Logstash to fetch data from another host's mysql database and want to ship the data into Elastic database.After starting the Logstash ,I can observe the Logstash has already fetch the new data which I insert into mysql ,but I can not find the new data in my Elastic database. here are my jdbc.conf: input { stdin { } jdbc { # mysql jdbc connection string to our backup databse jdbc\_connection\_string =\> "jdbc:mysql://192.168.9.223:3306/FANTASY" # the user we wish to excute our statement as jdbc\_user =\> "root" jdbc\_password =\> "123456" # the path to our downloaded jdbc driver jdbc\_driver\_library =\> "E:\Elasticsearch\elasticsearch-6.2.4\logstash-6.2.4\logstash-core\lib\jars\mysql-connector-java-8.0.11.jar" # the name of the driver class for mysql jdbc\_driver\_class =\> "com.mysql.jdbc.Driver" jdbc\_paging\_enabled =\> "true" jdbc\_page\_size =\> "50000" record\_last\_run =\> true last\_run\_metadata\_path =\> "E:\Elasticsearch\elasticsearch-6.2.4\logstash-6.2.4\logs.logstash\_jdbc\_last\_run" use\_column\_value =\> true tracking\_column =\> "id" tracking\_column\_type =\> "numeric" clean\_run =\> false statement\_filepath =\> "E:\Elasticsearch\elasticsearch-6.2.4\logstash-6.2.4\jdbc.sql" schedule =\> "\*/10 \* \* \* \*" type =\> "jdbc" } } filter { json { source =\> "message" remove\_field =\> ["message"] } } output { elasticsearch { hosts =\> ["192.168.9.58:9200"] #port =\> "9200" #protocol =\> "http" index =\> "lee-index" document\_id =\> "%{id}" } stdout { codec =\> json\_lines } } and I do nothing changed in pipelines.yml and logstash.yml. How can I get the data into Elastic ?Any suggestions would be welcomed I have check the result in command line : [2018-06-01T16:28:50,028][WARN][logstash.filters.json] Parsed JSON object/hash requires a target configuration option {:source=\>"message", :raw=\>""} [2018-06-01T16:28:50,045][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>"%{id}", :\_index=\>"lee-index", :\_type=\>"doc", :\_routing=\>nil}, #LogStash::Event:0x2b149481], :response=\>{"index"=\>{"\_index"=\>"lee-index", "\_type"=\>"doc", "\_id"=\>"%{id}", "status"=\>400, "error"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"Rejecting mapping update to [lee-index] as the final mapping would have more than 1 type: [fulltext, doc]"}}}} It seems there is mapping error in logstash. Here are my index setting : > PUT /lee-index > PUT /lee-index/fulltext/\_mapping > { > > ``` > "properties": { > > "content": { > > "type": "text", > > "analyzer": "ik_max_word", > > "search_analyzer": "ik_max_word" > > } > } > > ``` > > } Why there is an fulltext type? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 1, 2018, 8:02am UTC](https://discuss.elastic.co/t/logstash-has-fetched-the-data-but-not-shipped-into-elasticsearch/134154/2 "2018-06-01T08:02:35Z")
Have you looked in the Logstash log for clues? What if you turn up the loglevel? ---
**Author:** ![Lee\_Jack](https://avatars.discourse-cdn.com/v4/letter/l/839c29/32.png) [@Lee\_Jack](https://discuss.elastic.co/u/Lee_Jack)\ **Post date:** [June 1, 2018, 8:35am UTC](https://discuss.elastic.co/t/logstash-has-fetched-the-data-but-not-shipped-into-elasticsearch/134154/3 "2018-06-01T08:35:17Z")
It seems I have find the reason , but why there is both doc type and fulltext type ? [2018-06-01T16:30:01,623][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>"5", :\_index=\>"lee-index", :\_type=\>"doc", :\_routing=\>nil}, #LogStash::Event:0x17f26092], :response=\>{"index"=\>{"\_index"=\>"lee-index", "\_type"=\>"doc", "\_id"=\>"5", "status"=\>400, "error"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"Rejecting mapping update to [lee-index] as the final mapping would have more than 1 type: [fulltext, doc]"}}}} ---
**Author:** ![Lee\_Jack](https://avatars.discourse-cdn.com/v4/letter/l/839c29/32.png) [@Lee\_Jack](https://discuss.elastic.co/u/Lee_Jack)\ **Post date:** [June 1, 2018, 8:43am UTC](https://discuss.elastic.co/t/logstash-has-fetched-the-data-but-not-shipped-into-elasticsearch/134154/4 "2018-06-01T08:43:17Z")
I have solved my problem .Thanks for you advice ! ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 1, 2018, 8:44am UTC](https://discuss.elastic.co/t/logstash-has-fetched-the-data-but-not-shipped-into-elasticsearch/134154/5 "2018-06-01T08:44:10Z")
> It seems I have find the reason , but why there is both doc type and fulltext type ? Hard to tell. There's nothing in your configuration that adds a "fulltext" type so I don't know where it comes from. ---
**Author:** ![Lee\_Jack](https://avatars.discourse-cdn.com/v4/letter/l/839c29/32.png) [@Lee\_Jack](https://discuss.elastic.co/u/Lee_Jack)\ **Post date:** [June 1, 2018, 8:45am UTC](https://discuss.elastic.co/t/logstash-has-fetched-the-data-but-not-shipped-into-elasticsearch/134154/6 "2018-06-01T08:45:31Z")
After changing the index type into doc ，the data can be shipped correctly. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 29, 2018, 8:45am UTC](https://discuss.elastic.co/t/logstash-has-fetched-the-data-but-not-shipped-into-elasticsearch/134154/7 "2018-06-29T08:45:41Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.