It turned out to be directly output to elasticsearch, I can see the information in apm ui, but after outputting to kafka, what data seems to be lost, now apm ui is empty, is there any way to restore the data of apm ui?
I have tried to import the template of apmserver before entering es in logstash, but it has no effect
You can have APM Server output to Kafka and then Logstash to pull the events out of Kafka and index them in Elasticsearch. You should see events in the UI without problem, but with the information you give I can't pinpoint the problem.
One of the things you need to do first is to point APM Server to Elasticsearch and run apm-server -e setup to save the correct index template.
With the info you provide I can not say what is the issue. It seems there are some typos in your config: it is output.elasticsearch.enabled, not output.elasticsearch.enable. The same for output.kafka.
Indentation is lost in the copy paste, but it is also important that yml files are indented correctly.
Generally the right approach would be go start from the beginning step by step, with the simplest configurations possible, eg:
Check that APM Server can connect to Elasticsearch: /apm-server test output -e -E 'output.elasticsearch.hosts=["http://localhost:9200"]' -E output.elasticsearch.username=admin -E output.elasticsearch.password=changeme
Setup templates with the same arguments: ./apm-server setup -e -E 'output.elasticsearch.hosts=["http://localhost:9200"]' -E output.elasticsearch.username=admin -E output.elasticsearch.password=changeme
Check that templates exists: curl -u admin:changeme http://localhost:9200/_cat/templates/apm\*
Now check connection to Kafka: ./apm-server test output -e -E output.elasticsearch.enabled=false -E output.kafka.enabled=true -E 'output.kafka.hosts=["localhost:9092"]' -E output.kafka.topic=apm
Keep going with minimal config and ingesting data. You can query Elasticsearch directly to see if documents have been indexed: curl -u admin:changeme 'http://localhost:9200/apm*/_search'.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.