No results found for View transaction in Discover

rcoundon · September 16, 2018, 11:54am

If I load a transaction in APM and then select to View Transaction in Discover, no results are returned.

The IDs are copied over to the search bar successfully:

processor.event:transaction AND transaction.id:b2355f9f-f8ef-45e7-9d44-a24218e0b09b

but there are no results returned. Is there anything special I need to do to enable this feature?

axw · September 17, 2018, 1:25am

Hello, and thanks for trying out Elastic APM!

How recent is the transaction? What is the time range set to (top-right corner of Kibana)? It may be that the UI is not setting the time range appropriately for older transactions. It should just work, but it's possible that the time range isn't being properly set by the APM UI.

rcoundon · September 17, 2018, 6:32am

Thanks for getting back to me.

You're right, the time range doesn't get carried through to Discover, nor does the index. Setting the index to apm* (Kibana has two index patterns of apm* after loading them through the GUI) and a matching time range does show the transaction.

felixbarny · September 17, 2018, 7:05am

That sounds a bit strange. Could you post screenshots how they are set up? Which one works and which one does not?

rcoundon · September 17, 2018, 7:36am

Hi - they both return data but they are slightly different - one has @timestamp formatted as Date and one doesn't:

rcoundon · September 17, 2018, 7:43am

I've just refreshed the field list for each and now I'm getting the message:

There's no APM index pattern available. To use the Query bar, please choose to import the APM index pattern in the [Setup Instructions.]

From another thread in here I'd previously set this value in the kibana.yml

apm_oss.indexPattern: 'apm*'

which caused that message to go away, but now it's back and that line is still in kibana.yml

makwarth · September 17, 2018, 8:37am

Hi @rcoundon,

it sounds like the APM UI is still looking for the apm-* index pattern (with -), which is the default. Did you restart Kibana after updating kibana.yml with apm_oss.indexPattern: 'apm*'?

Did you import the index pattern via the APM setup started instructions screen, or via the Index management UI?

best,
Rasmus

rcoundon · September 17, 2018, 1:02pm

Hi - Yes, it was restarted - I used the set instructions screen. I've tried restarting again but still see the same message.

sqren · September 17, 2018, 1:34pm

Hi rcoundon,

To shed some more light on your current APM index patterns, can you please run the following in Kibana Dev Tools, and paste the output here (baring sensitive data if any):

GET .kibana/_search
{
  "size": 10,
  "_source": ["type", "updated_at", "index-pattern.title"], 
  "query": {
    "prefix" : { "index-pattern.title" : "apm" }
  }
}

If you import the index patterns via the APM getting started instructions screen, you should receive a warning about overwriting objects. Do you get that? Also, if you don't mind overwriting your APM dashboards and visualisations, please proceed and overwrite them.
I expect this will fix the error with missing index patterns.
Let me know if it doesn't.

rcoundon · September 17, 2018, 1:54pm

Yes - I've already tried reloading and accepting the overwrite but the message persists. I have a dashboard that contains some APM objects, should I go and delete that and try again?

sqren · September 17, 2018, 2:39pm

Yes, you could try deleting the APM objects (visualisations and dashboards), and the APM index patterns via /app/kibana#/management/kibana/indices.

Now you should no longer get an error, when you import the objects via Getting Started instructions (/app/kibana#/home/tutorial/apm).

I'm still interested to see the output of the above query.

rcoundon · September 17, 2018, 5:59pm

Apologies - missed your request for the query output.

 {
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 1,
    "hits": [
      {
        "_index": ".kibana",
        "_type": "doc",
        "_id": "index-pattern:apm-*",
        "_score": 1,
        "_source": {
          "index-pattern": {
            "title": "apm*"
          },
          "updated_at": "2018-09-17T07:36:59.321Z",
          "type": "index-pattern"
        }
      },
      {
        "_index": ".kibana",
        "_type": "doc",
        "_id": "index-pattern:bec35920-64e2-11e8-a49b-cf834cb72410",
        "_score": 1,
        "_source": {
          "index-pattern": {
            "title": "apm*"
          },
          "updated_at": "2018-09-17T07:37:06.312Z",
          "type": "index-pattern"
        }
      }
    ]
  }
}

rcoundon · September 17, 2018, 6:03pm

I've now deleted and reloaded the Kibana APM objects. I still get the message on the APM screen that there's no APM index pattern however I do only have one index pattern listed. The result of that query is now:

{
  "took": 5,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 1,
    "hits": [
      {
        "_index": ".kibana",
        "_type": "doc",
        "_id": "index-pattern:apm-*",
        "_score": 1,
        "_source": {
          "index-pattern": {
            "title": "apm*"
          },
          "updated_at": "2018-09-17T18:01:34.456Z",
          "type": "index-pattern"
        }
      }
    ]
  }
}

sqren · September 17, 2018, 7:44pm

Thanks for posting the query result. Very useful.

Now I'm interested in knowing whether Kibana is correctly picking up your changes to kibana.yml. There is no direct way to figure this out, so it'll be slightly messy. I hope you'll indulge me

After opening the APM tab in Kibana, please click "View source" in your browser and search for "apmIndexPattern" - it should be hiding inside a blob of urlencoded data. I'm interested in the first 60ish characters after apmIndexPattern like:

"apmIndexPattern":"apm*",&quot"

rcoundon · September 17, 2018, 7:54pm

Happy to help! I've removed the IP address, hopefully this gives you what you need

apmIndexPattern&quot;:&quot;apm*&quot;,&quot;isCloudEnabled&quot;:false,&quot;elasticsearchUrl&quot;:&quot;http://[REDACTED]:9200&quot;,&quot;esRequestTimeout&quot;:30000,&quot;esShardTimeout&quot;:30000,&quot;esApiVersion&quot;:&quot;master&quot;,&quot;esDataIsTribe&quot;:false,&quot;kbnIndex&quot;:&quot;.kibana&quot;,&quot;kbnBaseUrl&quot;:&quot;/app/kibana&quot;,&quot;vegaConfig&quot;:

sqren · September 19, 2018, 8:45am

Hi rcoundon,

From the encoded blob, I can see that your config change works, and apm* is the index pattern title we are looking for. I can also see that you index pattern is correctly created with a title of apm*. Therefore it should work as expected.

Since you are still seeing an error message, there must be an error elsewhere, and I'll have to spent some more time looking into this before I can give you a good answer. I don't have a timeline right now for when that'll be.

rcoundon · September 19, 2018, 8:58am

Hi - no problem, let me know if you need anything more from me to assist.

system · October 10, 2018, 4:58am

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.