Security operations teams often maintain repositories of threat intelligence reports that contain a wealth of knowledge from the vendor producing the report. The challenge, however, is that the content of these reports typically sits in PDFs, making it difficult to retrieve and reference relevant information from the report during an incident or investigation or leverage any indicators of compromise (IoCs) for threat hunting. With the ability to use these reports as knowledge within the Elastic AI Assistant, this dynamic changes entirely.
Let’s use the Elastic Global Threat Report for 2024 as an example.
Step 1. Enabling and setting up the knowledge base
This is a very simple step that takes care of some of the prerequisites necessary for the knowledge base content to be used by Elastic AI Assistant. It’s a single button in the assistant management settings. The process only takes a few minutes to complete.
Step 2. Uploading the PDF
Once the knowledge base setup is complete, we can proceed to upload the PDF. To do this, we can use the integration titled Upload a file from the Integrations page.
You can select the PDF from the next screen.
Click Import when prompted.
For the next step, we will need to pivot to the Advanced tab. Once uploaded, this PDF will live in its own index, so feel free to name the index accordingly. There is no need to create a data view.
There is one last step before clicking on the import button. We need to add a semantic text field. This allows the assistant to retrieve the correct information from the report.
Click on Add additional field and then Add semantic text field.
You can leave the default settings that appear after clicking Add semantic text field.
You can now click on Import.
When the file is imported successfully, you should see the following status:
It’s important to note that while we used the File Upload user interface to add this PDF, it’s possible to automate this functionality as part of any ingest process using the attachment processor.
Step 3. Adding the PDF index as custom knowledge
Returning to the AI Settings page, select New to add a new knowledge entry, and then select Index from the list.
You’ll then be asked to select the index that was just created (“global-threat-report-kb” in our example), the semantic text field we just created (content), and a description of how and when the assistant should use this knowledge. This should be a simple sentence description of what the data is and when and how it should be queried. You can also set the relevant permissions for this knowledge entry from this view. When ready, hit Save.
Once added, you should see the new knowledge entry in the list:
The threat report is now available as knowledge and is ready to be used by the assistant.
Comparing the results
If we compare results from the assistant before and after we add the knowledge base entry, we can see a clear difference.
Before the knowledge was added:
After the knowledge was added:
Our PDF went from being an idle bit of important — yet hard-to-use — information to being immediately accessible to our security operations team. The great thing about knowledge sources is that the Elastic AI Assistant is able to use a combination of them, depending on the questions asked. Remember that the Elastic AI Assistant can also ingest 500 of your latest alerts as knowledge by default, which allows for a powerful combination of questions that can be asked.
This one example clearly highlights the usefulness of having custom knowledge sources available to the assistant. And as we highlighted earlier, there are many other scenarios and examples of where custom knowledge sources can be useful.
For more information on how to add different types of knowledge sources, you can refer to our detailed documentation.