How to Map Kibana's KQL to URL and Elasticsearch Query
How do you map a query in Kibana, like languages : "English" to the Kibana URL and the underlying Elasticsearch query? This mapping comes in handy when you want to create links to specific Kibana queries, debug what is happening in the background, or understand what is (not) easy to connect or reuse — starting from a question on Twitter.
Mapping a Kibana Query #
The default query language in Kibana has been the Kibana Query Language (KQL) for a while now. Once you run a KQL query like languages : "English", Kibana’s URL automatically changes, so you can copy and share it right away; or adjust the query straight from the URL. The main parts of the URL are:
https://kibana-demo.europe-west3.gcp.cloud.es.io:9243/orhttp://localhost:5601: The base URL of your Kibana instance.app/discover#/?: The application within Kibana, likediscover,dashboards,observability,security,…_g=(filters:!(),query:(language:kuery,query:''),refreshInterval:(pause:!t,value:0),time:(from:'2020-04-30T22:00:00.000Z',to:now))&: The “global state”, which moves between apps with you such as pinned filters; here the end of April 2020 until now._a=(columns:!(),filters:!(),index:c259d110-b169-11ea-85e2-456c6b200223,interval:auto,query:(language:kuery,query:'languages%20:%20%22English%22%20'),sort:!(!('@timestamp',desc))): The “app state”, which keeps the KQL query, sort order (not part of KQL), structured filters on top of the KQL query, the index pattern to query,…
To make more sense of it, you can use the Kibana URL parser to pick apart the RISON-encoded URL. And a word of caution: While the URL structure has been stable for years, it might change in the future.
While the mapping between KQL and URL is straightforward, it isn’t for the underlying Elasticsearch query. Most pages in Kibana have an Inspect button, which can reveal statistics, request, and response against Elasticsearch for this view in Kibana. This query isn’t using KQL but the Elasticsearch query DSL instead.
The query flow looks roughly like this:
- KQL is converted into the Elasticsearch query DSL in the browser already, so only the client-side / React part needs to know about KQL.
- From React, you send a query to Kibana’s server (Node.js).
- Node.js forwards the query to Elasticsearch.
- The response from Elasticsearch gets back to Node.js.
- And is then turned into the final response for rendering in the browser.
Sidenote on Query Languages in the Elastic Stack #
Why are there multiple query languages in the Elastic Stack?
For historical reasons: The Lucene query syntax or query_string query was widely known and straightforward to expose when Elasticsearch started. Thanks to its compact nature, it also made more sense for Kibana than the Elasticsearch query DSL, since you don’t want to type all the curly braces. For that reason, the Lucene query syntax was the default option in Kibana for many years, exposing advanced Lucene features but missing some Elasticsearch features like nested objects or scripted fields.
For specific requirements: To fill these gaps and add autocomplete to Kibana, which didn’t make much sense for the API in Elasticsearch or Lucene. Rather than trying to shoehorn additional features into the Lucene query syntax, a compact and autocompleting query language was born — initially called Kuery and then renamed to KQL. KQL can suggest field names, values, and operators as you type, which was the main reason for developing it.
There are similar stories for the addition of SQL, EQL, why Canvas uses (or at least used) SQL so heavily,…
Conclusion #
Mapping the main elements of this article in a single screenshot:
- KQL to search in the selected index pattern.
- Directly mapping to the URL is also straightforward to create manually.
- The underlying Elasticsearch query runs through a more complex transformation.
PS: Thanks to my colleague Matthias Wilhelm for providing feedback.