DavidStables (talk | contribs) |
DavidStables (talk | contribs) |
||
| Line 182: | Line 182: | ||
Queries and updates are initiated by a Query Request passed as a payload to the API. | Queries and updates are initiated by a Query Request passed as a payload to the API. | ||
A query request can contain a set of arguments or parameter variables passed into the query to be used at tun time. | A query request can contain a set of arguments or parameter variables passed into the query to be used at tun time. | ||
Revision as of 17:49, 10 March 2023
Background to IMQ
Its all very well modelling data, value sets, and ontologies. What about modelling the logical definitions of data sets or profiles, this being usually referred to as query?
It is common practice in health care to model query definitions in plain text documents, which leave the interpretation of the logic to the vendors internal informatics team. This process creates a bottle neck, and is prone to human error. An approach which uses machine readable definitions can reduce the time and remove much of the human error i.e. transferring the risk of error from a human to a computer.
Health data is generally held in relational, document, or graph databases, each of which have different schemas and each come with a high level query language used to extract or create data. The best known of these languages is SQL, with other languages such as CYPHER, SPARQL, ELASTIC DSL, less well known. Each of these languages are designed "close to the tin" so that the query engines can perform highly optimised query.
It is possible to model a query definition in SQL, but an SQL query brings with it the specific database schema. Also, using SQL as a means of modelling query brings the risk of poorly performing SQL, as SQL performs differently at different scales. For example a correlated subquery on a huge database may perform poorly and sometimes better represented as a Window function. One systems good SQL is another systems bad SQL.
Is is very difficult to construct understandable user interfaces directly from SQL or SPARQL, or vice versa, and thus most search and report applications create some form of intermediate representation. The principle behind IMQ is thus already established and IMQ is therefore "yet another DSL".
IM Query is a health query domain specific approach, designed to operate as an intermediary between plain language and the underlying run time query languages. The grammar is a constraint of main steam query languages and is a constraint on the query logic itself designed to match most health queries used in practice
IMQ overview
IMQ can be considered as a DSL (a language with Grammar like GraphQL or SQL), or an Object model DSL such as Elastic. Both forms are supported. IMQ provides a pragmatic JSON based object model and a more succinct text to represent query definitions, and the IM services provide implementations in SQL and SPARQL via the use of interpreters.
The grammar or "structure" of an IMQ query definition precisely follows the logic of a plain language description of the the criteria to be applied to filter out sets from sets, and define the output required, and collectively this is a data set definition?
The structure of a query definition closely follows the standard patterns as illustrated in the main stream languages of SQL, SPARQL and CYPHER with object nesting supported to enable hierarchical object retrieval, akin to GRAPHQL.
Implementation syntax includes a JSON object format (IMQ-JSON) and a succinct text language equivalent (IMQ-text)
An IM query definition follows a straightforward set of predicates which can be summarised as:
"From a set of things, where those things have certain properties and values, Select the properties of those things, where the properties have certain values."
i.e. a familiar approach of from/where/select/where clauses, each type of clause being made up of the necessary predicates to enable most rule based health queries to be modelled.
In line with the rest of the IM languages, IMQ uses an RDF approach to identifiers, thus enabling global identifiers for types, properties and value sets.
IMQ Structure
An IM query consists of a query request, which includes the necessary components to define a query, as well as a set of arguments that can be passed into the query and used at run time.
Query
A simple overall structure with nestable elements providing an object form input and output similar to GRAPHQL . A query may contain many queries, enabling a package of queries such as a column group report or full data set .
The request may fully define the query (dynamic query) or more commonly reference a pre-existing query definition via an IRI (i.e. a preformed query definition with variables resolved to the arguments passed in at run time). The pre-existing query definition is obtained from the "has Definition" property of a stored query entity.
High level query structure is as follows: (Select Expand to show)
| Plain | Json |
|---|---|
query {
from {
where {
with {
then {}
}
}
}
select {
where {}
select {}
}
}
|
{"query" : {"from" : {
"where" :[ {
"with" : {
"then" : {}
}
]}
},
"select" :[{
"where" : {},
"select" :[ {}]
}],
"subQuery" :[ {}]
}
|
Simple example
Get me the full name of all patients aged >= 18 years. Select Expand to show:
| Plain | Json |
|---|---|
query {
from {
@:Patient
where {
:age >=18 units: years
}
}
select { :fullName}
}
|
{
"from" : {"@type": ":Patient",
"where" :{"id" : ":age",
"operator" : ">=",
"value" : 18,
"unit" : "YEARS"}
},
"select" : [ {"id" : ":fullName"}]
}
|
Clause specification
Specification of query clauses are described in the article IMQueryClauses
Subsumption query
A key differentiator of IMQ is the support for a variety of subsumption patterns in both the from and where clause. This makes IMQ compliant with expression constraint language when applied to concepts, but can also be used to incorporate subtypes of data model types.
The indicators are:
- Include Subtype ? true or false as to whether subtypes (or subclasses) are incorporated at run time. The can apply either in the from clause, the where property, or the value.
- Exclude self? true or false, rarely used but indicates only subtypes are examined (ECL compliance)
- Include super types? True or false to enable the parent hierarchy to be transitively examined. Used in assessing allowable ranges and properties of concepts.
Example subsumption on the IM itself Json and ECL equivalent. Select Expand to show:
| IMQ | ECL |
|---|---|
{
"from": {
"@id": "sn:763158003","name": "Medicinal product (product)","includeSubtypes" : true,
"where": {
"bool": "and",
"where": [
{
"@id": "sn:127489000","name": "Has active ingredient (attribute)", "includeSubtypes": true,
"in": [
{
"@id": "sn:372665008","name": "Non-steroidal anti-inflammatory agent","includeSubtypes" : true },
"anyRoleGroup": true
},
{
"@id": "sn:411116001","name": "Has manufactured dose form (attribute)","includeSubtypes": true,
"in": [
{"@id": "sn:385268001","name": "Oral dose form (dose form)","includeSubtypes": true}
],
"anyRoleGroup": true}]}}}
|
<<763158003| Medicinal product (product)|
: <<127489000 | Has active ingredient (attribute)| =
<<372665008| Non-steroidal anti-inflammatory agent|,
<< 41116001|Has manufactured dose form (attribute)| =
<<385268001|Oral dose form (dose form)| |
Query Request
IMQ supports conventional query for extract, query based updates (deletion) and a special 'path query' for determining paths between two classes. In addition to rule based query, a free text search using Lucene indexing is supported providing a term filter on the query rules.
Queries and updates are initiated by a Query Request passed as a payload to the API.
A query request can contain a set of arguments or parameter variables passed into the query to be used at tun time.
This is the section on grammar