IMQGrammar: Difference between revisions

From Endeavour Knowledge Base
 
(33 intermediate revisions by the same user not shown)
Line 1: Line 1:
__TOC__
__TOC__


These pages describe the IM Query grammar conventions and meanding. The formal IMQ Grammar can be accessed in ABNF form at https://github.com/endeavourhealth-discovery/IMDirectory/blob/develop/im_library/src/antlr4/grammars/ECL.g4
These pages describe the IM Query grammar conventions and meaning.  
 
A formal IMQ Grammar can be accessed in ABNF form at https://github.com/endeavourhealth-discovery/IMDirectory/blob/develop/im_library/src/antlr4/grammars/ECL.g4
 
The following is used as a working example throughout the pages:<syntaxhighlight lang="actionscript3">
Find:
Patients
Registered for GMS
Aged 65 to 70 or Diabetic
Latest systolic BP within the last 6 months is >150
Not followed by a screening invite
Not already hypertensive
</syntaxhighlight>


=== Symbol conventions ===
=== Symbol conventions ===
A small set of symbols are used to represent meaningful language tokens. These are:
A small set of symbols are used to represent meaningful language tokens. These are:
 
{| class="wikitable"
* '{' and  '}' Used for clauses to differentiate the content from other similar clauses, as well as to indicate graph traversal. It is used in the same way as { } is used in Json to represent and object. For example 'where {age >75 units : years} encloses a where clause
|+
* ',' comma used to indicate a list entry number 2 or more. the language does not require brackets to show a list. For example
!Symbol
where { :concept in sct:29857009,sct:25064002 } means that the in predicate is looking at a list of two concepts
!Meaning
* '@' used as an identifier qualifier to indicate a type. @im:Concept indicates that this refers to instances of type im:concept .For example:
!Example
from { @:Patient} means that the query is looking for all instances of type patient.
|-
* '^' used as an identifier qualifier to indicate a set. For example:
|'{'   '}'  
from { ^ex:RegisteredPatients}  means the set of registered patients, used mostly in referencing base populations or value /reference sets
|Used for clauses to differentiate the content from other similar clauses, as well as to indicate graph traversal. It is used in the same way as { } is used in Json to represent and object.
* << subsumption qualifier used to indicate descendants or Self of against an identifier for example
|where {age >=65 to <75 units : years}
where { :concept in <<sct:29857009}  means the Snomed-CT concept identifier of 29857009 and any of its subtypes.
|-
* '<' used to indicate 'descendants of but not including self' against an identifier. For example
|','
where { :concept in <sct:29857009} would mean its subtypes but not itself
|comma used to indicate a list entry number 2 or more. the language does not require brackets to show a list.  
* '>>' used to indicate 'Ancestors of including self; against an identifier . For example
|<nowiki>where { :concept in ^:VSET_Diabetes, <<:sct:714628002|Prediabetes } means that the concept is either a member of the Diabetes value set or is prediabetes or subtype of it </nowiki>
where { :concept in >>sct:29857009} would mean all concepts that are ancestors of chest pain
|-
* '|' '|' used as a short cut to indicate the name of a concept or entity sct:29857009|Chest pain |
|| ' ( )'
| used for method arguments such as units of measure
| >18(years) to indicate that the number is qualified by a unit
|-
|'@'
|used as an identifier qualifier to indicate a type.  
|im:Concept  
indicates that this refers to instances of type im:concept  
from { @:Patient}
means that the query is looking for all instances of type patient.
|-
|<nowiki>''^'</nowiki>
|used as an identifier qualifier to indicate a set or population elsewhere defined
|rom { ^ex:RegisteredPatients}  means the set of registered patients, used mostly in referencing base populations or value /reference sets
|-
|'<<'
|subsumption qualifier used to indicate descendants or Self of against an identifier
|<nowiki><<:sct:714628002|Prediabetes|</nowiki>
means prediabetes or any of its descendants
|-
|'<'
|used to indicate 'descendants of but not including self'
|<nowiki><:sct:714628002|Prediabetes|</nowiki>
means  any of its descendants but not prediabetes itself * (rarely used)
|-
|'>>'
|ed to indicate 'Ancestors of including self; against an identifier .  
|<nowiki>from {>>sct:127489000| has active ingredients</nowiki>
means the property active ingredients or any of its super properties. Used to validate attributes in the domain range Snomed concept model.
|-
|<nowiki>''|' '|' </nowiki>
|used as a short cut to indicate the name of a concept or
|<nowiki>:sct:714628002|Prediabetes|</nowiki>
|}


=== Identifiers ===
=== Identifiers ===
Line 27: Line 72:


===Boolean operators===
===Boolean operators===
The boolean operators 'and' , 'or' and 'not' can operate in either the from or where clause, using a "bool" predicate. 'and' is the same as an intersection (all must be true), or is a union (at least one true) and 'not' is a union of negation (any one must be false). Further nesting can resolve boolean combinations (e.g. not/and)
The boolean operators 'and' , 'or' can operate in either the from or where clause, using a "bool" predicate. 'and' is the same as an intersection (all must be true), or is a union (at least one true).
Example boolean operators:
 
Exclude is the negation operator that negates a clause and all of its sub clauses.
 
Example use of boolean operators in where clause:


{| class="wikitable"
{| class="wikitable"
Line 35: Line 83:
!Json
!Json
|-
|-
|where{
|where {
     {: age>- 18 units: years}
     {:age>= 18 (years) }
     and  
     and  
     {:statedGender in :905031000252103|Male|}
     {:statedGender is :905031000252103|Male|}
    }


|<syntaxhighlight lang="json">
|<syntaxhighlight lang="json">
{"where" : {
{"where" : {
       "bool" : "and",
       "bool" : "and",
       "where" : [ { "id" : "age",
       "where" : [ { "@id" : ":age",
                     "operator" : ">=",
                     "operator" : ">=",
                     "value" : 18,
                     "value" : 18,
                     "unit" : "year"},
                     "unit" : "yeara"},
                     {"id" : "statedGender",
                     {"@id" : ":statedGender",
                     "in" : [ {"@id" : "im:905031000252103",
                     "in" : [ {"@id" : "im:905031000252103",
                               "name" : "Male (stated gender)" } ] } ] }       
                               "name" : "Male (stated gender)" } ] } ] }       
</syntaxhighlight>
</syntaxhighlight>
}


|-
|}
=== From Clause ===
The from clause means "get me a instances of....", returning a set of identifiers of object instances.
The set of instances are either instances of a type , or members of a set, or an instance identifier.
It has similar meaning to 'Select ID From TABLE ' in SQL but with the added convenience of simply indicating the source.<syntaxhighlight lang="actionscript3">
fromClause: 'from' from| booleanFrom ;
from:  { exclude? description (reference | fromClause |whereClause};
booleanFrom: from (and|or) from+;
</syntaxhighlight>A from clause consists of a single from or boolean from, a single from consisting  optional exclusion, description and has either a reference, a from clause, or where clause.
Examples of  use of from:
{| class="wikitable"
|+
!Plain
!Json
|-
|from { @:Patient }
|<syntaxhighlight lang="json">
{"from" : {
      "@type" : ":Patient"}
</syntaxhighlight>
| get me instances of type patient
|-
|from {^myQuery:PatientsWithdiabetes}
|<syntaxhighlight lang="json">
{"from" : {
      "@set" : "muQuery:PatientsWithDiabetes"}
</syntaxhighlight>
|get me the the instances from the query (or set of) Patients with diabetes
|-
|<nowiki> from { <<sct:763158003| Medicinal product (product) |} </nowiki>
|<syntaxhighlight lang="json">
{"from" : {
      "@id" : "sct:763158003",
      "name" : "Medicinal product (product)",
      "descendantsOrSelfOf" : true}
</syntaxhighlight>
</syntaxhighlight>
|get me medicinal product concepts or subtypes of
|-
|-
| <nowiki> from { @:Patient
                }
                and { ^im:PatientsRegisteredForGMS
                }
</nowiki>
|<syntaxhighlight lang="json">
{"from" : {
      "boolFrom" : "and",
        "from" : [
                  {"@id" : ":Patient"
                  },
                  { "@set" : "im:PatientsRegsteredForGMS"
                  }
                ]
      }
}
</syntaxhighlight>
| get me things that are both patients and gms registered  (note that as the gms registered ARE a set of  patients, the patient type filter is not needed)
       
|}
|}
=== Where clause ===
The where clause navigates the graph and filters properties and values from the set of instances identified in the from clause or a further filter on properties  from a select clause.
In effect it combines JOIN and WHERE and UNION  from SQL with the additional ordering and limits from a subquery  (SELECT)  clause, which is ideal for representing "latest of" type clauses.
Because the where clause supports ordering (in effect turning it into a subquery), a convenient THEN clause (itself a where clause) can be used to further test the results of a filtered, ordered, and limited set of records. This enables a more intuitive rule based approach than the equivalent correlated subquery that is needed in SQL e.g. get me patients whose  latest blood pressure within the last 6 months is >150.
Like SPARQL, the where clause can follow a property path and with nested boolean logic, enables highly granular filtering at leaf level without the need to bind aliases as in SQL e,g.  where  { home address  where { post code starts with "YO18" } }

Latest revision as of 13:07, 20 March 2023

These pages describe the IM Query grammar conventions and meaning.

A formal IMQ Grammar can be accessed in ABNF form at https://github.com/endeavourhealth-discovery/IMDirectory/blob/develop/im_library/src/antlr4/grammars/ECL.g4

The following is used as a working example throughout the pages:

Find: 
Patients
Registered for GMS
Aged 65 to 70 or Diabetic
Latest systolic BP within the last 6 months is >150
Not followed by a screening invite
Not already hypertensive

Symbol conventions

A small set of symbols are used to represent meaningful language tokens. These are:

Symbol Meaning Example
'{' '}' Used for clauses to differentiate the content from other similar clauses, as well as to indicate graph traversal. It is used in the same way as { } is used in Json to represent and object. where {age >=65 to <75 units : years}
',' comma used to indicate a list entry number 2 or more. the language does not require brackets to show a list. where { :concept in ^:VSET_Diabetes, <<:sct:714628002|Prediabetes } means that the concept is either a member of the Diabetes value set or is prediabetes or subtype of it
' ( )' used for method arguments such as units of measure >18(years) to indicate that the number is qualified by a unit
'@' used as an identifier qualifier to indicate a type. im:Concept

indicates that this refers to instances of type im:concept from { @:Patient} means that the query is looking for all instances of type patient.

''^' used as an identifier qualifier to indicate a set or population elsewhere defined rom { ^ex:RegisteredPatients} means the set of registered patients, used mostly in referencing base populations or value /reference sets
'<<' subsumption qualifier used to indicate descendants or Self of against an identifier <<:sct:714628002|Prediabetes|

means prediabetes or any of its descendants

'<' used to indicate 'descendants of but not including self' <:sct:714628002|Prediabetes|

means any of its descendants but not prediabetes itself * (rarely used)

'>>' ed to indicate 'Ancestors of including self; against an identifier . from {>>sct:127489000| has active ingredients

means the property active ingredients or any of its super properties. Used to validate attributes in the domain range Snomed concept model.

''|' '|' used as a short cut to indicate the name of a concept or :sct:714628002|Prediabetes|

Identifiers

Query definitions include reference to a data model types, properties, concepts or sets. Identifiers are represented as RDF IRIs in both the full or prefixed form. For example:

http://snomed.info/sct# and sct: 29857009 may be equivalent where the query contains a prefix list

Boolean operators

The boolean operators 'and' , 'or' can operate in either the from or where clause, using a "bool" predicate. 'and' is the same as an intersection (all must be true), or is a union (at least one true).

Exclude is the negation operator that negates a clause and all of its sub clauses.

Example use of boolean operators in where clause:

Plain Json
where {
    {:age>= 18 (years) }
    and 
    {:statedGender is :905031000252103|Male|}
   }
{"where" : {
       "bool" : "and",
       "where" : [ { "@id" : ":age",
                     "operator" : ">=",
                     "value" : 18,
                     "unit" : "yeara"},
                    {"@id" : ":statedGender",
                     "in" : [ {"@id" : "im:905031000252103",
                               "name" : "Male (stated gender)" } ] } ] }

From Clause

The from clause means "get me a instances of....", returning a set of identifiers of object instances.

The set of instances are either instances of a type , or members of a set, or an instance identifier.

It has similar meaning to 'Select ID From TABLE ' in SQL but with the added convenience of simply indicating the source.

fromClause: 'from' from| booleanFrom ;
from:  { exclude? description (reference | fromClause |whereClause};
booleanFrom: from (and|or) from+;

A from clause consists of a single from or boolean from, a single from consisting optional exclusion, description and has either a reference, a from clause, or where clause.

Examples of use of from:

Plain Json
from { @:Patient }
{"from" : {
       "@type" : ":Patient"}
}
get me instances of type patient
from {^myQuery:PatientsWithdiabetes}
{"from" : {
       "@set" : "muQuery:PatientsWithDiabetes"}
}
get me the the instances from the query (or set of) Patients with diabetes
from { <<sct:763158003| Medicinal product (product) |}
 {"from" : {
       "@id" : "sct:763158003",
       "name" : "Medicinal product (product)",
       "descendantsOrSelfOf" : true}
 }
get me medicinal product concepts or subtypes of
from { @:Patient } and { ^im:PatientsRegisteredForGMS }
 {"from" : {
       "boolFrom" : "and",
        "from" : [
                  {"@id" : ":Patient"
                  },
                  { "@set" : "im:PatientsRegsteredForGMS"
                  }
                 ]
       }
}
get me things that are both patients and gms registered (note that as the gms registered ARE a set of patients, the patient type filter is not needed)

Where clause

The where clause navigates the graph and filters properties and values from the set of instances identified in the from clause or a further filter on properties from a select clause.

In effect it combines JOIN and WHERE and UNION from SQL with the additional ordering and limits from a subquery (SELECT) clause, which is ideal for representing "latest of" type clauses.

Because the where clause supports ordering (in effect turning it into a subquery), a convenient THEN clause (itself a where clause) can be used to further test the results of a filtered, ordered, and limited set of records. This enables a more intuitive rule based approach than the equivalent correlated subquery that is needed in SQL e.g. get me patients whose latest blood pressure within the last 6 months is >150.

Like SPARQL, the where clause can follow a property path and with nested boolean logic, enables highly granular filtering at leaf level without the need to bind aliases as in SQL e,g. where { home address where { post code starts with "YO18" } }