Welcome to the Endeavour Health knowledge base: Difference between revisions

From Endeavour Knowledge Base
(45 intermediate revisions by 2 users not shown)
Line 1: Line 1:


__TOC__
__TOC__
These articles describe the Discovery Data Service; the architecture that underpins the data service, the components that make up the service, the testing and assurance processes, and the resources, open source technologies, and software that are used to host, develop, and support the service.


==Discovery data service status==
These articles describe the various technologies underpinning the Endeavour open source technologies which have been adopted and used by a large scale NHS  London Data linkage and normalisation Service covering a population of 7 million registered citizens.
The data service is live.
==Health Record Data Service and Health Information model==
The health record data service is based on the hypothesis that; If health related data is brought together at the level of the individual in close to real time, and stored together at the level of a medium size residential population, and made available via a common data model as part of a common ontology, and then subsequently used for individual and population based decision support, great benefits to health can accrue.


It currently stores and processes the data from 28 million linked organisational patient records in the uk.
The service uses technologies that consist of a set of components that link near real time data from millions of records from thousands of provider organisations, with scores of different formats and code systems and converts them to a single common information model, made up of a super ontology, a data model, ,maps between source data and the common model, sets of concepts sets and value sets for query, and query libraries.  


It contains long term records on 14 million unique uk citizens who have been resident in London at some point in the last 30 years (including people who have left or died),  
All components are open source and are themselves built from open source and open standards such as the semantic web languages, JSON-LD and RESTful APIs. The ontology uses the NHS core ontologies of Snomed-CT and the NHS Data Dictionary, supplemented to accommodate the many local codes and text expressions. 


Data on individuals are held in a single common data model with data values represented by a super ontology composed of the world leading health ontologies.
== Information model Components ==


Data in discovery is submitted by health providers who provide primary and secondary care and urgent care services to patients or clients.
*[[Discovery health information model|Health Information model]] - An overview of the approach to the Health information model,  the purpose, and type of content.
*[[Health Information modelling language|Information model language]] - The Semantic Web languages used to build the various components of the information models
*[[Information model meta model]]. The class model (shapes model) of the classes used to hold the model content.
*[[Mapping and matching concepts|Mapping concepts and transforming published data]] - Introduces the approaches to matching and mapping concepts and the structural maps used in transforming published data,
*[[Architectures|Architectures -]] A high level overview of the architectures that the technologies contribute to
*[[GitHub repositories]] - descriptions and information relating to the application source code, .


Data is kept up to date in close to real time (at worst within 24 hours).
= Applications and APIs =


= Discovery overview =
These articles provide information about the applications that have been by the technologies


These articles provide background to the ideas, philosophies and approaches that have resulted in the Discovery Data Service. They cover the original objectives, the approach to collaboration and shared tenancy, and the information governance rules that underpin the way privacy and sharing are balanced.
*[[Information_model_service|Information model applications and services]] - the applications and technical services , such as APIs and apps that enable an model instance to be used or to implement an EHR based on the model or particular data model
 
*[[FHIR Get Structured Record API|get Structured Record APIs]] - returns a project specific patient record in a structured format using FHIR standards.
*[[What_is_Discovery|What is Discovery]]? - This article describes what Discovery is and how it is conceptualised at a very high level.
*[[Monitoring and alerting]] - provides an overview of the monitoring and alert systems in place.
*[[Case studies]] - An overview of some initial projects that access and utilise data from Discovery; information includes the applicant, project summary, and the latest status (Development/Live).
*[[Latest news]] - The latest information for all Discovery Data Service releases and changes, plus known issues and updates from system suppliers and other third parties.
*[[Current_data_sets|Current data sets]] - the broad types of data received from various systems.
*[[Current published data]] - provides the latest data publisher information by sector, geographical area, service, and system provider.
 
= Information model topics =
 
These articles provide information about the services, structure, and functionality that operate using the Information Models; one of the fundamental tenets of the Discovery Data Service. This is the starting point for those interested in health data and the information derived from it, and why Discovery may be different from other approaches:
 
*[[Discovery health information model|Health Information model]] - An overview of the approach to the Discovery information model,  the purpose, and type of content.
*[[Health Information modelling language|Modelling language]] - The W3C standard languages used to build the various components of the information model
*[[Health_data_model_content|Basic health data set ]] - a very high level overview of the types of data mapped from data held  within the Discovery data stores.
*[[Information model reference data|Reference data -]] an overview of additional data held in the information model.
*[[Mapping and matching concepts|Mapping and matching concepts and structures]] - Introduces the approaches to matching and mapping concepts and the structural mappings in the model
*[[Information_model_service|Information model services]] - the technical services , such as APIs and apps that enable an model instance to be used or to implement an EHR based on the model or particular data model
 
= Publishers and subscribers =
These articles provide some brief high level information for organisations that send data to Discovery and those that use the dats.how oDiscovery and pro
 
* [[Publishers]] - includes technical interface information, FHIR mapping, system requirements, and how to make changes to publishing services.
* [[Subscribers]] - includes set up, data mapping, and schema information.
 
= Services and applications =
 
These articles provide information about the services and applications provided by the Discovery information services.
*[[Monitoring and alerting]] - provides an overview of the monitoring and alert systems in place for the DDS and the infrastructure that supports


*[[Data Sharing Manager|Data Sharing manager]]- DSM provides a visual representation of data that is being shared and processed and by which organisations, regions, and/or services.
*[[Data Sharing Manager|Data Sharing manager]]- DSM provides a visual representation of data that is being shared and processed and by which organisations, regions, and/or services.
*[[UPRN address match application|ASSIGN- UPRN address matching application -]] a web based application that matches single or batches of hand entered address to authoritative addresses and assigns a unique property reference number.
*[[UPRN address match application|ASSIGN- UPRN address matching application -]] a web based application that matches single or batches of hand entered address to authoritative addresses and assigns a unique property reference number.
*[[Information model manager]] - an application designed to provide a view of the common information model, and support the authoring of the content of the model.
*[[Alert Generator]] - APIs that generates a patient alert based on a query, sends a notification to recipients and provides recipients with access to a  web application to view the content of the record according to the data set
*[[Information_model_service|Information model services]] - the technical services that enable the common information model to be used.
*[[Case studies|Data Distribution services]] that distribute daily weekly or adhoc small data sets from the linked core data stores with examples from 2020
*[[FHIR Get Structured Record API]] - returns a complete patient record in a structured format following GP and Care Connect standards.
*
*[[HL7v2 API]] - Send HL7v2 ADT and ORU (clinical events) data to Discovery.
*Get health record - the service that obtains a fully structured health record about a person, in real time via a standards based API.
*Record publication service - the service that publishes data from health records for specific purposes, often immediately on receipt.
*Data set distribution service - the service that distributes sets of data for populations of patients where the definition is complex or where the subscriber is hosting a copy of the data permanently.
*[[NHS 111 Discovery frailty flagging]] / [[Frailty algorithm]]
*[[GitHub repositories]] - descriptions and information relating to the application source code.
 
= Architectures =
These articles provide information about the overall high level software architecture models providing a descriptions and visualisations of the Discovery Data Service and information service components and how they interact.
 
* The [[Data Storage Architecture]] illustrates how data is stored within the service and the various implementation options available for organisations that wish to use some or parts of the services.
*[[Identity Authentication Authorisation|Identity, authentication and authorisation]] specifies the approach taken by Discovery for accessing utilities and controlling some aspects of access to application functionality and some data resources, after all of the other locks have been opened.
*[[Application zone architecture]] illustrates how the data is accessed in different places and in different ways according to different use cases
*[[Pseudonymisation]] describes how Discovery generates pseudonymised data.
*[[Data linking]] describes how Discovery links several patient records into a single person/citizen record.
 
= Application help =
 
*[[Data_Sharing_Manager|Data Sharing Manager]] - Data Sharing Manager provides a visual representation of data that is being shared and processed and by which organisations, regions, and/or services.
*:You can also amend or cancel data sharing and processing agreements in real time.
*[[Record Viewer]] - Record Viewer shows a combined view of an individual's patient records, using data from different source organisations.
*[[Jira]] - Jira is an effective way to share and track information and helps to make sure that tasks and issues are completed, and signed off, by all relevant teams.
 
= Wiki tips =
 
*[[Adding a new page to the wiki]]
*[[Adding tables from word documents]]
*[https://www.mediawiki.org/wiki/Help:Tables Mediawiki Help:Tables]
*[[Creating a DDS wiki watchlist]]
*[[Creating a new user]]
*[https://en.wikipedia.org/wiki/Help:List Wikipedia Help:Lists]
*[[Test page]]

Revision as of 11:34, 5 October 2022

These articles describe the various technologies underpinning the Endeavour open source technologies which have been adopted and used by a large scale NHS London Data linkage and normalisation Service covering a population of 7 million registered citizens.

Health Record Data Service and Health Information model

The health record data service is based on the hypothesis that; If health related data is brought together at the level of the individual in close to real time, and stored together at the level of a medium size residential population, and made available via a common data model as part of a common ontology, and then subsequently used for individual and population based decision support, great benefits to health can accrue.

The service uses technologies that consist of a set of components that link near real time data from millions of records from thousands of provider organisations, with scores of different formats and code systems and converts them to a single common information model, made up of a super ontology, a data model, ,maps between source data and the common model, sets of concepts sets and value sets for query, and query libraries.

All components are open source and are themselves built from open source and open standards such as the semantic web languages, JSON-LD and RESTful APIs. The ontology uses the NHS core ontologies of Snomed-CT and the NHS Data Dictionary, supplemented to accommodate the many local codes and text expressions.

Information model Components

Applications and APIs

These articles provide information about the applications that have been by the technologies

  • Data Sharing manager- DSM provides a visual representation of data that is being shared and processed and by which organisations, regions, and/or services.
  • ASSIGN- UPRN address matching application - a web based application that matches single or batches of hand entered address to authoritative addresses and assigns a unique property reference number.
  • Alert Generator - APIs that generates a patient alert based on a query, sends a notification to recipients and provides recipients with access to a web application to view the content of the record according to the data set
  • Data Distribution services that distribute daily weekly or adhoc small data sets from the linked core data stores with examples from 2020