EMIS published data transformation and mapping

From Endeavour Knowledge Base

This article describes the mapping of EMIS published data into the intermediate FHIR store.

 

Location and organisation

Admin_Location

Each record maps to a FHIR Location resource.

CSV Field FHIR Target
LocationGuid Id*
LocationName Name
LocationTypeDescription Type::Text
ParentLocationGuid PartOf::Reference *
OpenDate primarycare-activeperiod-extension::ValuePeriod::Start
CloseDate primarycare-activeperiod-extension::ValuePeriod::End
MainContactName primarycare-location-main-contact-extension::String
FaxNumber Telecom (use=work, system=fax)
EmailAddress Telecom (use=work, system=email)
PhoneNumber Telecom (use=work, system=phone)
HouseNameFlatNumber Address::Line
NumberAndStreet Address::Line
Village Address::Line
Town Address::City
County Address::District
Postcode Address::PostalCode
Deleted <If true, FHIR resource is deleted>
ProcessingId **

 

Admin_Organisation

Each record maps to a FHIR Organization resource.

CSV Field FHIR Target
OrganisationGuid Id*
CDB Identifier::Value (use=secondary, system=cdb-number)
OrganisationName Name
ODSCode Identifier::Value (use=officail, system=ods-organization-code)
ParentOrganisationGuid PartOf::Reference *
CCGOrganisationGuid PartOf::Reference *
OrganisationType Type***
OpenDate primarycare-activeperiod-extension::ValuePeriod::Start
CloseDate primarycare-activeperiod-extension::ValuePeriod::End
MainLocationGuid primarycare-mainlocation-extension::ValueReference*
ProcessingId **

 

Admin_OrganisationLocation

Each record is used to provide extra information for FHIR Location resources generated from the Admin_Location transform.

CSV Field FHIR Target
OrganisationGuid ManagingOrganization*
LocationGuid <used to match to the corresponding FHIR Location>
IsMainLocation <used to order the ManagingOrganization references>
Deleted <not mapped, since the Admin_Location deleted field is used>
ProcessingId **

 

Patient and users

Admin_Patient

Each record maps to a single FHIR Patient resource and a FHIR EpisodeOfCare resources. Note that if the patient is deducted and subsequently re-registers, then a second EpisodeOfCare resource is created that all future updates are mapped to.

CSV Field FHIR Target
PatientGuid Patient::Id*
EpisodeOfCare::Id*
OrganisationGuid EpisodeOfCare ::ManagingOrganization::Reference *
UsualGpUserInRoleGuid EpisodeOfCare ::CareManager::Reference *
Patient::CareProvider::Reference *
Sex Patient::Gender
DateOfBirth Patient::BirthDate
DateOfDeath Patient::Deceased
Title Patient::Name::Prefix (use=official)
GivenName Patient::Name::Given
MiddleNames Patient::Name::Given
Surname Patient::Name::Family
DateOfRegistration EpisodeOfCare ::Period::Start
Also used as part of the EpisodeOfCare Id
NhsNumber Patient::Identifier::Value (use=official, system=nhs-number)
PatientNumber Patient::Identifier::Value (use=secondary, system=patient-number)
PatientTypeDescription EpsisodeOfCare:: primarycare-patient-registration-type-extension::ValueCodeableConcept***
DummyType Patient::patient-is-test-patient-extension::ValueBoolean (extension is only created if value is true)
HouseNameFlatNumber Patient::Address::Line
NumberAndStreet Patient::Address::Line
Village Patient::Address::Line
Town Patient::Address::City
County Patient::Address::District
Postcode Patient::Address::PostalCode
ResidentialInstituteCode Patient::primarycare-patient-residential-institute-code-extension::ValueString
NHSNumberStatus Patient::primarycare-nhs-number-verification-status-extension::ValueCodeableConcept

(this is either blank or “Verified” in all known cases; if any other value is ever received, an error will be thrown)

CarerName Patient::Contact::Name::Text
CarerRelation Patient::Contact::Relationship::CodeableConcept
PersonGuid <not mapped>
DateOfDeactivation EpisodeOfCare ::Period::End
Deleted <If true, all FHIR resources for this patient are deleted>
SpineSensitive Patient::primarycare-patient-spine-sensitive-extension::ValueBoolean
IsConfidential Patient::primarycare-confidential::ValueBoolean (extension only created if value is true)
EmailAddress Patient::Telecom::Value (use=home, system=email)
HomePhone Patient::Telecom::Value (use=home, system=phone)
MobilePhone Patient::Telecom::Value (use=mobile, system=phone)
ExternalUsualGPGuid Patient::CareProvider::Reference *
ExternalUsualGP Patient::CareProvider::Reference *
ProcessingId **

 

Admin_UserInRole

Each record maps to a FHIR Practitioner resource.

CSV Field FHIR Target
UserInRoleGuid Id*
OrganisationGuid Role::ManagingOrganisation::Reference *
Title Name::Prefix
GivenName Name::Given
Surname Name::Family
JobCategoryCode Role::Role::Code
JobCategoryName Role::Role::Display
ContractStartDate Role::Period::Start
ContractEndDate Role::Period::End
ProcessingId **

 

Agreements_SharingOrganisation

Each record represents the state of the Emis-DDS sharing agreement and is not mapped to FHIR.

CSV Field FHIR Target
OrganisationGuid <not mapped>
IsActivated <not mapped>
LastModifiedDate <not mapped>
Disabled <If true, and transform not configured to whitelist this organisation as deleted, then the transform will fail>
Deleted <not mapped>

 

Appointments

Appointment_Session

Each record maps to a FHIR Schedule resource.

CSV Field FHIR Target
AppointmentSessionGuid Id*
Description Comment
LocationGuid primarycare-location-extension::ReferenceValue *
SessionTypeDescription Type::Text
SessionCategoryDisplayName Comment
StartDate PlanningHorizon::Start
StartTime PlanningHorizon::Start
EndDate PlanningHorizon::End
EndTime PlanningHorizon::End
Private <not mapped>
OrganisationGuid <not mapped>
Deleted If true, FHIR resource is deleted
ProcessingId **

 

Appointment_SessionUser

Each record is used to provide extra information for FHIR Practitioner resources generated from the Appointment_Session transform.

CSV Field FHIR Target
SessionGuid <used to match to the corresponding FHIR Schedule>
UserInRoleGuid Actor::Reference *
Deleted <if true, this “actor” is removed from the FHIR Schedule>
ProcessingId **

 

Appointment_Slot

Each record maps to both a FHIR Appointment a FHIR Slot resource.

CSV Field FHIR Target
SlotGuid Appointment::Id *
Slot::Id *
AppointmentDate Appointment::Start
Slot::Slot
AppointmentStartTime Appointment::Start
Slot::Slot
PlannedDurationInMinutes Appointment::End
Slot::End
<number of minutes is added to start time to derive end time>
PatientGuid Appointment::Participant::Reference *
SendInTime Appointment::primarycare-appointment-sent-in-extension::DateTimeValue
Appointment::Status (=Arrived)
LeftTime Appointment::primarycare-appointment-left-extension::DateTimeValue
Appointment::Status (=Fulfilled)
DidNotAttend Appointment::Status (=NoShow)
PatientWaitInMin Appointment::primarycare-appointment-wait-extension::DurationValue
AppointmentDelayInMin Appointment::primarycare-appointment-delay-extension::DurationValue
ActualDurationInMinutes Appointment::MinutesDuration
OrganisationGuid <not mapped>
SessionGuid Slot::Schedule::Reference *
DnaReasonCodeId Appointment::primarycare-appointment-dna-reason-extension::CodeableConceptValue (code and term looked up via Coding_ClinicalCode content)
Deleted <If true, both FHIR resources are deleted>
ProcessingId **

 

 

Care Record

CareRecord_Consultation

Each record maps to a FHIR Encounter resource.

CSV Field FHIR Target
ConsultationGuid Id *Ca
PatientGuid Patient::Reference *
OrganisationGuid ServiceProvider::Reference *
EffectiveDate Period::Start
EffectiveDatePrecision Period::Start
EnteredDate primarycare-recorded-date-extension::ValueDateTime
EnteredTime primarycare-recorded-date-extension::ValueDateTime
ClinicianUserInRoleGuid Participant::Reference *
EnteredByUserInRoleGuid primarycare-recorded-by-extension::ValueReference *
AppointmentSlotGuid Appointment::Reference *
ConsultationSourceTerm primarycare-encounter-source::ValueCodeableConcept::Text
ConsultationSourceCodeId primarycare-encounter-source::ValueCodeableConcept ****
Complete primarycare-encounter-incomplete::ValueBoolean (extension only set if value is false)
Deleted <If true, FHIR resource is deleted>
IsConfidential Patient::primarycare-confidential::ValueBoolean (extension only created if value is true)
ProcessingId **

 

CareRecord_Diary

Each record is mapped to a FHIR ProcedureRequest resource.

CSV Field FHIR Target
DiaryGuid Id *
PatientGuid Subject::Reference *
OrganisationGuid <not mapped>
EffectiveDate ScheduledDateTime
EffectiveDatePrecision ScheduledDateTime
EnteredDate OrderedOn
EnteredTime OrderedOn
ClinicianUserInRoleGuid Performer::Reference *
EnteredByUserInRoleGuid Orderer::Reference *
CodeId Code ****
OriginalTerm Code::Text
AssociatedText Notes
DurationTerm primarycare-procedure-request-schedule-text-extension::StringValue
LocationTypeDescription primarycare-procedure-request-location-extension::StringValue
Deleted <If true, FHIR resource is deleted>
IsConfidential Patient::primarycare-confidential::ValueBoolean (extension only created if value is true)
IsActive Status (=Requested)
IsComplete Status (=Completed)
ConsultationGuid Encounter::Reference *
(the target Encounter resource is also updated with a reference back to this resource)
ProcessingId **

 

 

 

CareRecord_Observation

Care record observations are a high level entity type. CareRecord_Observation contains data which is mapped to multiple FHIR resource types. Target FHIR profiles depend on content.

Each record maps to one of a FHIR DiagnosticReport, Observation, Procedure, Condition, AllergyIntolerance, FamilyMemberHistory, Immunization, Specimen, ReferralRequest or DiagnosticOrder. The algorithm used to determine the target type is listed following the standard map

CSV Field FHIR Target
ObservationGuid DiagnosticReport::Id * or
Observation::Id * or
Procedure::Id * or
Condition::Id * or
AllergyIntolerance::Id * or
FamilyMemberHistory::Id * or
Immunization::Id * or
Specimen::Id * or
ReferralRequest::Id * or
DiagnosticOrder::Id *
PatientGuid DiagnosticReport::Subject::Reference * or Observation::Subject::Reference * or
Procedure::Subject::Reference * or
Condition::Patient::Reference * or
AllergyIntolerance::Patient::Reference * or
FamilyMemberHistory::Patient::Reference * or
Immunization::Patient::Reference * or
Specimen::Subject::Reference * or
ReferralRequest::Patient::Reference * or
DiagnosticOrder::Subject::Reference *
OrganisationGuid <not mapped>
EffectiveDate DiagnosticReport::EffectiveDateTime or
Observation::EffectiveDateTime or
Procedure::PerformedPeriod::Start or
Condition::OnsetDateTime or
AllergyIntolerance::Onset or
FamilyMemberHistory::Date or
Immunization::Date or
Specimen::Collection::Collected::ValueDateTime or
ReferralRequest::Date or
DiagnosticOrder::Event::DateTime
EffectiveDatePrecision <same as above>
EnteredDate DiagnosticReport::primarycare-recorded-date-extension::DateTimeValue or
Observation::primarycare-recorded-date-extension::DateTimeValue or
Procedure:: primarycare-recorded-date-extension::DateTimeValue or
Condition::DateRecorded or
AllergyIntolerance::RecordedDate or
FamilyMemberHistory:: primarycare-recorded-date-extension::DateTimeValue or
Immunization:: primarycare-recorded-date-extension::DateTimeValue or
Specimen:: primarycare-recorded-date-extension::DateTimeValue or
ReferralRequest::primarycare-recorded-date-extension::DateTimeValue or
DiagnosticOrder::primarycare-recorded-date-extension::DateTimeValue or
EnteredTime <same as above>
ClinicianUserInRoleGuid DiagnosticReport::primarycare-diagnostic-report-filed-by-extension::ValueReference * or
Observation::Performer::Reference * or
Procedure::Performer::Reference * or
Condition::Asserter::Reference * or
AllergyIntolerance::Recorder::Reference * (note, this FHIR field should not have been used for this source field, but it’s too late to change without converting DDS content) or
FamilyMemberHistory:: primarycare-family-member-history-reporter-extension::ValueReference * or
Immunization::Performer::Reference * or
Specimen::Collection::Collector::Reference * or
ReferralRequest::Requester or Recipient::Reference * or
DiagnosticOrder::Orderer::Reference *
EnteredByUserInRoleGuid primarycare-recorded-by-extension::ValueReference *
(same extension used for all resources)
ParentObservationGuid parent-resource::ValueReference *
(same extension used for all resources, and the target resource is also updated with a reference back to this one)
CodeId DiagnosticReport::Code **** or
Observation::Code **** or
Procedure::Code **** or
Condition::Code **** or
AllergyIntolerance::Substance **** or
FamilyMemberHistory::Condition::Code **** or
Immunization::VaccinationCode **** or
Specimen::Type **** or
ReferralRequest::ServiceRequested::CodeableConcept **** or
DiagnosticOrder::Item::Code ****
ProblemGuid primarycare-condition-partofproblemepisode-extension::ValueReference *
(same extension used for all resources, and the target Condition resource is also updated with a reference back to this one)
AssociatedText DiagnosticReport::Conclusion or
Observation::Comments or
Procedure::Notes or
Condition::Notes or
AllergyIntolerance::Note or
FamilyMemberHistory::Note or
Immunization::Note or
Specimen::Collection::Comment or
ReferralRequest::Description or
DiagnosticOrder::Note
ConsultationGuid DiagnosticReport::Encounter::Reference * or
Observation::Encounter::Reference * or
Procedure::Encounter::Reference * or
Condition:: Encounter::Reference * or
AllergyIntolerance::encounter-associatedEncounter::ReferenceValue * or
FamilyMemberHistory:: encounter-associatedEncounter::ReferenceValue * or
Immunization::Encounter::Reference * or
Specimen::encounter-associatedEncounter::ReferenceValue * or
ReferralRequest::Encounter::Reference * or
DiagnosticOrder::Encounter::Reference *

(the target Encounter resource is also updated with a reference back to this one)

Value <not mapped for DiagnosticReport>
Observation::ValueQuantity::Value
<not mapped for Procedure>
<not mapped for Condition>
<not mapped for AllergyIntolerance>
<not mapped for FamilyMemberHistory>
<not mapped for Immunization>
<not mapped for Specimen>
<not mapped for ReferralRequest>
<not mapped for DiagnosticOrder>
NumericUnit <not mapped for DiagnosticReport>
Observation::ValueQuantity::Unit
<not mapped for Procedure>
<not mapped for Condition>
<not mapped for AllergyIntolerance>
<not mapped for FamilyMemberHistory>
<not mapped for Immunization>
<not mapped for Specimen>
<not mapped for ReferralRequest>
<not mapped for DiagnosticOrder>
ObservationType <not mapped>
NumericRangeLow <not mapped for DiagnosticReport>
Observation::ReferenceRange::Low
<not mapped for Procedure>
<not mapped for Condition>
<not mapped for AllergyIntolerance>
<not mapped for FamilyMemberHistory>
<not mapped for Immunization>
<not mapped for Specimen>
<not mapped for ReferralRequest>
<not mapped for DiagnosticOrder>
NumericRangeHigh <not mapped for DiagnosticReport>
Observation::ReferenceRange::High
<not mapped for Procedure>
<not mapped for Condition>
<not mapped for AllergyIntolerance>
<not mapped for FamilyMemberHistory>
<not mapped for Immunization>
<not mapped for Specimen>
<not mapped for ReferralRequest>
<not mapped for DiagnosticOrder>
DocumentGuid primarycare-external-document-extension::ValueString
(same extension used on all resource types)
Deleted <If true, FHIR resource is deleted>
IsConfidential primarycare-confidential::ValueBoolean
(extension only created if value is true, and same extension used on all resource types)
ProcessingId **

 

 


Observation content dependent rules

The target resource type is decided by the following rules, using the code and 'CodeType' from the Coding_ClinicalCode file (looked up using the CodeId from CareRecord_Observation) and the Value field from CareRecord_Observation.

Code type Rule Target profile
If the Value is empty, and the CodeType is one of (Biochemistry, Cyology_Histology,Haematology,Immunology,Microbiology,Radiology, Health_Management) DiagnosticReport

If the Value is not empty OR the CodeType is one of

(Biochemistry,Biological_Values,Cytology_Histology, Haematology, Health_Management, Immunology,Microbiology,Radiology,Symptoms_Findings, Procedure (note, the codes is this category aren't actually "procedures"), Administration_Documents_Attachments, Body_Structure //dental structures,Care_Episode_Outcome, Dental_Finding,Diagnostics, Discharged_From_Service,EMIS_Qualifier,Ethnicity,HMP,Intervention_Category, Intervention_Target, KC60,;Marital_Status, Nationality,Nursing_Problem, Nursing_Problem_Domain, Obsteterics_Birth,Person_Health_Social, Planned_Dental, Problem_Rating_Scale, Reason_For_Care, Referral_Activity, Referral_Rejected, Referral_Withdrawn, Regiment, Religion, Trade_Branch, Unset)

Observation

 

 

If the CodeType is Conditions_Operations_Procedures and the Code is in the original READ 2 categories of  (Preventive procedures, "Operations, procedures, sites", Other therapeutic procedures) Procedure
If the CodeType is Conditions_Operations_Procedures and the Code is in the oringal READ 2 categories of (Infectious and parasitic diseases, Neoplasms,Endocrine, nutritional, metabolic and immunity disorders, Diseases of blood and blood-forming organs, Mental disorders, Nervous system and sense organ diseases, Circulatory system diseases, Respiratory system diseases, Digestive system diseases, Genitourinary system diseases, Skin and subcutaneous tissue diseases, Musculoskeletal and connective tissue diseases)

Condition

 

Else If the CodeType is Conditions_Operations_Procedures Observation
If the CodeType is any of (Allergy_Adverse_Drug_Reations, Allergy_Adverse_Reations) AllergyIntolerance
If the CodeType is any of (Dental_Disorder, Dental_Procedure) Condition
If the CodeType is (Family_History) FamilyMemberHistory
 
f the CodeType is (Immunisations) Immunization
If code type is (Investigation_Requests) DiagnosticOrder
If code type is Pathology_Specimen Specimen
If code type is Referral Referral request

There is a mismatch between EMIS observations and FHIR resource profiles

Structural Rule Target profile

IF target profile IS Condition

   IF CSV Care Record observation ID is in CSV Care Record Problem ID

Combine problem and Condition into problem onset profile

In addition to the above mappings

R1: IF  the target resource select from above is not condition

       R2: IF the CSV CareRecord_Observation ID  is in CSV CareRecord_Problem record ID

Combine problem and other into Condition with "problem" profile
       R2: ELSE Observation
R1: Else    : See R3  

R3: If the CSV Care Record Observation has PROBLEM ID

        R4: IF the CSV care record Observation ID is NOT the PROBLEM ID

           R5: IF the CSV care record CODE = the linked problem ID CODE

Condition with episode indicator = review
(R3,R4,R5 : Else No change 

 

CareRecord_ObservationReferral

This file is used to provide additional data for CareRecord_Observation records, populating the extra data onto FHIR ReferralRequest resources.

CSV Field FHIR Target
ObservationGuid <used to match to the CareRecord_Observation record>
PatientGuid <used to match to the CareRecord_Observation record>
OrganisationGuid <not mapped>
ReferralTargetOrganisationGuid Recipient::Reference *
ReferralUrgency Priority::CodeableConcept ***
ReferralServiceType Type::CodeableConcept ***
ReferralMode primarycare-referral-request-send-mode-extension::CodeableConcept ***
ReferralReceivedDate <field is always empty> *****
ReferralReceivedTime <field is always empty> *****
ReferralEndDate <field is always empty> *****
ReferralSourceId <field is always empty> *****
ReferralSourceOrganisationGuid Requester::Reference *
ReferralUBRN Identifier::Value (use=official, system= ubrn)
ReferralReasonCodeId <field is always empty> *****
ReferringCareProfessionalStaffGroupCodeId <field is always empty> *****
ReferralEpisodeRTTMeasurementTypeId <field is always empty> *****
ReferralEpisodeClosureDate <field is always empty> *****
ReferralEpisodeDischargeLetterIssuedDate <field is always empty> *****
ReferralClosureReasonCodeId <field is always empty> *****
ProcessingId **

 

CareRecord_Problem

This file is used to provide additional data for CareRecord_Observation records, populating the extra data onto FHIR Condition resources. Any FHIR Condition matched to by this file uses the “problem” FHIR profile rather than the standard “condition” profile.

CSV Field FHIR Target
ObservationGuid <used to match to the CareRecord_Observation record>
PatientGuid <used to match to the CareRecord_Observation record>
OrganisationGuid <not mapped>
ParentProblemObservationGuid primarycare-problem-related-extension::Target::ReferenceValue *
Deleted <If true, the FHIR condition is downgraded from having the “problem” profile to the “condition” profile>
Comment <field not mapped as it’s always empty, but if data is ever detected, then an error will be raised so this can be implemented>
EndDate AbatementDate
EndDatePrecision AbatementDate
ExpectedDuration primarycare-problem-expectedduration-extension::IntValue
LastReviewDate primarycare-problem-lastreviewed-extension::Date::DateValue
LastReviewDatePrecision primarycare-problem-lastreviewed-extension::Date::DateValue
LastReviewUserInRoleGuid primarycare-problem-lastreviewed-extension::Performer::ReferenceValue *
ParentProblemRelationship primarycare-problem-related-extension::Type::StringType ***
ProblemStatusDescription AbatementBoolean <problem status is either “Active Problem” or “Past Problem”>
SignificanceDescription primarycare-problem-significance-extension::CodeableConceptValue ***
ProcessingId **

 

Coding_ClinicalCode

This file provides reference data on clinical codes, needed by other files. The content is loaded into a database table so records can be looked up. No FHIR resource is directly created from this file.

Coding_DrugCode

This file provides reference data on drug codes, needed by other files. The content is loaded into a database table so records can be looked up. No FHIR resource is directly created from this file.

Prescribing_DrugRecord

Each record in this file is mapped to a FHIR MedicationStatement resource.

CSV Field FHIR Target
DrugRecordGuid Id *
PatientGuid Patient::Reference *
OrganisationGuid <not mapped>
EffectiveDate DateAsserted
EffectiveDatePrecision DateAsserted
EnteredDate primarycare-recorded-date-extension::DateTimeValue
EnteredTime primarycare-recorded-date-extension::DateTimeValue
ClinicianUserInRoleGuid InformationSource::Reference *
EnteredByUserInRoleGuid primarycare-recorded-by-extension::ReferenceValue *
CodeId Medication ****
Dosage Dosage
Quantity primarycare-medication-authorisation-quantity-extension::QuantityValue
QuantityUnit primarycare-medication-authorisation-quantity-extension::QuantityValue
ProblemObservationGuid ReasonForUseReference::Reference *

(target Condition resource is also updated with a reference back to this one)

PrescriptionType primarycare-medication-authorisation-type-extension::CodeableConceptValue ***
IsActive Status
CancellationDate primarycare-medication-authorisation-cancellation-extension::DateValue
NumberOfIssues primarycare-medication-authorisation-numberofrepeatsissued-extension::IntValue
NumberOfIssuesAuthorised primarycare-medication-authorisation-numberofrepeatsallowed-extension::IntValue
IsConfidential Patient::primarycare-confidential::ValueBoolean (extension only created if value is true)
Deleted <If true, FHIR resource is deleted>
ProcessingId **

 

Prescribing_IssueRecord

Each record in this file is mapped to a FHIR MedicationOrder resource.

CSV Field FHIR Target
IssueRecordGuid Id *
PatientGuid Patient::Reference *
OrganisationGuid <not mapped>
DrugRecordGuid primarycare-medication-order-authorisation-extension::ValueReference *
EffectiveDate DateWritten

(this date is also used to populate the primarycare-medication-authorisation-firstissuedate-extension::ValueDate and primarycare-medication-authorisation-mostrecentissuedate-extension::ValueDate extensions on the FHIR MedicationStatement)

EffectiveDatePrecision DateWritten
EnteredDate primarycare-recorded-date-extension::DateTimeValue
EnteredTime primarycare-recorded-date-extension::DateTimeValue
ClinicianUserInRoleGuid Prescriber::Reference *
EnteredByUserInRoleGuid primarycare-recorded-by-extension::ReferenceValue *
CodeId Medication ****
Dosage DosageInstruction::Text
Quantity DispenseRequest::Quantity::Value
QuantityUnit DispenseRequest::Quantity::Unit
ProblemObservationGuid Reason::Reference *

(target Condition resource is also updated with a reference back to this one)

CourseDurationInDays DispenseRequest.ExpectedSupplyDuration.Value
EstimatedNhsCost primarycare-medication-order-estimatednhscost-extension::DecimalValue
IsConfidential Patient::primarycare-confidential::ValueBoolean (extension only created if value is true)
Deleted <If true, FHIR resource is deleted>
ProcessingId **

 

 

  • Source Emis identifiers are translated to DDS UUIDs using a persisted mapping.
    • ProcessingId columns are used to sequence records in the SFTP Reader application, but aren’t used in the FHIR transform.
      • Emis free-text mapped to value set for FHIR.
        • Code and Term looked up from CodeId using Coding_ClinicalCode or Coding_DrugCode content.
          • Emis specification states that these fields will be empty.

Prescribing_DrugRecord End Dates

The Prescribing_DrugRecord file contains both a Boolean active indicator (IsActive) and the medication end date (CancellationDate). Within Emis Web, it’s the IsActive indicator that is used to determine whether medication is active or not, the CancellationDate just providing additional information for ended medication. Analysing the received data has highlighted that there are a number of cases where the CancellationDate doesn’t match what the IsActive flag states:

  1) IsActive = true but a CancellationDate is present; 
  1) IsActive = false but a CancellationDate is absent.

From the dates of affected data, these appear to be related to either a service moving to Emis Web from another GP system, and are an artefact of that transfer.

To ensure consistency in the FHIR MedicationStatements created from this file, the Cancellation date will only be carried over if the DrugRecord is non-active. If no cancellation date is present, it will be derived from the medication start and course length (found from the associated Prescribing_IssueRecord file).

Prescribing_DrugRecord End Dates

The Prescribing_DrugRecord file contains both a Boolean active indicator (IsActive) and the medication end date (CancellationDate). Within Emis Web, it’s the IsActive indicator that is used to determine whether medication is active or not, the CancellationDate just providing additional information for ended medication. Analysing the received data has highlighted that there are a number of cases where the CancellationDate doesn’t match what the IsActive flag states:

  1. IsActive = true but a CancellationDate is present;
  2. IsActive = false but a CancellationDate is absent.

From the dates of affected data, these appear to be related to either a service moving to Emis Web from another GP system, and are an artefact of that transfer.

To ensure consistency in the FHIR MedicationStatements created from this file, the Cancellation date will only be carried over if the DrugRecord is non-active. If no cancellation date is present, it will be derived from the medication start and course length (found from the associated Prescribing_IssueRecord file).

Special Considerations

The following sections describe all the special cases that the transform needs to support.

Version Support

The two versions of the specification supplied by Emis are 5.1 and 5.3.1. The test pack data, although similar to the 5.1 specification has a number of differences, mostly column names being spelled incorrectly (e.g. “clinican” rather than clinician).

The extract data doesn’t declare its version in any of the files.

Because of the above two points, the transform will attempt to determine the version from the files and fields present, ensuring that all files/fields are consistent with a known format. For example, if a field contains a mis-typed column called “clinican” then it will validate that all other files/fields match the test pack format. If a known format cannot be matched, an error will be thrown. This will ensure that changes to the format (which are generally not announced) do not slip through unnoticed and relevant code changes can be implemented to support the changes.

 

Deleting Entire Records

Many of the files have a “Deleted” column to indicate that CareRecord_Observation, Prescrtiing_DrugRecord etc. should be deleted. The Admin_Patient file has a similar field, but we have determined that if this is set to true, the entire patient record should be deleted, not just data derived from the Admin_Patient record itself. Accordingly, the transform will support this, and treat an Admin_Patient delete as an instruction to delete all FHIR resources for the patient (within the scope of that service).

 

Deleting CareRecord_Problem Records

Problem records are represented by content in two files, CareRecord_Observation and CareRecord_Problem, both of which have a “Deleted” column. If an update is received with just “Deleted” true in CareRecord_Problem (and not in CareRecord_Observation), then the FHIR Condition resource should not be deleted, but should just be down-graded to being a non-problem Condition (i.e. change the profile URL).

If a CareRecord_Observation update is received with “Deleted” true, then the FHIR Condition should be deleted.

 

Deleting Deducted and Deceased Records

The data sharing agreement Emis have implemented for the DDS extracts includes active patients and non-active patients who were deducted or deceased in the last year. We have discovered that this rule is not only applied when a bulk extract is generated but is also applied to each daily extract.

For example, a bulk extract is received 01/01/2017 containing an active patient. If that patient leaves the practice on 01/02/2017, the delta extract the day after, on 02/02/2017, will inform of the deduction with an updated Admin_Patient record. A year later, on 02/02/2017 (or within a few days), a further update will be received for that patient with the “delete” field set to true.

To ensure that the DDS maintains patient records more than a year after deduction or death, the transform will ignore delete instructions for deducted or deceased patients except in the case where the Agreements_SharingOrganisation file (which represents the sharing agreement) shows that the agreement has been disabled (although extracts with disabled agreements are only processed if it is explicitly configured to do so for that organisation).

 

New Ethnicity and Marital Status Observations

If a patient has their ethnicity or marital status changed in Emis Web but no other demographic altered, the extract will contain the new CareRecord_Observation(s) but no Admin_Patient record.

The transform will ensure that these new ethnicities and marital statuses are added to the FHIR Patient resource even though no Admin_Patient record processing will be done for the patient.

 

Admin_Patient and Patient Re-registrations

The Admin_Patient file only contains one record per patient, giving the registration and deduction date (if present) of their most recent registration at the service. This means that if a patient is deducted and subsequently re-reigsters at the service, the Admin_Patient record is simply “updated” with the new registration date, and has the deduction date removed, overwriting any trace of their previous registration.

To avoid losing the history of past registrations, which affects analytics run using a past reference date, the transform will factor in the registration start date when creating EpisodeOfCare resources. Each distinct registration start date will result in a distinct EpisodeOfCare resource.

 

Missing Clinical and Drug Codes

Occasionally, when Emis release new clinical and drug codes into Emis Web, they fail to release the same update to their extract servers. This means that once clinicians start using these new codes, the extracts contain content that refers to these new codes (e.g. CodeId on CareRecord_Observation) but no corresponding Coding_ClinicalCode or Coding_DrugCode record was ever received.

Since Coding_ClinicalCode and Coding_DrugCode are needed to look up Read2 and DM+D codes and terms, the transform should fail if it detects that this has happened, reporting an error so the issue can be raised with Emis.

When this has previously happened, Emis have generally corrected the issue within a few days, and the next extracts received contain the missing Coding_ClinicalCode and Coding_DrugCode records.

Although processing for the affected service(s) is halted while there are missing codes (which also prevents processing of the subsequent extracts for those services), any extract processed for any unaffected service will resolve the issue for the affected service, meaning that processing can be resumed.

 

“Disabled” Extracts

The Agreements_SharingOrganisation file represents the Emis-DDS sharing agreement. It has a “disabled” field which is intended to indicate if the service ever manually disables the sharing agreement, which may happen if the practice is closing.

However, on a semi-regular basis (around ten times per month) a service’s extract will come through showing the sharing agreement as disabled when the service hadn’t disabled it. After investigation, it was believed that if an error occurred during the generation of the Emis extract, then this would result in this field being wrongly set to true. Typically, whatever error occurred would be fixed automatically on the next day (Emis seemingly being unaware this is happening) and the extract generation would resume.

Prior to April 2019, when the above problem happened, the Admin_Patient file would also include a “delete” for every patient, and when the extract was fixed it would contain a complete re-bulk of the data. To prevent the DDS inadvertently processing this mass delete of data (which would then reflect the delete to all data subscribers), the transform detects the disabled sharing agreement and halts processing data for that service. To allow processing of data when a service has legitimately disabled the sharing agreement, a whilelist of allowed disabled practices is maintained in the application configuration. Only services on the whitelist will have extracts processed if their sharing agreement shows as disabled.

To prevent the major disruption of deleting all data for a service only to re-add it the next day, when the fixed extract was received, the extract files for the date the problem happened (which will not have been processed) were replaced with new ones without the delete statement and with the sharing agreement changed to be not disabled. Additionally, a routine was run on previously received files and the re-bulk to detect any clinical records (e.g. CareRecord_Observation) that had legitimately been deleted on the day of the problem. These new files were then processed by the transform, processing the small number of actual deletes, if any, and then the re-bulk files were processed. The generic FHIR filing code supports re-processing data already saved, detecting duplicates and ignoring them, so processing this bulk data didn’t cause a problem (except in the time taken to do so).

Towards the end of April 2019, Emis seemingly changed their extract software (although no communication was received to say so). From this date onwards, when a service goes into the disabled state, no delete is received for any patient data. When the feed resumes, the next extract simply contains all the changes since the previous valid extract – so only the sharing agreement file needs to be corrected to allow processing to continue.

 

Non-Patient (Admin) Resources

Each of the extract feeds from Emis contains multiple services, up to around fifty. When each feed was first started, the first extract files received contained all the non-patient data, such as Organisations (in Admin_Organisation) and Practitioners (in Admin_UserInRole). When new services are added to a feed, these non-patient files aren’t re-sent.

However, DDS requires each service to have its own set of non-patient data, so that it can be co-located in the same FHIR database. So the transform much retain the non-patient data for subsequent processing as and when new services are added to the extract feed. This is done using the Emis Admin Cache, which is a table used to store the non-patient data on a per-extract feed basis. When a new service is added to the extract, this is detected and all non-patient data for that extract feed is transformed and saved before any actual data for the new service.

 

New Children for Pre-existing Observations

Although not common, it’s possible to receive CareRecord_Observation records that refer to ParentObservationGuid records that were received in a previous extract. Since ParentObservationGuid relationships are maintained in both child and parent resource (i.e. the child has a Reference to the parent and the parent has a Reference to the child) the transform must update any pre-existing parents with References to any new children.

 

New Children for Pre-existing Consultations

It’s possible to receive CareRecord_Observation and CareRecord_Diary records that refer to CareRecord_Consultation records received previously. Since ConsultationGuid relationships are maintained in both Encounter and child resources (i.e. the Encounter has References to its child resources, and the child resources have an Encounter reference), the transform must ensure that pre-existing Encounter resources are updated with any newly received child References.

 

New Children for Pre-existing Problems

When new observations or medications are recorded in Emis Web and linked to a problem, the subsequent update included the new CareRecord_Observation, Prescribing_DrugRecord or Prescribing_IssueRecord records with the ProblemObservationGuid set, but will not include any details on the problem itself (unless the problem was amended in some other way). Problem-child linkage is represented in both Condition and child resources (i.e. the Condition has References to its child resources, and the child resource have a Condition reference). The transform must ensure that the pre-existing FHIR Condition is updated with any newly received child References.

 

New IssueRecords for Pre-existing DrugRecords

If an existing, active repeat medication template is issued in Emis Web, the subsequent extract will contain a new record in Prescribing_IssueRecord, but no corresponding update to the Prescribing_DrugRecord file. The transform must handle this, not only saving the new FHIR MedicationOrder, but updating the “last issue date” extension on the linked FHIR MedicationStatement.

 

Updates to Pre-existing Consultations

If an aspect of a previously saved consultation is changed, the subsequent update will include the updated CareRecord_Consultation record. However, the extract won’t contain any linked records that weren’t directly updated themselves. In this scenario, the transform should ensure that any child References on the previously saved consultation are carried over onto the new, updated version.

 

Updates to Pre-existing Problems

When a “problem” is recorded in Emis Web, this new information is represented in both the CareRecord_Observation and CareRecord_Problem files, the former containing the common observation fields (date, code etc.) and the latter file having problem-specific fields. From that point on, if any of the problem-specific fields is amended, the subsequent extract may only contain an update in the CareRecord_Problem file, without an update to CareRecord_Observation. So the transform must support an update to a Condition resource (with the “problem” profile) that is only present in the CareRecord_Problem file.

The reverse, however, is not the case. If one of the common observation fields is amended in Emis Web, the subsequent extract will contain both CareRecord_Observation and CareRecord_Problem records for the problem.

In both cases, the list of child resources on the pre-existing Condition resource should also be carried over to the new, updated version.

 

Blood Pressures

Emis Web represents BP recordings using three CareRecord_Observation records: observations each with numeric value for the systolic and diastolic, both linked to a common parent observation.

In addition to the normal transform rules for CareRecord_Observation records, which will result in three FHIR Observations, with links between parent and children, the transform should also duplicate the numeric values in the Component structure on the parent Observation.

 

Read Code Standardisation

Whenever a FHIR resource is created or updated that contains a Read2 code (e.g. FHIR observation) the Read2 code is standardised to be in the five-character format used in TRUD files and other suppliers (e.g. Vision). This means that any code shorter than five characters is padded to five with dots. So “C10” is stored as “C10..”.

 

Ignoring Bulks

In a number of cases, multiple bulks of services were received before the migration of DDS to AWS. To reduce the amount of processing needed to be done to load the data into the AWS-based DDS, only the later bulk was processed, the earlier one (and any deltas before the subsequent re-bulk) being ignored. This was managed on a service-by-service basis, with the date of the second bulk being configured against the service ODS code and any patient-related data (e.g. Admin_Patient, CareRecord_Observation) before this date being ignored by the transform. Non-patient-related files for this period are still processed, so updates to staff, organisations etc. aren’t missed, as these files aren’t re-bulked when patient data is.

 

Appointment Cancellations

The Appointment_Slot file contains all the details of appointment slots, plus any details of appointments booked into those slots. If an appointment is cancelled, the following update will contain an updated Appointment_Slot record, but simply without any appointment details in it (or with different appointment details if the slot was rebooked before the extract).

The transformer must detect that a slot record has a different PatientGuid to the previous extract(s) and if so, mark the previously saved FHIR Appointment resource as cancelled.

 

Detecting Problem Reviews

Emis Web stores Problem reviews with a new Observation record with the same code as the Problem itself and linked to the Problem. Within Emis Web this is flagged as a review, so it is distinguishable from a new diagnosis. The review flag isn’t present in the extract, so it is necessary to infer the review status from the data available.

When a CareRecord_Observation record is received, if it links to a Problem (i.e. ProblemGuid isn’t empty) and the ReadCode for the new Observation is the same as that for the Problem (looked up using Coding_ClinicalCode content) then the new Observation will be flagged as a review, using the primarycare-problem-review extension.

Note that this will correctly not flag the initial Observation as a review, because the Observation that defines the Problem isn’t linked to the Problem in the same way (i.e. the ProblemGuid is empty but the CareRecord_Problem file links back to the Observation).

 

 

Mapping to FHIR Value Sets

There are a number of fields in the Emis extract that are mapped to FHIR fields that are value sets. In these cases, the mappings are performed using the files listed below. In each case, if a code is found that isn’t in the relevant mapping file, then an exception is thrown and the transform halted.

Field Mapping File
Admin_Patient::PatientTypeDescription https://github.com/endeavourhealth/Transforms/blob/master/src/main/resources/EmisPatientTypeMap.csv

 

Admin_Organisation::Type https://github.com/endeavourhealth/Transforms/blob/master/src/main/resources/EmisOrganisationTypeMap.csv
CareRecord_ObservationReferral::ReferralUrgency https://github.com/endeavourhealth/Transforms/blob/master/src/main/resources/EmisReferralPriorityMap.csv
CareRecord_ObservationReferral::ReferralServiceType https://github.com/endeavourhealth/Transforms/blob/master/src/main/resources/EmisReferralServiceTypeMap.csv
CareRecord_ObservationReferral::ReferralMode https://github.com/endeavourhealth/Transforms/blob/master/src/main/resources/EmisReferalModeMap.csv
CareRecord_Problem::ParentProblemRelationship https://github.com/endeavourhealth/Transforms/blob/master/src/main/resources/EmisProblemRelationshipTypeMap.csv
CareRecord_Problem::SignificanceDescription https://github.com/endeavourhealth/Transforms/blob/master/src/main/resources/EmisProblemSeverityMap.csv
Prescribing_DrugRecord::PrescriptionType https://github.com/endeavourhealth/Transforms/blob/master/src/main/resources/EmisDrugRecordPrescriptionTypeMap.csv

 

 

 

Mapping Ethnicity Codes

If the CodeType of a CareRecord_Observation is “Ethnicity”, then the record is used to populate the Ethnicity extension on the Patient resource (in addition to creating an Observation FHIR resource). The Patient extension requires the ethnicity to be mapped to a value set, which is done using these mappings: https://github.com/endeavourhealth/Transforms/blob/master/src/main/resources/EmisEthnicityMap.csv

Mapping Marital Status Codes

If the CodeType of a CareRecord_Observation is “Marital_Status”, then the record is used to populate the MaritalStatus field on the Patient resource (in addition to creating an Observation FHIR resource). The Patient resource requires the marital status to be mapped to a value set, which is done using these mappings: https://github.com/endeavourhealth/Transforms/blob/master/src/main/resources/EmisMaritalStatusMap.csv