Traffic Light Grading Scheme for EHR Data Readiness in Clinical AI Applications

Traffic Light Grading Scheme for EHR Data Readiness in Clinical AI Applications

A PRSB-aligned framework for evaluating FHIR compatibility and data structure quality

Summary:
This framework introduces a traffic light grading scheme to assess the quality and readiness of electronic health record (EHR) data for use in clinical AI summaries. The scheme categorizes data into three tiers—Green, Orange, and Red—based on its structural integrity, semantic richness, and degree of alignment with HL7 FHIR standards.

• Green Tier denotes FHIR-native, complete, and semantically rich data ready for immediate AI consumption with no transformation required.

• Orange Tier includes data that is structured but not FHIR-native, requiring automated wrangling and standardization to meet AI readiness criteria.

• Red Tier highlights incomplete or unstructured data that lacks clinical context, requiring significant manual intervention before safe use.

This grading model supports the Professional Record Standards Body (PRSB) in promoting data interoperability, safety, and transparency in the deployment of clinical AI systems, while helping health IT teams and vendors prioritize data quality improvements.

🟩 GREEN TIER – High Quality

“FHIR Native and Complete”

🔍 Technical Definition:

Data is natively stored as FHIR resources, structured and semantically rich with complete metadata. It meets all criteria for safe, immediate use by AI systems or clinical applications without transformation.

✅ Key Criteria:

• Data is already structured in FHIR format (e.g. Observation, Encounter, Condition, etc.)

• Uses standard terminologies (e.g. SNOMED CT, LOINC, ICD-10).

• Metadata like timestamp, practitioner, encounter ID, measurement units are stored in discrete fields.

• FHIR references maintain relationships between records (e.g., linking an observation to a patient and encounter).

• All required fields are present—no significant omissions or inferred values.

🧠 Lay Explanation:

This is the equivalent of a digital medical file that’s clean, complete, and ready to use. Everything’s labelled clearly—who, what, when, where, and why—so a computer can instantly understand it without confusion or human help.

🟧 ORANGE TIER – Moderate Quality

“Wranglable to FHIR”

🔍 Technical Definition:

Data is not in FHIR format, but roughly aligns with FHIR resource structures. It contains complete clinical content, and can be reliably transformed into FHIR using automated scripts or standard extract-transform-load (ETL) pipelines.

✅ Key Criteria:

• Clinical data is stored in structured but non-FHIR formats (e.g. CSV, relational databases, flat JSON).

• Most values are discrete (e.g. systolic blood pressure = 140), but may lack linked context (e.g. no encounter ID or provenance).

• Uses some coding systems, though inconsistently or incompletely (e.g. mixed use of free text and codes).

• Metadata may be implicit or inferable, requiring logic to extract.

• Flattened structure: hierarchical relationships may be lost or oversimplified (e.g. observations not tied to specific encounters).

🧠 Lay Explanation:

The information is all there, but it’s stored like a pile of receipts instead of a well-kept folder. It can be cleaned up and used with some smart software, but it needs work before it’s safe and useful.

🟥 RED TIER – Low Quality

“Incomplete or Disconnected”

🔍 Technical Definition:

Data is incomplete, poorly structured, and not readily transformable into FHIR. It may contain important clinical content, but significant manual intervention is required to extract, interpret, or structure it.

❌ Key Criteria:

• Critical fields are missing (e.g. date, performer, diagnosis code, identifiers).

• Heavy reliance on unstructured free text (e.g. “BP was high during clinic”) with no discrete data points.

• Free text is disconnected from structured observations (e.g. notes reference lab results that are not linked).

• Contextual relationships are missing—observations not tied to a patient, encounter, or time.

• Data is shallowed or flattened:

  • Flattened: relationships between data points (e.g. labs and encounters) are stripped.

  • Shallowed: only surface-level information exists without provenance or metadata.

🧠 Lay Explanation:

This is the messy drawer of medical data. Pieces are missing or jumbled, and someone has to go through it manually to figure out what’s what. It’s risky and unreliable for use in AI unless it’s heavily cleaned up.