Vancouver Coastal Health’s cloud modernization enables faster access

Every day, clinicians across Vancouver Coastal Health (VCH) document patient care in notes that are rich with clinical detail. Those narratives often capture the circumstances, contributing factors, and clinical context that never appear in traditional structured data fields.

The details are important. They can reveal why a patient presented to the emergency department or what influenced their course in hospital.

A patient may arrive intoxicated but leave with a different primary diagnosis code. An e-scooter injury may be recorded under a broad trauma category.

The coded record remains useful, but the narrative often tells a fuller story. For years, VCH’s on-premises data warehouse could only analyze structured data fields; the technology to extract and interpret free text at scale simply did not exist within the legacy platform.

That paradigm shifted when the VCH Data and Analytics team modernized its data foundation through a cloud-based analytics solution using the Databricks Data Intelligence Platform.

This platform introduced advanced analytics and artificial intelligence (AI) capabilities designed specifically to read plain-language free text clinical notes.

Under what is now established as the “Emergency Department Provider Notes” pipeline, the team deployed a large language model (LLM) classifier to scan 19 million Emergency Department notes. The AI successfully identified seven times more alcohol related ED visits than using traditional, structured discharge diagnostic coding and increased the case detection rate from a baseline of roughly one percent to eight percent.

Today, the team is scaling AI across more than 10 active use cases and establishing cross-organizational data federations, showing what becomes possible when cloud modernization unlocks data and AI unlocks its potential.

A platform that couldn’t keep pace: Serving 1.25 million people across the traditional territories of the Musqueam, Squamish, and Tsleil-Waututh Nations, VCH generates a large volume of clinical documentation through its Oracle Health electronic health record.

The challenge was not a lack of information, but the technological limitations of an aging on-premises infrastructure that struggled to keep pace. Queries were slow. Running AI workloads on aging architecture simply was not realistic.

The team could not query free-text at scale, could not prototype new models, and could not share data across organizations without unneeded data movement. Supporting the next generation of analytics and AI required modernizing the infrastructure while building a data foundation designed to scale for years to come.

The solution – Cloud-native, governed, reusable foundation: To support future analytics and AI needs, VCH Data and Analytics established a modern, scalable data foundation in the cloud, guided by Data Mesh principles: domain ownership, data as a product, self-serve infrastructure, and federated governance.

A semantic, reusable data foundation was built from source-system data objects, creating a critical mass of trusted data that analytics initiatives and AI models could draw upon without rebuilding from scratch. Rather than delivering project-by-project data solutions, the Data & Analytics team established a strategic enterprise asset that continues to accelerate innovation and scale new analytics and AI use cases across the organization.

This foundation was reinforced by robust governance, metadata management, and data lineage, while seven analyst workspaces opened up independent prototyping for the first time.

The human side of modernization mattered just as much. Executive sponsors cleared barriers, directors acted as change sponsors, managers as change leaders, and subject matter experts as super users and local champions who made AI adoption stick.

The culture prioritized momentum and ownership over perfection, transforming the department into active AI adopters.

Scalable natural language processing (NLP) and the modular architecture philosophy: The platform’s value showed up fast. The Emergency Department Provider Notes NLP pipeline project established more than just clinical insight, it established a repeatable framework where future use cases required no more than a prompt change to get moving.

Previously, validating the true burden of alcohol use would have taken months and dedicated research assistants. Any condition buried in free text but missed by structured coding became a candidate for discovery.

However, it was not technology alone that made this possible. Clinicians, data scientists, and engineers worked together to define patient populations, review model outputs, incorporate clinical feedback, and establish performance thresholds that ensured reliable results.

This collaboration enabled a modular “build once, deploy many” approach that reduced new use case development to just one or two days.

The pipeline could process millions of records in 30 to 90 minutes, followed by approximately one day of focused review and validation to generate actionable insights.