Find a way or make a way: Building an interactive analytics platform that speaks healthcare
By: Ali Koc, VP of Engineering
Find a way or make a way. It’s a phrase that gets bantered around in strategy sessions and brainstorming meetings at Prognos. It is part of our dogged commitment to transform the patient journey, impact care earlier, and improve healthcare outcomes. Faced with a quintillion bytes of data each day and a challenge to process and query this data quickly and effectively, this mantra echoed in our minds.
The recent launch of prognosFACTORTM revolutionized the way that healthcare data is queried – creating an interactive platform to answer complex healthcare questions in real-time. But in order to launch, we first needed a healthcare data store to power the new platform that speaks the language of healthcare, makes data accessible at web speeds, and performs data analysis in a truly new way.
Building a new analytics engine
Initially we set out to find a way. We explored expanding our usage of Apache Spark for this purpose, changing to an open source online analytical processing (OLAP) technology such as Apache Druid, ClickHouse or Apache Pinot, experimented with graph databases, and ultimately went the furthest with an in-memory bitmap index, Pilosa. While each of these technologies had certain advantages, none were able to query data in ways that are uniquely important for healthcare data and meet our ambitious speed requirements.
When we could not find a data technology solution in the market that met the challenge, it was clear we needed to make a way. We set out to build a technology solution that is lightweight, fast, and unique to the needs of healthcare data.
Healthcare data presents a number of challenges that make the use of conventional data technologies difficult. At Prognos we regularly manage billions of lab data records and more than 325 million patient records, so our depth of understanding of how to work with patient-centric data gave us confidence that we could build a technology that “speaks healthcare.” This meant we needed a technology that could partition data for millions of patients, utilize raw concurrency without a synchronization mechanism to improve performance, store information in-memory to provide a pure patient-centric storage mechanism, and provide dynamic views of the data. And, we wanted it to be able to do all of this and return query results in a few seconds or less. It was a big ask.
Setting a new industry standard
Our engineers, data scientists, and clinicians got to work and in the matter of 11 months conceptualized, built, and launched FACTOR Logic. As part of this development, they developed a purposeful query language that was optimized for patient-centric healthcare questions. The FACTOR Logic query (FLQ) language allows users to speak healthcare while intuitively asking questions of FACTOR Logic, and could be adopted across the industry for healthcare focused queries.
With FACTOR Logic and the prognosFACTOR platform which it powers, Prognos is the first in class to create an intuitive, flexible, powerful, and fully-functional interface that seamlessly takes advantage of the unique properties of healthcare data. FACTOR Logic’s ability to query vast amounts of patient-centric healthcare data with ease, return results in seconds, and illuminate the entire patient journey as it happens will change the way patient journey mapping is done and help companies better align brand strategy to clinical treatment decisions.
To dive deeper into the development of FACTOR Logic, download our white paper: The Search for a “Perfect” Healthcare Data Store: Building an Interactive Analytics Platform for Big Healthcare Data