Hippocrates, Winslow & Babbage HWB 
Foundation
HWB Mission Statement

The mission of the HWB foundation is to collect uniform and well specified clinical data in the form of text and graphics from reliable, university-affiliated sources and make that data, in quantities of statistical significance, available in the public domain. 

There, in an electronic bulletin board database format, collected data may be evaluated and re-evaluated by any party - particularly university affiliated research groups.

The foundation endeavors to establish a new pattern of research whereby instead of the basic data being available only in small samples within the purview of a select few determined to make a specific case - an open database deriving input from multiple sources is created to permit a larger sample size with equal access from all points of view. Thus, enhanced validation of the reporting is possible.
Registries - Why?
"Maintenance of an active registry must be viewed as important as the medical care rendered, if the right person is going to receive the right treatment in a timely fashion without undue cost to society.“

•	Gillot et al, Development of a Statewide Trauma Registry, J Trauma 29:1667, 1989
Data Collection - Why?
"The future belongs to whoever best measures quality of care and then markets it best. 

Whoever does will absolutely control the market, and everyone who doesn't will disappear."

•	Richard L. Scott, Esq, former CEO Columbia/HCA  HealthCare Corp.
Data Collection - Why?
"Tens of millions of dollars have been spent in the establishment of trauma databases over the past 15 years. Much data is gleaned from ICD entries into administrative databases. 

The emphasis here is for maximum reimbursement. Because of the lack of clinical detail the data within these registries, the data is inadequate for sophisticated research and outcome analysis. "  

Cushing, Champion - J TRAUMA 1995 
Open Tibia Fracture = ICD 823.3

                                  =
Practice Profiling
•	Pennsylvania Health Care Cost Containment Council 
•	Maine Quality Forum
•	Medicare Hospital Quality Initiative
Data Collection – How?
Data Collection – How?
HWB Shortfall – Not Unique
Electronic Medical Record -  a cresting wave for 30 years - has not broken.
10% adoption - Berner et al,  JAMIA Feb 2005 

Data Collection – How?
President Bush – State of the Union
Proposes all Americans have an EMR in 10 years to prevent medical errors and needless costs.

•	Spend $125 million                                 to get the job done.

                                                                                                                                http://www.jibjab.com
Data Collection – Failure - Causes 
 JAMIA Feb 2005 

EMR Problems:
•	Malaligned Incentives
•	Product Viability
•	Data Quality
•	Standards - Interoperability

Data Collection – Failure - Causes 
 JAMIA Feb 2005 
Malaligned Incentives

Minimal Return On Investment 
with 
Information vs Imaging technology.

CT study billable - not EMR data.
Data Collection – Failure - Causes 
 JAMIA Feb 2005
Malaligned Incentives - Rx
Indirect Reduction
through Practice Management Integration 

•	Billing
•	Reporting
•	Scheduling
•	Transcription
•	Archiving
Data Collection – Failure - Causes 
 JAMIA Feb 2005
Malaligned Incentives – Rx 
Indirect Reduction
through facilitation of:

•	Risk Adjusted Practice Profiling
•	Error Checking - Liability Reduction
•	RRC Certification
•	Academic Promotion

Data Collection – Failure - Causes 
 Leape, Massachusetts Plan, NPR 02/04/05
Malaligned Incentives – Rx
3rd Party Payors to purchase MD EMR.

EMR Cost @ $35,000/MD ($800 million)

3rd Party Savings ($ 5 billion) through:

•	fewer errors and complications
•	fewer inappropriate tests
•	fewer inappropriate meds
•	fewer billing errors

Data Collection – Failure - Causes 
 JAMIA Feb 2005 

EMR Problems:
•	Malaligned Incentives
•	Product Viability
•	Data Quality
•	Standards - Interoperability

Database Production and Maintenance 
Engh et al. CORR 421  April 2004
Product Viability
1977

Starts porous coated THA stem. 
Seeks documentation of outcomes. 

•	Data stored on paper forms in a physical chart.

Database Production and Maintenance 
Engh et al. CORR 421  April 2004 
Product Viability
1983 – 1992

Starts electronic archiving - 1 data entry person

•	dBASE II (Ashton-Tate)
•	dBASE III
•	dBASE IV
Database Production and Maintenance 
Engh et al. CORR 421  April 2004 
Product Viability
Early 1990's
Need:
Automated data entry 
Standardized classifications and outcomes

•	dBASE IV to International Documentation and Evaluation System (IDES) migration
Database Production and Maintenance 
Engh et al. CORR 421  April 2004 
Product Viability 

Mid 1990's 

IDES does not allow local modification.

•	IDES to FoxPro 2.6 migration with
•	Customized scanable forms. 
•	Codes for a comprehensive Orthopedic Research Database (ORDB)
•	Third-party applications to facilitate data entry, maintenance, and report generation. 
Database Production and Maintenance 
Engh et al. CORR 421  April 2004
Product Viability
2004
3rd party apps not compatible with new FoxPro
FoxPro 2.6 no longer supported by Microsoft 
FoxPro 2.6 not compatible with Windows XP.

Database not reflecting contemporary clinical concerns

•	Currently migrating to another database application.
Database Production and Maintenance 
Engh et al. CORR 421  April 2004
Product Viability
Summary:

•	7 major database changes in 21 years 
•	associated data corruption/loss
•	$50,000 - estimated annual data entry & maintenance cost ( does not include the surgeons’ time). 
•	New upgrades will substantially increase annual cost.
•	Consider saving the paper.
Database Production and Maintenance 
HWB Experience – Software Changes in 12 years
Product Viability

•	5 major database changes in 12 years 
•	Hypercard 1.0
•	Fox Pro 2.5 
•	Visual Fox Pro 3.0
•	FileMaker Pro
•	MySQL & PHP
•	4 major front-end changes in 12 years 
•	Hypercard 1.0
•	Hypercard 2.0
•	Metacard 2.5
•	HTML
Database Costs
Data Collection – Failure - Causes 
 JAMIA Feb 2005 

EMR Problems:
•	Malaligned Incentives
•	Product Viability
•	Data Quality
•	Standards - Interoperability

Data Quality

"It has become increasingly clear that much of the clinical research that has long been published and on which we base much of our education and practice activity is, in fact, severely flawed."

Keller, SPINE 20 384 1995

OTA Specialty Day Debate 1999
	Swiontkowski
Data Quality
 Tang, Shortliffe et al, JAMIA Proc 1994
An observational study of physicians using 168 medical records in an internal medicine outpatient clinic 

•	81% of cases failure of chart to provide all information deemed necessary by the physician for appropriate management of the patient. 

•	38% of a clinic encounter is taken up by chart review.

Data Quality
GIGO
What is written without pain is in general read without pleasure
 – Samuel Johnson 1709-1784

The quality of the data begins with the diligence of those who generate it. 
– Charles Engh - CORR 421
Data Quality
 Barrie, Marsh, BMJ. 1992

Manchester  physician-entered orthopaedic database

Data Quality 
= 
Completeness X Accuracy
			62%                   96%

Data Quality
Political Factors
Health Info Tech Success 
•	80% dependent on political factors 
•	20% dependent on info technology

Reed Gardner 1998 
Past President of AMIA
Data Quality
Political Factors
•	2% surgeons over 55 have a major interest in orthopaedic trauma. 
•	As it is these surgeons who tend to be politically influential it is clear that orthopaedic trauma is considerably disadvantaged. 
Data Quality
 Prospective vs. Retrospective
Do clinical databases give rise to
retrospective paper mills ?

Data Quality
 Random vs. Non-Random

Prospective randomized trials are well-proven for evaluating pharmaceuticals. 

Different hands administering a pill, however, have less influence on the result than different hands performing a surgical procedure. 

Matta JOT August 2001
Prospective vs. Retrospective
 Swiontkowski, JOT 2001 Aug 
Because of the difficulty with performing high-quality control trials, observational studies are often the best evidence we have. 
However
 When orthopaedists rely on these weaker forms of evidence, they must acknowledge the risk of utilizing a suboptimal or potentially even detrimental intervention for patients.
Prospective vs. Retrospective
 Benson et al, NEJM - June 22, 2000
Observational study advantages over PRCT:
•	lower cost
•	greater timeliness
•	a broader range of patients
However
Bias is a problem 
Some say:
•	observational studies are not reliable.
•	observational studies should not be funded.
•	observational studies should not be published. 
Prospective vs. Retrospective
 Bhandari et al, Arch Orthop Trauma Surg. 2004 Jan 
Femoral Neck Fx 
THA vs ORIF 
14 randomized vs 
13 non-randomized 

Prospective vs. Retrospective
 Benson et al, NEJM - June 22, 2000
•	 136 reports about 19 diverse treatments

•	We found little evidence that estimates of treatment effects in observational studies reported after 1984 are either consistently larger than or qualitatively different from those obtained in randomized, controlled trials
Prospective vs. Retrospective
 Concato et al, NEJM - June 22, 2000
•	 99 reports about 5 clinical topics

•	The results of well-designed observational studies (with either a cohort or a case-control design) do not systematically overestimate the magnitude of the effects of treatment as compared with those in randomized, controlled trials on the same topic.
Prospective vs. Retrospective
 Tornetta et al, OTA  2000
•	 Randomized trials are subject to Beta (Type II) error -  insufficient sample size. 
•	Beta (Type II) error is the probability of concluding that no difference between treatment groups exists, when, in fact, there is a difference.
•	The beta (Type II) error rate for randomized trials in orthopaedic trauma is exceedingly high, averaging 90%.
Prospective vs. Retrospective
 Sarmiento, JOT 2001 Aug 
•	There is not a system today, and there will not be a system tomorrow, that will guarantee the elimination of intended or unintended bias or prejudice from the medical literature. 

•	That being the case, let us accept outcome studies as well as traditional retrospective ones. 
Data Quality
 End User Impact 
Data Quality
EMR Design - Embi, JAMIA April 2004
End User Impact 
Portland VAH

•	Documentation Availability 
•	Work Processes and Communication
•	Alterations in Document Structure and Content
•	Mistakes, Concerns, and Decreased Confidence 

Data Quality
Patel, JAMIA November 2000
End User Impact
•	EMR use changes physician information gathering and reasoning strategies. 
•	Technology has a profound influence in shaping cognitive behavior.
•	Effects on cognition by technology design needs to be explored. 
Data Quality
AMNews   Feb. 17, 2003 
End User Impact
EMR Rejection - Cedars Sinai Revolt
"They poorly designed the system, poorly sold it and then jammed it down our throats and had the audacity to say everybody loves it and that it's a great system.“
 
Cedars-Sinai Medical Center in Los Angeles has indefinitely suspended use of its computerized physician order entry (CPOE) system, after hundreds of doctors complained it was difficult to use and compromised patient safety
 Los Angeles Times
Data Collection – Failure - Causes 
 JAMIA Feb 2005 

EMR Problems:
•	Malaligned Incentives
•	Product Viability
•	Data Quality
•	Standards - Interoperability

Standards & Interoperability 
Open Source

?? Solution ?? - Open Source Software – e.g. Linux

PROS
•	Low Cost - Free
•	Stable
•	Could provide essential infrastructure 
•	US Interstate Highways System
•	Internet Transmission Control Protocol / Internet Protocol
•	Public Library of program components

CONS
•	Difficult to program and obtain local support
•	Microsoft buys IT lunch – Significant kickbacks barriers
Standards & Interoperability 
Open Source

?? Solution ?? - Open Source Software – e.g. Linux

PROS
•	Low Cost - Free
•	Stable
•	Could provide essential infrastructure 
•	US Interstate Highways System
•	Internet Transmission Control Protocol / Internet Protocol
•	Public Library of program components

CONS
•	Difficult to program and obtain local support
•	Microsoft buys IT lunch – Significant kickbacks barriers
Summary
 EMR Potential
Data Collection – How?
Conclusion
 Hill, NEJM 1953

"One must go seek more facts, paying less attention to technique of handling the data and far more to the development and perfection of the method for obtaining them."

Revised HWB Mission Statement

The mission of the HWB foundation is to find methods to routinely collect well-specified, structured and privacy-protected clinical data from reliable  sources and make that data, in quantities of statistical significance, available in the public domain where it may be interpreted from all points of view.