OTA 2005 Posters

Scientific Poster #65 Basic Science

Level of Evidence in Orthopaedic Journals

William T. Obremskey, MD, MPH1; Emad Attallah-Wasif, MD1;
Nick Pappas MS1, Paul Tornetta, III MD2, Mohit Bhandari MD3
(n-all authors);
1Vanderbilt University, Nashville, Tennessee, USA;
2Boston Medical Center, Boston University, Boston, Massachusetts, USA;
3McMaster University, Hamilton, Ontario, Canada

Purpose: The purpose of this study was to assess the type and level of evidence (LOE) in several orthopaedic journals.

Methods: We reviewed all clinical articles published from January to June 2003 in the following journals: Journal of Bone and Joint Surgery (Am), Journal of Bone and Joint Surgery (Br), Journal of Orthopaedic Trauma, Journal of Shoulder and Elbow Surgery, American Journal of Sports Medicine, Journal of Pediatric Orthopaedics, Foot and Ankle International, Journal of Hand Surgery (Am), and Journal of Arthroplasty. The exclusion criteria were as follows: case reports, editorials, review articles, and basic science articles. The 382 articles that satisfied the above criteria were randomly assigned to three experienced reviewers and two inexperienced reviewers who graded the articles based upon the grading system described in the Journal of Bone and Joint Surgery (Am), January 2003. Each reviewer first determined whether studies were therapeutic, prognostic, diagnostic, or economic and then rated the LOE as l, 2, 3, or 4. The inexperienced reviewers' articles were regraded by a more senior reviewer, and the articles of the two experienced reviewers were reevaluated by the third senior reviewer. Reviewers were blinded to the grading of other reviewers.

Results: Of all articles 70.5 % were therapeutic, 20% were prognostic, 9% were diagnostic, and 0.5% were economic. Of all articles 11.2% were level 1, 20.8% were level 2, 9.9% were level 3, and 58.1% were level 4. Kappa values between senior and junior reviewers were 0.62 for level and 0.76 for study type; kappa values between senior and senior reviewers were 0.75 for level and 0.85 for study type (P values for the four kappas were all <0.0001). Kappa values between reviewers and JBJSA were 1.00 for type and 0.84 for level. Descriptive statistical analysis is shown in Table 1 and 2. The percentage of level 1 and/or level 2 articles increased proportionately with the 2003 impact score for each journal (P=0.0061).

Conclusion/Significance: Orthopaedic journals with higher impact scores (JBJSA, JBJSB, and AJSM) are more likely to publish level 1 or 2 articles. The type and level of information in orthopaedic journals can be reliably quantified and editors should be encouraged to publish the best evidence available.

Level of Evidence Data Tables

Table 1: Level of Evidence by Journal
 Journal
LOE		  AJSM  FA  JA  JBJSA  JBJSB  JH  JOT  JPO  JSES  Total
 1 11  3  7  8  10  1  1  1  1  43
 2  6  5  9  17  10  8  7  11  6  79
 3  3  2  8  8  5  1  1  5  5  38
 4 15  30  30  43  27  22  13  31  11  222
 Total  35  40  54  76  52  32  22  48  23  382

Table 2: Type of Study by Journal

 Journal
Study Type		  AJSM  FA  JA  JBJSA  JBJSB  JH  JOT  JPO  JSES  Total
Diagnostic 4 5 3   6  4  2  2  5  3  34
Economic 0 0 1   0  0  1  0  0  0  2
Prognostic 14 6 10  17  6  4  6  11  2  76
 Therapeutic 17  29  40   53  42  25  14  32  18  270
 Total  35  40  54  76  52  32  22  48  23  382

If noted, the author indicates something of value received. The codes are identified as a-research or institutional support; b-miscellaneous funding; c-royalties; d-stock options; e-consultant or employee; n-no conflicts disclosed, and *disclosure not available at time of printing.