Comparison of Publication Rates for Musculoskeletal Oncology Abstracts Presented at National Meetings

Collier, Christopher D.; Quereshy, Humzah A.; Getty, Patrick J.

doi:https://doi.org/10.1155/2021/8326318

Sarcoma

On this page

Abstract Introduction Methods Results Discussion Conclusions Data Availability Conflicts of Interest Authors’ Contributions References Copyright Related Articles

Research Article | Open Access

Volume 2021 | Article ID 8326318 | https://doi.org/10.1155/2021/8326318

Comparison of Publication Rates for Musculoskeletal Oncology Abstracts Presented at National Meetings

Christopher D. Collier,¹Humzah A. Quereshy,²and Patrick J. Getty²

Academic Editor: Valerae O. Lewis

Received09 Jul 2020

Revised10 Jan 2021

Accepted20 Feb 2021

Published08 Mar 2021

Abstract

Background. Scientific meetings provide a forum to disseminate new research and advance patient care. The American Academy of Orthopaedic Surgeons (AAOS), Connective Tissue Oncology Society (CTOS), and Musculoskeletal Tumor Society (MSTS) annual meetings are examples of such gatherings in the field of musculoskeletal oncology. After a review of select MSTS abstracts from 1991 to 1999 revealed a 41% publication rate in scientific journals, previous authors cautioned meeting attendees that the majority of abstracts may not survive rigorous peer review and may not be scientifically valid. Since two decades have passed, this study reexamined publication rates and characteristics in a contemporary and expanded cohort of oncology abstracts presented at the AAOS, CTOS, and MSTS annual meetings. Methods. 1408 podium and poster abstracts from the AAOS (oncology-focused from 2013 to 2015), CTOS (2012 to 2014), and MSTS (2012 to 2014) annual meetings were reviewed to allow for a four-year publication window. Searches were performed with PubMed and Google Scholar databases to identify full-text publications using abstract keywords. Characteristics of each abstract and resulting publication were collected. Statistical analysis was performed using the chi-square and Kruskal-Wallis tests for time-independent comparisons, and the log-rank test after reverse Kaplan-Meier analysis for time-dependent comparisons. Results. Abstract publication rates overall were higher for podium presentations (67%, 280 of 415) compared to poster presentations (53%, 530 of 993; ). When both abstract types were combined, differences between meetings did not meet statistical significance (AAOS: 65%, 106 of 162; CTOS: 57%, 521 of 909; MSTS: 54%, 183 of 337, ). Abstracts from AAOS meetings were more often published prior to the first day of the meeting (AAOS: 24%, 25 of 106; CTOS: 10%, 52 of 521; MSTS: 14%, 25 of 183; ). After excluding previously published abstracts, AAOS abstracts had the shortest time to publication (median: 10.8 months, interquartile range (IQR): 4.4 to 18.8 months), compared to those from CTOS (16.0 months, 8.4 to 25.9 months, ) and MSTS (15 months, 7.9 to 25.0 months, ) meetings. CTOS abstracts were published in higher impact journals (median: 3.7, IQR: 2.9 to 5.9), compared to those from AAOS (2.9, 1.9 to 3.2, ) and MSTS (3.1, 2.3 to 3.1, ) meetings. Finally, 7.7% (62 of 810) of published abstracts were presented at more than one meeting. Conclusions. Publication rates in this study were higher than previous reports in musculoskeletal oncology and comparable or better than recent reports for other orthopedic meetings. Comparisons across the AAOS, CTOS, and MSTS annual meetings highlight notable differences but suggest similarity overall in the quality of evidence presented with little overlap between meetings. Taken together, this study points to progress in the review processes used by the program committees, reaffirms the importance of critical appraisal when considering abstract findings, and supports the continued organization of multiple scientific meetings in musculoskeletal oncology.

1. Introduction

Scientific meetings provide an opportunity for scientists and clinicians to disseminate new research prior to publication in an environment that fosters innovation, collaboration, and debate. These forums are longstanding traditions and play an important role in advancing scientific discovery and patient care. New information is typically presented as an abstract, by podium or poster, following a peer-review selection process coordinated by the meeting program committee. Presentation is ideally followed by full-text publication in a peer-reviewed journal, which provides a more rigorous appraisal that meets the highest standards of scientific scrutiny. The quality of abstracts presented at a meeting can therefore be estimated by their publication rate.

The field of musculoskeletal oncology is comprised of many specialists, including orthopedic oncologists for whom the American Academy of Orthopaedic Surgeons (AAOS), Connective Tissue Oncology Society (CTOS), and Musculoskeletal Tumor Society (MSTS) annual meetings are frequently attended. Each of these meetings targets a slightly different audience but it is currently unclear how much overlap exists between meetings, and whether one meeting offers superior evidence over another. To our knowledge, only publication rates from the MSTS meeting have been examined previously. In a 2003 study, a review of select MSTS podium abstracts from 1991 to 1999 revealed a 41% publication rate [1]. The authors of this study strongly caution meeting attendees to “beware of the unpublished abstract”. They reason that the majority of abstracts presented may not survive rigorous peer review and, therefore, may not be scientifically valid. Overinterpretation of such abstracts, without a proper critical appraisal, can misdirect future research and clinical care. Examination of publication rates in other orthopedic subspecialties, notably trauma, demonstrate significant improvement over time [2, 3]. A more recent analysis of the MSTS meeting or related musculoskeletal oncology meetings has not been published. Such an analysis is desirable to set the stage for the judicious interpretation of abstracts, justify continued participation in multiple meetings, and evaluate progress over time. This study, therefore, asked the following: (1) what are the publication rates for oncology abstracts presented at the AAOS, CTOS, and MSTS annual meetings? and (2) what are the characteristics of publications following abstracts for each meeting?.

2. Methods

A complete collection of abstracts from the final programs of the AAOS (2013 to 2015), CTOS (2012 to 2014), and MSTS (2012 to 2014) annual meetings were retrieved from organizational websites and organized into a single database. The 2015 AAOS annual meeting, which occurred in March of 2015, was the most recent meeting included to allow for a four-year publication window at the time of review. All abstracts presented as a podium or poster at the oncology-focused CTOS and MSTS meetings during this time period were included in this study. For AAOS meetings, only podium abstracts presented during oncology sessions or poster abstracts classified under tumor were included. For each abstract, a PubMed (https://pubmed.ncbi.nlm.nih.gov) and Google Scholar (https://scholar.google.com) search was performed to determine if there was a full-text publication associated with the abstract. Searches included both abstract keywords and either the first or last author of the abstract.

An abstract was considered published if the direct comparison of the meeting abstract and the full-text publication contained substantial similarities in methodology and results within four years of abstract presentation, in accordance with previously described methods [1]. For example, if the presented abstract was altered in the published form, but still had the same focus, it was scored as a published abstract. Similarly, presented abstracts with data that was included as part of a larger publication were considered published. Overall, the criteria for publication were liberal. If no match was identified, the abstract was not considered published. To ensure the reproducibility of these methods, an independent search was first performed for ten abstracts by three authors (CDC, HAQ, and PJG) with complete agreement. One author (HAQ) then completed a search for each of the 1408 podium and poster abstracts included. Of these, 138 abstracts were identified as having uncertain publication status and underwent a second independent review by a more senior author (CDC) who made the final determination of publication status. All searches were conducted from April to November of 2019.

Abstract characteristics were collected, including the year of presentation, name of the meeting, presentation type (podium or poster), and country of origin. The program for the 2013 MSTS meeting did not include the country of origin. For abstracts meeting criteria to be considered published, publication characteristics were collected, including date of online publication, journal of publication, and the journal impact factor, as reported in the 2015 Journal of Citation Reports [4]. It was also noted if an abstract was published prior to the first day of the meeting or if a published abstract was presented at more than one meeting, the latter identified only if abstracts from multiple meetings resulted in the same publication. Time to publication was determined in months from the first day of the meeting to the date of online publication, excluding abstracts published before the meeting.

2.1. Statistical Analysis

Descriptive statistics provided include counts with proportions for categorical variables and median values with interquartile ranges for continuous variables. Categorical variables were analyzed using the chi-square test. Continuous variables were not normally distributed and were therefore analyzed using the Kruskal-Wallis test followed by Dunn’s test for multiple comparisons. The cumulative incidence of publication over time was estimated using the reverse Kaplan-Meier method and compared across groups using the log-rank test. To visualize the overlap of published oncology abstracts presented at AAOS, CTOS, and MSTS meetings, an area-proportional Venn diagram was constructed using eulerAPE (University of Kent, Canterbury, UK) [5]. All other statistical and graphical analyses were performed using Prism 8 (GraphPad, La Jolla, CA, USA). All statistical testing was two-sided, with a value less than 0.05 being considered significant.

3. Results

3.1. Publication Rates for AAOS, CTOS, and MSTS Oncology Abstracts

Abstract publication rates overall were higher for podium presentations (67%, 280 of 415) compared to poster presentations (53%, 530 of 993; ). Differences in podium publication rates between the AAOS (69%, 66 of 96), CTOS (70%, 127 of 181), and MSTS (63%, 87 of 138) meetings were not statistically significant () (Table 1). Poster publication rates were statistically different and higher for AAOS meetings (AAOS: 61%, 40 of 60; CTOS: 54%, 394 of 728, MSTS: 48%, 96 of 199; ). When both abstract types were combined, differences between meetings did not meet statistical significance (AAOS: 65%, 106 of 162; CTOS: 57%, 521 of 909, MSTS: 54%, 183 of 337, ). Publication rates increased overall from year one after presentation (AAOS: 43%, 95% confidence interval (CI) 35% to 52%; CTOS: 26%, 21% to 30%, MSTS: 28%, 21% to 35%) to year two (AAOS: 57%, 95% CI 50% to 64%; CTOS: 43%, 39% to 46%, MSTS: 42%, 36% to 48%) to year three (AAOS: 64%, 95% CI 57% to 69%; CTOS: 52%, 49% to 55%, MSTS: 50%, 45% to 56%) in a time- and meeting-dependent manor () (Figure 1(a)). Considering each meeting type separately, only the MSTS meeting demonstrated a significant change over time, with a higher overall publication rate for the smaller 2014 meeting (71%, 57 of 80) compared with the 2012 (48%, 54 of 112) and 2013 (50%, 72 of 145) meetings () (Figures 1(b)–1(d)).

(a)

(b)

(c)

(d)

Figure 1

Publication rates over time for oncology abstracts presented at select AAOS, CTOS, and MSTS annual meetings are shown in (a). The publication rates over time are also shown for individual AAOS (b), CTOS (c), and MSTS (d) meetings. Time to publication was calculated as the interval from the first day of the meeting to online publication. Abstracts published prior to the first day of the meeting are depicted by the origin of each line on the y-axis. AAOS indicates the American Academy of Orthopaedic Surgeons; CTOS, Connective Tissue Oncology Society; MSTS, Musculoskeletal Tumor Society.

3.2. Characteristics of Publications Resulting from AAOS, CTOS, and MSTS Oncology Abstracts

Abstracts from AAOS meetings were more often published prior to the first day of the meeting (AAOS: 24%, 25 of 106; CTOS: 10%, 52 of 521, MSTS: 14%, 25 of 183; ). After excluding previously published abstracts, AAOS abstracts had the shortest time to publication (median: 10.8 months, interquartile range (IQR): 4.4 to 18.8 months), compared to those from CTOS (16.0 months, 8.4 to 25.9 months, ) and MSTS (15 months, 7.9 to 25.0 months, ) meetings. CTOS abstracts were published in higher impact journals (median: 3.7, IQR: 2.9 to 5.9), compared to those from AAOS (2.9, 1.9 to 3.2, ) and MSTS (3.1, 2.3 to 3.1, ) meetings. The most common journals of publication and countries of origin varied by meeting, as demonstrated in Tables 2 and 3, respectively. Finally, 7.7% (62 of 810) of published abstracts were presented at more than one meeting (1.0% [8 of 810] presented at AAOS, CTOS, and MSTS; 1.0% [8 of 810] presented at AAOS and CTOS only; 2.7% [22 of 810] presented at AAOS and MSTS only; and 3.0% [24 of 810] presented at CTOS and MSTS only) (Figure 2).

4. Discussion

The AAOS, CTOS, and MSTS annual meetings are important forums to advance research and patient care in musculoskeletal oncology. From 2012 to 2015, the overall publication rates for podium and poster presentations at these meetings were 67% and 57%, respectively. AAOS oncology abstracts were characterized by higher poster publication rates, publication prior to the meeting, and shorter time to publication. CTOS abstracts had longer time to publication and were published in higher impact journals. Only the MSTS meeting demonstrated a significant change in publication rates over time. Despite these notable differences, there was no statistically significant difference in the podium and overall publication rates between meetings or evidence of significant overlap in the information presented. Together, these findings suggest a continued role for multiple scientific meetings in musculoskeletal oncology.

There are important limitations to our study. First, the publication-search methodology, though modeled after similar studies, is inherently susceptible to missed or misidentified events that could have resulted in an underestimation or overestimation of publications rates and characteristics. Considerable efforts were made to mitigate this risk, including searching multiple abstract author names in two databases (PubMed and Google Scholar) by multiple authors of the current study, but the risk of missed or misidentified publications is not negligible. Second, abstracts resulting in publications that occurred more than four years after the meeting were not considered published abstracts, which allowed direct comparison of meetings over time. Though previous studies and our experience indicate that a publication resulting from an abstract presented over four years prior is rare, this may have resulted in an underestimation of publication rates. Third, though we reviewed 1408 abstracts over three years, a larger cohort with earlier meetings would have improved our assessment of trends over time. Despite these limitations, this study is strengthened by the direct comparison of three major meetings in musculoskeletal oncology, which has not been previously performed to our knowledge.

For all meetings included in this study, the publication rate was higher for podium presentations than poster presentations. This finding is consistent with prior reports on the AAOS, Orthopaedic Trauma Association (OTA), American Orthopaedic Society for Sports Medicine (AOSSM), and American Orthopaedic Foot and Ankle Society (AOFAS) annual meetings and suggests that podium presentations contain higher quality evidence that is more likely to survive the peer-review required for full-text publication [2, 6–8]. Comparing oncology abstracts from the three meetings reviewed here, the AAOS meetings had higher publication rates than CTOS or MSTS meetings, though this only reached statistical significance for poster presentations. Though not studied directly, this may be related to meeting size. The AAOS meetings and the highly published 2014 MSTS meeting had fewer presentations overall, which may have resulted from greater selectivity by program committees. Still, all publication rates reported in our study were higher than the 41% publication rate for MSTS podium abstracts from 1991 to 1999 [1]. This trend towards higher publication rates in recent years is reflected throughout orthopedics. Publication rates for national orthopedic meetings ranged from 34% to 67% in the 1990s, compared to 54% to 74% in recent reports, which is similar to our results [1–3, 8–16]. The OTA meeting is one well-studied example, with relatively high publication rates overall that increased from 67% (1994 to 1998) to 73% (2008 to 2012) for podium presentations [2, 3]. This increase corresponded to an increase in the level of evidence, as defined by commonly used evidence-based guidelines, presented over the same time period [17]. However, a similar study examining the level of evidence presented at MSTS meetings did not identify an improvement over time [18]. This suggests that the higher publication rates observed in our study are not related to improved levels of evidence. Alternatively, higher publication rates may reflect an improvement in the review process used by program committees, an increase in the number of journals available for publication, accessibility of open-access journals, more pressure to publish, or other unidentified factors.

There were some differences in the characteristics of publications resulting from AAOS, CTOS, and MSTS meetings. First, 24% of AAOS successfully published oncology abstracts were published prior to the meeting, and the remaining were published a median of 10.8 months thereafter, which was significantly faster than CTOS (16.0 months) and MSTS (15.0 months) abstracts. As noted previously, this finding may be related to the smaller number of abstracts presented at the AAOS meeting and greater selectivity that resulted in the presentation of more developed projects. It may also reflect the timing of the AAOS meeting, which accepts abstracts in the late summer before presentation in spring of the following year. By contrast, the CTOS and MSTS meetings accept abstracts in the late summer for presentation in the fall of the same year. The longer time interval between abstract acceptance and presentation for AAOS meetings allows more time for publication. Despite these differences, all meetings studied here had a shorter time to publication than the 21.8 months average previously reported for MSTS podium presentations from 1991 to 1999 [1]. Second, CTOS abstracts were published broadly and in significantly higher impact journals than AAOS or MSTS abstracts, most commonly in Annals of Surgical Oncology (5%), European Journal of Cancer (4%), and PloS One (4%). This finding likely reflects broader participation by medical oncologists, radiation oncologists, basic scientists, and other nonorthopedic surgeons at CTOS meetings, for whom higher impact journals are frequently targeted for publication. For both AAOS and MSTS, the most common journal of publication was Clinical Orthopaedics and Related Research, which accounted for 14% and 27% of publications, respectively. Clinical Orthopaedics and Related Research is the official publication of the MSTS and publishes proceedings from the MSTS meeting annually. Third, consistent with the organizational mission of CTOS to be international, 67% of CTOS abstracts originated from outside the United States. This is in contrast to the United States-based AAOS and MSTS, for which 49% and 35% of abstracts originated outside the United States, respectively. Finally, only 7.7% of published abstracts were presented at more than one meeting to suggest that the majority of the information presented at each meeting is unique. Therefore, each meeting serves a different audience, and together, they offer three unique programs to choose from, as attendees consider the best use of their time and resources.

5. Conclusions

In summary, publication rates in this study were higher than previous reports in musculoskeletal oncology and comparable or better than recent reports for other orthopedic meetings. These findings must be considered along with the recognition that many abstracts presented at national meetings are not subsequently published in peer-reviewed journals. Meeting attendees are therefore encouraged to think critically when digesting research from any source, particularly meeting abstracts, and ideally should weigh a body of evidence before altering research or clinical practices. Comparisons across the AAOS, CTOS, and MSTS annual meetings also highlight notable differences but reveal similarity overall in the quality of evidence presented with little overlap between meetings. These findings do not suggest that one meeting is superior or redundant. Taken together, this study points to progress in the review processes used by the program committees, reaffirms the importance of critical appraisal when considering abstract findings, and supports the continued organization of multiple scientific meetings in musculoskeletal oncology.

Data Availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Conflicts of Interest

The authors report no conflicts of interest related to this work.

Authors’ Contributions

C. D. C and H. A. Q contributed equally to this publication.

References

J. J. Jasko, J. H. Wood, and H. S. Schwartz, “Publication rates of abstracts presented at annual musculoskeletal tumor society meetings,” Clinical Orthopaedics and Related Research, vol. 415, pp. 98–103, 2003.
View at: Publisher Site | Google Scholar
C. F. Preston, M. Bhandari, E. Fulkerson, D. Ginat, K. J. Koval, and K. A. Egol, “Podium versus poster publication rates at the orthopaedic trauma association,” Clinical Orthopaedics and Related Research, vol. 437, pp. 260–264, 2005.
View at: Publisher Site | Google Scholar
B. R. Williams, L. M. MacCormick, D. L. McCreary, A. J. Dugarte, and B. P. Cunningham, “Publication of podium presentations at the orthopaedic trauma association annual meetings,” Journal of Orthopaedic Trauma, vol. 32, no. 5, pp. e166–e170, 2018.
View at: Publisher Site | Google Scholar
T. Reuters, Journal Citation Reports®, Thomson Reuters Website, Toronto, ON, Canada, 2015.
L. Micallef and R. P. eulerAPE, “Drawing area-proportional 3-Venn diagrams using ellipses,” PLoS One, vol. 9, Article ID e101717, 2014.
View at: Publisher Site | Google Scholar
D. Amirhamzeh, M. A. Moor, K. Baldwin, and H. S. Hosalkar, “Publication rates of abstracts presented at pediatric orthopaedic society of North America meetings between 2002 and 2006,” Journal of Pediatric Orthopaedics, vol. 32, no. 2, pp. e6–e10, 2012.
View at: Publisher Site | Google Scholar
D. J. Donegan, T. W. Kim, and G.-C. Lee, “Publication rates of presentations at an annual meeting of the american academy of orthopaedic surgeons,” Clinical Orthopaedics & Related Research, vol. 468, no. 5, pp. 1428–1435, 2010.
View at: Publisher Site | Google Scholar
B. R. Williams, G. C. Kunas, J. T. Deland, and S. J. Ellis, “Publications rates for podium and poster presentations from the American orthopaedic Foot & Ankle society,” Foot & Ankle International, vol. 38, Article ID 1071100716688723, 2017.
View at: Google Scholar
M. Bhandari, P. J. Devereaux, G. H. Guyatt et al., “An observational study of orthopaedic abstracts and subsequent full-text publications,” The Journal of Bone and Joint Surgery-American Volume, vol. 84, no. 4, pp. 615–621, 2002.
View at: Publisher Site | Google Scholar
W. P. Hamlet, A. Fletcher, and R. A. Meals, “Publication patterns of papers presented at the annual meeting of the American academy of orthopaedic surgeons,” The Journal of Bone & Joint Surgery, vol. 79, no. 8, pp. 1138–1143, 1997.
View at: Publisher Site | Google Scholar
I. Harris, M. Mourad, A. Kadir, M. Solomon, and J. Young, “Publication bias in abstracts presented to the annual meeting of the American Academy of Orthopaedic Surgeons,” Journal of Orthopaedic Surgery, vol. 15, no. 1, pp. 62–66, 2007.
View at: Publisher Site | Google Scholar
A. M. Hollenberg, D. N. Bernstein, M.-J. Beltejar, T. Terry, and A. Mesfin, “Publication rate of podium presentations from the orthopaedic research society annual meeting,” Journal of Orthopaedic Research, vol. 37, no. 2, pp. 288–292, 2019.
View at: Publisher Site | Google Scholar
S. D. Kinsella, T. J. Menge, A. F. Anderson, and K. P. Spindler, “Publication rates of podium versus poster presentations at the American orthopaedic society for Sports medicine meetings,” The American Journal of Sports Medicine, vol. 43, no. 5, pp. 1255–1259, 2015.
View at: Publisher Site | Google Scholar
P. R. Melvin and L. K. Cannada, “Publication rates of posters from the orthopaedic trauma association annual meeting 2011-2016,” Journal of Orthopaedic Trauma, vol. 34, no. 4, pp. e121–e124, 2020.
View at: Publisher Site | Google Scholar
D. B. Murrey, R. W. Wright, J. G. Seiler 3rd, T. E. Day, and H. S. Schwartz, “Publication rates of abstracts presented at the 1993 annual Academy meeting,” Clinical Orthopaedics and Related Research, vol. 359, pp. 247–253, 1999.
View at: Publisher Site | Google Scholar
Q. Naziri, P. J. Mixa, D. P. Murray et al., “Adult reconstruction studies presented at AAOS and AAHKS 2011-2015 annual meetings. Is there a difference in future publication?” The Journal of Arthroplasty, vol. 33, no. 5, pp. 1594–1597, 2018.
View at: Publisher Site | Google Scholar
G. P. Slobogean, C. Dielwart, H. S. Johal, J. A. S. Shantz, and K. Mulpuri, “Levels of evidence at the orthopaedic trauma association annual meetings,” Journal of Orthopaedic Trauma, vol. 27, no. 9, pp. e208–e212, 2013.
View at: Publisher Site | Google Scholar
D. M. Lerman, M. G. Cable, P. Thornley et al., “Has the level of evidence of podium presentations at the musculoskeletal tumor society annual meeting changed over time?” Clinical Orthopaedics & Related Research, vol. 475, no. 3, pp. 853–860, 2017.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2021 Christopher D. Collier et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

276

Downloads

709

Citations