Abstract Background In the healthcare system, Teaching Hospitals(THs) not only provide care, but also train healthcare professionals and carry out research activities. Research is a fundamental pillar of THs’ mission and relevant for the healthcare system monitored by Performance Evaluation Systems. Research activities can be measured using citation index services and this paper highlights differences between two services based on bibliometrics, describes opportunities and risks when performance indicators rely on data collected, controlled and validated by external services and discusses the possible impact on health policy at a system and provider level. Methods A bibliometric analysis was done on data between 2014-2016 from ISI Web of Science and Scopus of 18.255 physicians working in 26 Italian THs. Quantity was defined as the number of publications and quality as Impact Factor or Field-Weighted Citation Impact. Results Overall, 41.233 and 66.409 documents were extracted from respectively ISI Web of Science and Scopus. While benchmarking results, significant differences in ranked position both in metrics emerged. Discussion Utilizing secondary data sources to measure research activities of THs allows benchmarking at an (inter)national level and overcoming self-referment. To utilize indicators for multiple governance purposes at the system and provider level, indicators need to be profoundly understood, require formalizations in data validation, internal analysis and a sharing process among health professionals, management and policymakers. Abbreviations IRPESInter-Regional-Performance Evolution systemLHALocal Health AuthorityPESPerformance evaluation systemTHTeaching hospitalWoSISI Web of Science Keywords Teaching hospitalsResearch productivityPerformance evaluationImpact factorField-Weighted Citation Impact

Measuring research in the Big Data Era: the evolution of Performance Measurement Systems in the Italian Teaching Hospitals

Horenberg F;Lungu DA;Nuti S
2020-01-01

Abstract

Abstract Background In the healthcare system, Teaching Hospitals(THs) not only provide care, but also train healthcare professionals and carry out research activities. Research is a fundamental pillar of THs’ mission and relevant for the healthcare system monitored by Performance Evaluation Systems. Research activities can be measured using citation index services and this paper highlights differences between two services based on bibliometrics, describes opportunities and risks when performance indicators rely on data collected, controlled and validated by external services and discusses the possible impact on health policy at a system and provider level. Methods A bibliometric analysis was done on data between 2014-2016 from ISI Web of Science and Scopus of 18.255 physicians working in 26 Italian THs. Quantity was defined as the number of publications and quality as Impact Factor or Field-Weighted Citation Impact. Results Overall, 41.233 and 66.409 documents were extracted from respectively ISI Web of Science and Scopus. While benchmarking results, significant differences in ranked position both in metrics emerged. Discussion Utilizing secondary data sources to measure research activities of THs allows benchmarking at an (inter)national level and overcoming self-referment. To utilize indicators for multiple governance purposes at the system and provider level, indicators need to be profoundly understood, require formalizations in data validation, internal analysis and a sharing process among health professionals, management and policymakers. Abbreviations IRPESInter-Regional-Performance Evolution systemLHALocal Health AuthorityPESPerformance evaluation systemTHTeaching hospitalWoSISI Web of Science Keywords Teaching hospitalsResearch productivityPerformance evaluationImpact factorField-Weighted Citation Impact
2020
File in questo prodotto:
File Dimensione Formato  
1-s2.0-S0168851020302554-main.pdf

accesso aperto

Licenza: Licenza non conosciuta
Dimensione 641.08 kB
Formato Adobe PDF
641.08 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11382/534791
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 7
social impact