Jointly published by Akadémiai Kiadó, Budapest and Kluwer Academic Publishers, Dordrecht
Scientometrics, Vol. 53, No. 1 (2002) 39–48
Significant Mexican research in the health sciences: A bibliometric analysis JUDITH LICEA DE ARENAS,1 HERIBERTA CASTAÑOS-LOMNITZ,2 JUDIT ARENAS-LICEA3* 1
Faculty of Philosophy and Letters, National University of Mexico, Mexico City (Mexico) Institute of Economics Research, National University of Mexico, Mexico City (Mexico) 3 Department of Biochemistry and Molecular Biology, University College London, London (UK) 2
In the 1970s Mexico started to consolidate its S&T system by training human resources and actively preventing brain drain, mainly by motivating researchers through economic incentives. Considering Bradford’s Law, an analysis of significant Mexican research in the health sciences, i.e., papers published in journals with a high-impact factor which grant a degree of credibility and importance was carried out. Significant papers produced in Mexico show a measure of the country’s productivity, and these papers’ citations measure the country’s international impact.
Introduction Thirty years ago Mexico started to build up its science and technology (S&T) system since S&T are important components of development. Several events have taken place since: • The body responsible for S&T, the National Council for Science and Technology, Consejo Nacional de Ciencia y Tecnología (CONACYT), was created in the 70s. • A human resources recruitment programme, at undergraduate level, has been in action since the 70s. Thousands of Mexicans have benefited from scholarships awarded by the government of Mexico and have obtained their degrees in countries such as the USA, the UK, Spain and France. • The National Researchers System, Sistema Nacional de Investigadores (SNI), was founded in the 80s to motivate productive researchers through economic incentives. • Mexican postgraduate programmes were strengthened partly as a result of a shortage in hard currency.
*
Present address: Coalition to Stop the Use of Child Soldiers, 436 Essex Rd., London N1 3QP, UK.
[email protected]
0138–9130/2002/US $ 15.00 Copyright © 2002 Akadémiai Kiadó, Budapest All rights reserved
J. LICEA DE ARENAS et al.: Mexican research in the health sciences
•
In the 90s competitive positions, especially at academic and government research centres, were opened for those trained abroad in order to prevent brain drain.
Nevertheless, circumstances remain difficult for scientists whatever the government’s policies. Although Mexico’s social and economic development seems to be faltering, Mexico’s science exerts both domestic and international influence. Indicators of productivity report that Mexico published, according to the Institute for Scientific Information (ISI) databases, 3,693 papers in 1995 and 4,129 in 1996.1 A Mexican source reports that the country published, again according to ISI, a total of 3,572 papers in 1997. 2 Several studies have carried out bibliometric analysis of Mexican research, however, most of them have focused on the health sciences.3-5 While some counted publications, others have counted papers retrieved from the ISI databases. From these studies it is apparent that productivity has been concentrated in areas such as biomedical and clinical medicine research. Considering that “. . . a small number of journals account for the bulk of significant scientific results”,6-7 a principle often referred to as Bradford Law8 and that Bradford realised that the core literature for any given discipline was composed of fewer than 1,000 journals,9 an analysis of significant Mexican research remained to be carried out. It would be expected that the number of significant papers produced in Mexico show some measure of the country’s productivity, and that the number of citations received by these papers give a measure of the country’s international impact. We therefore, aimed to identified significant health sciences research and to relate data on levels of research output, parent institution, geographic and discipline distributions, and citation impact. Methods The primary data sources were Garfield’s “Long-term vs Short-term impact” reports.10-11 These lists of the 200 journals with the highest cumulative impact, based on 15 years of data, take into account the fact that a journal’s impact factor helps to evaluate the relative importance of journals and ultimately the research that is published in them. A total of 153 journal titles are health sciences-related. PubMed was accessed via the World Wide Web (http://www.nlm.nih.gov) and MEDLINE was searched on each health sciences-related journal title with Mexico in the institution search field. We identified 1,025 papers authored by Mexicans or foreigners for the period 1990-1998 (for convenience, institutional affiliation was equated with Mexican nationality).
40
Scientometrics 53 (2002)
J. LICEA DE ARENAS et al.: Mexican research in the health sciences
In addition, we searched the printed edition of Science Citation Index (1990-1997) and the online version of Science Citation Index Expanded available in the Web of Science (1998-1999) in order to determine the number of cited papers and the number of citations each paper attracted. Both the papers retrieved from MEDLINE and those cited were grouped by category according to ISI Journal Citation Reports. Results Over the nine year period of this study there was little variation in the level of research activity among Mexican scientists, with annual output running at approximately one hundred papers. When we looked at the editorial category of papers it became apparent that Mexican researchers mainly publish original research articles (Table 1). Research output is, however, highly skewed, since more than 75% of the total publications were concentrated in Mexico City. Thirteen states produced less than 10 papers, without counting the zero producers (14 states) (Table 2). From Table 3 it is clear that the majority of research is being carried out by academic institutions (76%), followed by the public health sector (21%). The remainder is spread out amongst private health and higher education institutions or international organisations (e.g., Pan American Health Organization). When we turn to the relationship between the output published in Mexico City or by academic centres, they may be the primary, but not necessarily the most efficient producers since they are influenced by the Matthew effect. Mexican authors publish in a wide range of journals distributed across 6 countries, with the USA being the most important publishing country (Table 4), as well as the country where most researchers were trained. The scatter across journals is striking: 63% of the journals carried three or more articles. Mexican scientists published 1,025 significant papers in 124 journals distributed across 39 subfield categories where hot fields such as Biochemistry & Molecular Biology and Neurosciences were the most active. The quality of Mexico’s research effort was also gauged by using weighted output measures since the publication counts may also say something regarding activity. In addition, the relative quality of source journals became apparent as one partial indicator was combined with citation counts. In terms of the significant journals’ impact factor, 82% of the papers were published in journals with impact factors ranging from 20 to 40 (Table 5).
Scientometrics 53 (2002)
41
J. LICEA DE ARENAS et al.: Mexican research in the health sciences
Table 1 Type of publication Type Article Review Letter
No.
%
1015 9 1
99.02 0.88 0.09
Table 2 Geographic distribution of papers State
No.
%
Mexico City Morelos Guanajuato Nuevo Leon Colima Jalisco Mexico Puebla San Luis Potosi Veracruz Baja California Queretaro Tlaxcala Yucatan Baja California Sur Chihuahua Durango Sonora Zacatecas
769 139 27 16 12 11 9 9 7 6 4 4 4 2 1 1 1 1 1
75.02 13.56 2.63 1.56 1.17 1.07 0.88 0.88 0.68 0.58 0.39 0.39 0.39 0.19 0.09 0.09 0.09 0.09 0.09
1
0.09
Unidentified
Table 3 Author’s institutional affiliation Institution
No.
%
Academic Government Academic, private Health, private International organisations Research Research, private
778 212 15 7 5 3 2
75.90 20.69 1.46 0.68 0.49 0.3 0.19
Other
42
3
0.3
Scientometrics 53 (2002)
J. LICEA DE ARENAS et al.: Mexican research in the health sciences
Table 4 Country of publication of source journals Country
No.
%
US Netherlands UK Germany Ireland Switzerland
517 240 119 74 66 9
50.43 23.41 11.61 7.22 6.44 0.88
Table 5 Impact factor in terms of productivity IF 20-40 40-60 60-80 80-100 >100
No. papers 844 143 23 12 3
The 854 papers, out of 1,025 (i.e. 171 were uncited), that were retrieved from MEDLINE attracted 8,625 citations. Disciplines such as Biochemistry & Molecular Biology and Neurosciences were the most cited as well as the fields that published most of the papers. However, citations per paper were higher in Medicine, General and Internal ; Rheumatology ; Multidisciplinary Sciences, and Haematology (Table 6). Brain, Neuroscience Letters, Journal of Bacteriology, and American Journal of Physiology were among the journals that attracted the highest number of citations (Table 7) and published the largest number of highly cited papers (Table 8). Five institutions, two academic and three government research centres, ranked by citations, accumulated the greatest number of citations (Table 9). The above mentioned institutions accumulated (80%) of all citations to significant Mexican papers (1990-1998) and 78% of all cited papers.
Scientometrics 53 (2002)
43
J. LICEA DE ARENAS et al.: Mexican research in the health sciences
Table 6 The 39 disciplines of significant publications
44
Rank
Disciplines
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 20 21 22 23 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37
Biochem & Mol Biol Neurosciences Physiology Microbiology Pharmacol & Pharmacy Medicine Gen & Internal Medicine Res & Exp Multidisc Sciences Endocrinol & Metabolism Virology Biotechnol & Applied Micro Rheumatology Immunology Peripheral Vasc Diseases Oncology Public, Environ Infectious Diseases Nutrition & Dietetics Haematology Cell Biology Paediatrics Clin Neurology Develop Biology Cardiac & Cardiovasc Plant Sciences Gastroenterol &Hepatol Respiratory System Obtetrics & Gynaecol Biology Reprod Biology Radiology Psychiatry Biophysics Medical Lab Pathology Anaesthesiology Urology & Nephrology Surgery Genetics &Heredity
Papers 258 184 41 46 44 11 35 12 28 14 12 6 17 17 6 6 6 8 4 10 6 10 6 16 9 4 2 8 3 6 1 3 2 2 3 1 1 3 3
Citations 2221 1857 489 469 361 352 286 255 239 202 164 155 130 125 118 114 98 94 91 87 87 71 70 68 68 58 52 45 39 29 22 21 18 15 14 13 11 10 7
Citations per paper 8.6 10.09 11.92 10.19 8.20 32 8.17 21.25 8.53 14.43 13.66 25.83 7.65 7.35 19.66 19 16.33 11.7 22.7 8.7 14.5 7.1 11.66 4.25 7.55 14.5 26 5.62 13 4.83 22 7 9 7.5 4.66 13 11 3.33 2.33
Scientometrics 53 (2002)
J. LICEA DE ARENAS et al.: Mexican research in the health sciences
Table 7 Journals that accumulated the greatest number of citations Rank 1 2 3 4 5
Journal
No. papers
Brain Res J Bacteriol Neurosci Letters Am J Physiol Biochim Biophys Acta
59 42 40 25 43
Citations 636 442 375 343 308
Table 8 Journals that published the greatest number of highly cited papers (20 citations or more) Rank
Journal
Papers
1 2 3 4 5 6 7 8 9 10
Brain Res P Natl Acad Sci Am J Physiol Neurosci Lett J Bacteriol Lancet J Exp Med Appl & Environ Microb Arthritis & Rheum J Neurosci
10 6 5 3 2 1 1 3 1 2
Citations 285 201 172 155 142 137 121 113 109 96
Table 9 Top cited institutions (ranked by total citations to significant papers) Rank 1 2 3 4 5
Institution
Cited Papers 1990-1998
Citations
419
4022
45 158 35 9
1676 725 257 242
National University Centre of Research and Advanced Studies Institute of Nutrition Institute of Cardiology Institute of Public Health
Discussion Today’s rapid advances in science mean that communication of knowledge gives rise to new forms of competition. More than ever, it is essential to monitor trends and structural shifts in science since Mexico has consolidated its S&T system.
Scientometrics 53 (2002)
45
J. LICEA DE ARENAS et al.: Mexican research in the health sciences
In this paper we examined how Mexican authors have challenged publication and citation issues. The indicators we built reflect significant research performance and impact which allowed comparisons between institutions, states and disciplines and showed how well prepared the country is for the future, as well as identifying areas where greater efforts will be needed. The indicators implicitly reflected the ability of Mexican authors to publish in the best journals and become internationally visible. However, we expected a growing contribution to the world’s science considering that a large proportion of Mexican researchers were trained in the USA and European countries and they supposedly maintain their connections with the country of origin of their alma mater.12 We do not agree with Sancho13 in that less developed countries (LDC) are misrepresented in the ISI databases and that, therefore, conclusions cannot be accurate. Journals from LDC countries are not indexed by ISI because their content do not conform to the way science has to be carried out. Thus, we propose that LDC publications should be recorded in a national bibliography for future reference. Consequently, bibliometric analyses should distinguish what they are measuring whether simple papers or rather original research articles published in mainstream journals. It is true that science policies in Mexico imply that the only worthwhile scientific activity lies far to the north. Thus, success in research careers tends to be strongly correlated to the individual scientist’s publishing record: the higher the journal’s impact, the greater the recognition awarded by the local scientific community and ultimately by the government. Nevertheless, scientists should be evaluated by considering the purpose of the impact factor of journals14-15 and the editorial categories of items to be evaluated, for example, original research papers, reviews and letters, among others.16 Significant research activity was shown to be highly skewed in geographic, institutional and disciplinary terms. Fields such as Biochemistry & Molecular Biology, and Neurosciences concentrated most of the significant research, in a pattern similar to that of Brazil.17 Since Mexico has paid attention to higher education, universities attracted the largest share of human resources trained as researchers. The role of public higher education institutions became outstanding in contrast with private universities; the same pattern was revealed in a paper published in the 90s18 where the authors compared both type of institutions in terms of research activity. The pre-eminence of Mexico City,19 the country’s capital, is not entirely surprising: the megalopolis is the centre of all aspects of political, economic and cultural activity in the country. It is debatable whether such concentration is in the best interest of the country as a whole. Thus, it is necessary to know if the states that were low or zero
46
Scientometrics 53 (2002)
J. LICEA DE ARENAS et al.: Mexican research in the health sciences
producers have sufficient critical mass and autonomy to enable them to sustain high quality research and whether the absence of a strong research presence in so many states is appropriate. On the other hand, if “. . . to be an uncited scientist is no cause for shame”,20 Mexican researchers conform to the norm: the papers published in high-impact journals have a greater chance of attracting citations than those published in lower-impact journals. It should be taken into account that given the number of uncited papers in our sample (n=171 out of 1,025), their performance is similar to that of the most prolific scientist whose papers are heavily cited, less cited or uncited. Few papers of the most cited authors in our study constitute what Garfield 21 calls citation classic, however, despite awards honouring Mexican authors of highly cited papers by ISI,22 future research activity of the top researchers should be traced. Although health sciences research has been criticised for its gender bias23 in this paper we did not identify female researchers due to database limitations (e.g., use of initials). In the near future, however, a study of the performance of women in these sciences should be attempted. Our findings suggest that: • Scientific expertise in Mexico is strongly clustered in a few institutions in and around Mexico City. • Mexican authors have evidently been successful at targeting high-impact journals. • Mexican institutions, though possibly peripheral to international scientific communities in terms of funding, manage to participate in the worldwide research effort. Our conclusions are based on the assumption that journal impact factors are correlated with accumulated productivity and that the number of citations is related to the visibility of a given contribution. Nevertheless, a substantial part of the scientific enterprise remains irrelevant in terms of urgent health needs of Mexican society, i.e. the internal brain drain.
* The authors are grateful to Ludy Pérez, Horus Ruiseco and Javier Valles for their assistance. J. ArenasLicea was supported by a grant from the National Council for Science and Technology, Mexico (grant number 9303).
Scientometrics 53 (2002)
47
J. LICEA DE ARENAS et al.: Mexican research in the health sciences
References 1. Science & Technology Indicators: Ibero American 1995-1996. Red Iberoamericana de Indicadores de Ciencia y Tecnología, Buenos Aires, 1999. 2. Indicadores de actividades científicas y tecnológicas 1998. CONACYT, México, 1999. 3. J. L. DE ARENAS, Aligning research activity with national priorities: a case study of Mexican health sciences research. In: B. CRONIN, (Ed.), The Knowledge Industries: Levers of Economic and Social Development in the 1990s. ASLIB, London, 1999, pp. 231–239. 4. J. L. DE ARENAS, Partial assessment of Mexican health sciences research, 1982-1986, Scientometrics, 23 (1992) 47–55. 5. J. L. DE ARENAS, J. VALLES, M. ARENAS, Profile of the Mexican health sciences elite: a bibliometric analysis of research performance, Scientometrics, 46 (1999) 539–547. 6. E. GARFIELD, What is a significant journal? Essays of an Information Scientist, 1 (1962-1973) 106–107. 7. E. GARFIELD,The significant scientific literature appears in a small core of journals, The Scientist, 10 (17) (1996). 8. J. TESTA, The ISI database. The journal selection process. Available from: URL: http:///www.isinet.com/whatshot/essays/esay9701.html 9. C. BRADFORD, Documentation. Public Affairs Press, Washington, D.C., 1950, 156 p. 10. E. GARFIELD, Long-term vs short-term journal impact: Does it matter? The Scientist, 12 (3) (1998) 11–12. 11. E. GARFIELD, Long-term vs. short-term impact. Part II, The Scientist, 12 (14) (1998) 12–13. 12. Pulling together in Latin America, Nature, 398 (1999) 353. 13. R. SANCHO, Misjudgements and shortcomings in the measurement of scientific activities in lessdeveloped countries, Scientometrics, 23 (1992) 221-223. 14. E. GARFIELD, How can impact factors be improved? British Medical Journal, 313 (1992) 411–413. 15. P. SEGLEN, How representative is the journal impact factor? Research Evaluation, 2 (1992) 143–149. 16. E. GARFIELD, Random thoughts on citationology: its theory and practice, Scientometrics, 43 (1999) 69–76. 17. J. C. R. PEREIRA, M. M. L. ESCUDER, The scenario of Brazilian health sciences in the period of 1981 to 1995, Scientometrics, 45 (1999) 95–105. 18. J. L. DE ARENAS, B. CRONIN, The contribution of higher education institutions to the development of Mexican health sciences base, Journal of Information Science, 15 (1989) 333–338. 19. B. CRONIN, J. LICEA DE ARENAS, The geographic distribution of Mexican health sciences research, Scientometrics, 17 (1989) 39–48. 20. E. GARFIELD, To be an uncited scientist is no cause for shame, The Scientist, 5 (6) (1991) 12. 21. E. GARFIELD, Introducing Citation Classics: the human side of scientific reports, Essays of an Information Scientist, 3 (1980) 1–2. 22. ISI recognizes highly cited Mexican authors. Available from: URL: http://www.isinet.com/isi/newsfeatures/mexico/unam.html 23. K. ALEXANDERSON, An assessment protocol for gender analysis of medical literature, Women & Health, 29 (1999) 81–98.
Received February 9, 2001. Address for correspondence: JUDITH LICEA DE ARENAS Faculty of Philosophy and Letters, National University of Mexico Ciudad Universitaria, 04510, Mexico City, Mexico E-mail:
[email protected]
48
Scientometrics 53 (2002)