Abstract

Although several biomedical journals in India have crossed the 50 years mark in their publication history, not many have witnessed a commensurate increase in their performance metrics. Consequently, in comparison to the international journals with a high Impact Factor and successful publishing models, the national journals in India enjoy less traction. Due to this, the best data from Indian researchers almost always serves as fodder for journals published outside the country, even if there are high article processing charges involved. On the other hand, in order to promote Indian research globally, there is financial support available to the researchers to publish their results in big-banner journals. Against this background, we propose a generic mixed method approach (protocol) for Indian journals to undertake a gap analysis through participatory assessment, which may help generate critical solutions toward their future standing, visibility, promotional strategy, and performance. This generic approach is suggested to be adopted in 2 prongs: a quantitative trend analysis paired with a context specific qualitative inquiry, which would help triangulate perspectives of the managers, reviewers, as well as end-users, rather than being reliant on quantitative parameters only.

Introduction

As with any organization, quality performance by a biomedical journal and remaining relevant across genres depends largely on renewing its goals from time to time. This not only involves fresh target setting but also primarily identifying areas that could be improved.^1,2 Such a process facilitates positive growth. There are several long-lasting biomedical journals in India that have been publishing peer-reviewed research articles for a while. 

However, an analysis of the annual list released by Journal Citation Reports (JCR) in 2023³ revealed that there were only about 20 Science Citation Index Expanded (SCIE) journals that consistently had an impact factor of 1 or more over the prior 5 years, with at least over 1,000 citations.⁴ Contrastingly, quality research findings from within India are being published in high impact journals published from outside India, even against high article processing charges often provided by in-country research organizations. 

Noticeably, the COVID-19 pandemic brought an interesting change in the type and quality of articles received across all journals. First, during the pandemic, it was believed to be necessary to characterize the country-specific course of transition of the SARS-CoV2 virus to inform locally relevant interventions. Second, the urgency of the situation necessitated publishing emerging scientific evidence as early as possible and keeping it easily available in the public domain for wider engagement of the stakeholders. This, at times, was not possible through long publication processes associated with the international journals having different focus. Lastly, takers of research articles on “community voices”⁵ or innovations such as the “modelled projection of the third wave of COVID in India”⁶ were published in in-country biomedical research journals. Also, narrative reviews such as the one by Singhal⁶ captured early and emerging information about a new disease on the horizon gaining significant traction, with thousands of citations very early on, which no doubt significantly influenced the respective journal metrics. Such communications at the time of crisis, of course, were associated with some concerns as well.^8,9 However, these considerations, among many others, renewed interest in publishing crucial national level data in home-grown national journals.

Given this shift and a staggering increase in the number of submissions, and a sustained noticeability of some long standing journals in terms of peer citations, there have been drastic changes in journal metrics. For example, the Impact Factor of the Indian Journal of Medical Research¹⁰ (IJMR), which has been in a sustained publication mode since 1913, increased from 2.375 to 5.274 in 2021. 

Against this backdrop, we propose a critical approach for an internal performance audit that Indian journals in general, but low impact journals in particular, may undertake to assess their performances. The proposed mixed-method approach is based on a gap analysis followed by identification of solutions that could help journals to improve their performance as well as increase their visibility.  

Methods

Approach and Study Design

The proposed assessment would focus on the current situation pertaining to a journal to inform actionable solutions for intended consequence (pragmatism¹¹), rather than on antecedent conditions. We therefore propose a pluralistic approach in the form of a mixed-method design in which complementarity of quantitative and qualitative evaluation¹² would be harnessed. A journal intending to undertake such an exercise (study-journal) should constitute an assessment-team of in-house, as well as external, reviewers.

Trend Analysis 

This will provide a quantitative assessment of the study-journal and can be conducted based on secondary data analysis, on the journal’s backup database and online archival content. 

Several Indian biomedical journals went through a transition from a purely offline to an online manuscript submission/processing mode. To take into account this transition, data over the preceding 20 years (including pre- and postelectronic era) can be considered for analysis. For example, the shift to online mode in 2010 is treated as a central point and 3 blocks of 3 years each is demarcated for analysis, as shown in Figure 1. Starting with 2010, there may also be an overlap phase in which both online and offline submissions were received; an additional block of 3 years (2019–2022) can be considered such that there are at least 3 block points on each side of the graph for analysis of the trend. Furthermore, this extra 3 year block may help in analyzing the onset of the COVID-19 pandemic as another influencing factor to observe the trend.

Information regarding respondent categories, including authors (e.g., in regions of India, government/private institutions, etc.), can be extracted in order to observe the broad zonal shift in the number and type of submissions from various authors over time. This may provide a summative picture regarding shift in priorities for research and publication across the country over the years. Furthermore, parameters such as rejection/acceptance rates, etc., can also be determined.

The performance indicators that may be assessed temporally are as follows:

• Article types.   The shift in the different article types published in the journal of interest over the years may be mapped. For example, if the ratio of original articles (numerator) and research correspondences (denominator) decreases progressively, this may indicate predominance of shorter dataset studies, which may then be plotted against the article citations over the years. 
• Topics.   The subject shift can be examined against changing journal metrics as well. Typically, topics covered only under the current scope of the journal may be examined (e.g., communicable/noncommunicable versus Impact Factor of the years covering citations of such articles).
• Quantitative scores/metrics.   Bibliometrics across the chosen study period may also be noted. Journal/article metrics as reported by JCR common to the 3 phases of digital transformation as exemplified in Figure 1 may be compared. Other parameters, such as number of submissions received in each year, number of articles accepted/rejected/published, etc., can be recorded from the respective publisher/study-journal’s internal database, in addition to the time taken from submission to first decision or until publication/rejection, as delving into the life cycle of an article may provide information regarding the temporal gaps related to the decision-making process of the journal and its overall performance. 

Qualitative Assessment of the Journal’s Performance

In order to analyze the experiences of the submission or reviewing process and perspectives about a journal’s performance, preferably in person (otherwise through telephone/video conferencing), data should be collected using in-depth interview (IDI) guides and probes. Information obtained during trend analysis (quantitative component) can be used to identify respondent groups to be invited for qualitative inquiry. Furthermore, the trend analysis results will guide stratification of the respondent groups in the proposed assessment.  

The following respondent groups may be considered for qualitative inquiry:

• Readers.   This category may encompass postgraduate students of those institutions in defined geographical locations (e.g., North, South, East, West, Central region of the country) from where 2 or more articles were received over the past 5 years. The respondents include those who have never submitted an article to the study-journal, but aspire to do so and are familiar with it due to its access through their institutional libraries, or have known one of their seniors or guides/mentors who previously submitted or published their research in the study-journal.
• Authors.   This group could include first authors and/or corresponding authors who submitted 2 or more articles to the study-journal over the past 5 years, regardless of the publication status of the same. Subcategories of authors may also be considered based on the finding of the aforementioned trend analysis, including, but not limited to, the authors submitting articles less frequently, with differential experience of the peer review process with success or failure.   
•  Management.   This category of respondents may include the following:
  • Reviewers.   Those who have reviewed at least 10 manuscripts over the past 5 years for the study-journal. The subcategorization of this group of key informants may include, but not be limited to, the reviewers with differential participation in the review process and decision making.
  • Journal’s editorial board (past and present) members. 
  • Policy-makers.   This category may include key opinion leaders from the organization or the society hosting the journal (including senior scientists, organizational directors/members of national steering committees/ethics committee members, etc.) who may be familiar with the journal.  
  • Administrators.   This group may include senior individuals from within the hosting agency’s administrative chain and also past organizational heads who may have served as chair of the editorial board of the journal. 
  • Contemporary journal editors (national or international).   May also be considered in this category for IDIs.

Reduction of Potential Bias

The proposed protocol adopts a strategy for participatory assessment of the performance of the study-journal through engagement of external and internal researchers. The purpose of such an approach is to obtain both the insiders’ (Emic) as well as the outsiders’ (Etic) perspectives. This will help generate understanding about the overall functioning of the journal to be assessed by reducing potential bias and may help provide action points for enhancing quality outputs. As the key outcome indicator of this protocol is aligned with the purpose of this exercise (i.e., “assessment”), the investigators may undertake all necessary measures to ensure no potential conflict of interest during data collection as well as analysis. This will include responsible and consensual delegation of responsibilities and ongoing review of the assessment outputs. 

In addition to the common areas of exploration, exclusive domains/subdomains of inquiry tailored to each category of respondents should also be included in the IDI guides, such as “user experience,” “perceived gaps,” and “respondents” perspectives on the scope for improvement. So, publishing in the study-journal will not be the only criteria for inclusion of a participant. 

SWOT Analysis

Individuals associated with day-to-day functioning of the study-journal may be invited for a SWOT (Strengths and Weaknesses in the internal environment and Opportunities and Threats in the external environment) analysis. Two or 3 similar SWOT sessions may be organized based on the feasibility of assembling groups of individuals who would not be covered under IDI and would have insiders’ views regarding the journal’s management and review or publication process. The following premises could be adhered to while conducting SWOT sessions: 

• Strengths (S).   Parameters that inherently exist within the journal structure and influence its public standing in present day as well as in the future. “What the journal could be proud of” would constitute the central consideration under exploration of “strength.”
• Weaknesses (W).   Features, which would indicate inherent issues within the journal functioning, the journal’s internal structure, and areas to be improved to enhance its performance. “What are the areas that the journal could improve upon” would be the domain explored under this subhead.
• Opportunities (O).   These are external factors such as outside influences, policy changes, etc., which might have the potential to enhance the performance of the journal and how the journal could work through networking and advocacy efforts to reach a greater height.
• Threats (T).   These are internal or external factors, which might negatively influence the performance, as well as reputation, of the journal of which control might not be achieved.

Methodology for SWOT

Groups may be formed to collect feedback on various aspects of the journal’s performance and functionality. Groups of 4–6 participants from categories such as reviewers (who reviewed at least 5 articles in the past 5 years), past journal editors/editorial board members, scientific staff involved in technical support of the journal (past/present), key opinion leaders of the hosting agency who are familiar with the journal’s functioning as detailed earlier, administrative personnel, contemporary journal editors, etc., may be considered for inclusion as per feasibility.

Sample Size for Qualitative Inquiry

A geographical zone-wise selection of respondents may be carried out with about 10–15 respondents from each of the 5 chosen zones (e.g., North, South, East, West, Central region of the country) for IDI with “Readers.” About 10–15 respondents for the “Authors” and the “Reviewers” categories should be considered depending on the assessed interphase with the study-journal. The maximum limit for IDIs may be determined through attainment of the saturation point¹³ during qualitative investigation and as identified during ongoing analysis.

Implementation Plan

A schematic framework detailing the implementation plan has been provided as Figure 2.

Additional Source of Information to Be Used

During the study period, any spontaneous feedback received by the editorial office from authors, reviewers, readers, or policy/program influencers may also be recorded and synthesized in analysis as appropriate.

Participant Enrollment and Subsequent Steps 

End users/respondents may be identified for inclusion in the study on the basis of familiarity with the study journal. The process of identification of the respondents should be reliant on the outcome of the trend analysis as described earlier. The flow chart (Figure 3) presents a selection plan for readers.

Data Collection and Analyses

In addition to the trend analysis, a thematic content analysis may be carried out under this proposed mixed-method performance assessment proposal. Findings generated through trend and SWOT analysis should be triangulated against findings obtained from IDIs to create a comprehensive abstraction matrix.

Ethics Review

This performance audit should be ethically compliant given the inputs collected from human participants during the qualitative inquiry.¹⁴ A written consent should be obtained from the participants prior to IDIs or group discussions and for recording the responses.

Discussion and Conclusion

Long-term sustaining of a journal does not necessarily ensure quality nor relevance. There is an unmet need to understand the gaps in the processes, and redressal thereof, toward achieving better performance, as well as quality, of Indian journals. In this direction, analysis of gaps in a journal’s operations, existing priorities, and end users’ perspectives (authors, readers, reviewers, etc) may generate useful insights for improving its overall performance and in turn, enhance its capabilities. Analyzing, understanding, and eventually bridging these gaps may help achieve greater uptake by researchers and potential improvement in performance metrics.

The proposed mixed method protocol can be executed within a year. If implemented, the outcomes expected from this protocol are twofold: 1) a descriptive analysis depicting the change in publication profile of the study-journal, including change in Impact Factor over the prior 2 decades and 2) in-depth understanding of “good management practice” pertaining to the journal, as well as “what could be improved upon, pertaining to managing the Journal.” An example on similar lines can be obtained from a recent publication from India by Arjuman et al.⁴ The outcomes obtained following execution of this proposed protocol can be used directly to inform policy decisions regarding future practices of the journal and to decide on the way forward. As the proposed research is not reliant only on quantitative metrics, it is expected to provide information regarding the potential areas of functioning of the study journal for enhancing the journal’s overall achievement, visibility, and outreach, in which users’ perspectives will play an important role. Lastly, this protocol, although proposed in the context of Indian biomedical journals with low impact, can be easily adopted by any journal from any part of the world provided the right context and regional readership landscape are identified.

Acknowledgments

The authors acknowledge Dr Saritha Nair, Scientist, Indian Council of Medical Research, New Delhi, for participating in protocol-related discussions during the early phase of protocol development, in which the utility of SWOT was reflected on. This assessment protocol was reviewed and discussed by a technical advisory committee chaired by Dr DCS Reddy (Ex-Professor and HoD, PSM, IMS, BHU, and Retd. NPO, WHO-India) and members comprising Dr Beena E Thomas (Former Head, Socio Behavioural Research, ICMR-NIRT, Chennai), Dr Arvind Pandey (Former Director, ICMR-National Institute of Medical Statistics, New Delhi), Dr Vishnu Vardhan Rao (Former Director, ICMR-National Institute of Medical Statistics, New Delhi), Dr Siddhartha Shankar Ray (Ex-Librarian, ICMR-NICED, Ex-Librarian, ICSSR-Centre for Studies in Social Sciences, Kolkata, and Ex-Deputy University Librarian, Adamas University, West Bengal), and Dr Manoj V Murhekar (Director, ICMR-National Institute of Epidemiology, Chennai).

References and Links

1. Greve HR. Structuring the situation: organizational goals trigger and direct decision-making. Front Psychol. 2023;14:1140408. https://doi.org/10.3389/fpsyg.2023.1140408
2. Økland A. Gap analysis for incorporating sustainability in project management. Procedia Computer Science. 2015;64:103–109. https://doi.org/10.1016/j.procs.2015.08.469
3. https://impactfactorforjournal.com/jcr-impact-factor-2022/ 
4. Arjuman A, Roy P, Dasgupta R, Panda S. A qualitative inquiry on Indian biomedical Journals: Situation analysis and way forward. Indian J Pub Health. 2025;69:607–613. https://doi.org/10.4103/ijph.ijph_998_25
5. Kumar MS, Madhumathi J, Gayathri K, Rozario AAG, Vijayaprabha R, Balusamy M, Sonekar H, Panda S. Community voices around COVID-19 vaccine in Chennai, India: a qualitative exploration during early phase of vaccine rollout. Indian J Med Res. 2022;155:451–460. https://doi.org/10.4103/ijmr.ijmr_668_22
6. Mandal S, Arinaminpathy N, Bhargava B, Panda S. Plausibility of a third wave of COVID-19 in India: a mathematical modelling based analysis. Indian J Med Res. 2021;153:522–532. https://doi.org/10.4103/ijmr.ijmr_1627_21
7. Singhal, T. A review of Coronavirus Disease-2019 (COVID-19). Indian J Pediatr. 2020;87:281–286. https://doi.org/10.1007/s12098-020-03263-6
8. Arjuman A. A perspective on new dilemmas in scientific publishing in the time of a pandemic. Indian J Med Microbiol. 2023;45:100366. https://doi.org/10.1016/j.ijmmb.2023.100366
9. The Lancet Infectious Diseases. The COVID-19 infodemic. Lancet Infect Dis. 2020;20:875. https://doi.org/10.1016/s1473-3099(20)30565-x
10. Cherryholmes CH. Notes on Pragmatism and scientific realism. Educ Res. 1992;21:13–17. https://doi.org/10.2307/1176502
11. Patton MQ. Qualitative evaluation and research methods, 2nd ed. Sage; 1990.
12. Green J, Thorogood N. Qualitative methods for health research. Sage; 2000.
13. Mathur R, ed. National ethical guidelines for biomedical and health research involving human participants 2017. Indian Council of Medical Research; 2017. [accessed March 23, 2026]. https://ethics.ncdirindia.org/asset/pdf/ICMR_National_Ethical_Guidelines.pdf.
14. Walker J, Holloway I, Wheeler S. Guidelines for ethical review of qualitative research. Res Ethics Rev. 2005;1:90–96. https://doi.org/10.1177/174701610500100304 

Albina Arjuman, Executive Editor, Indian Journal of Medical Research, and Scientist, Indian Council of Medical Research, New Delhi. Samiran Panda, Former Editor‐In‐Chief, Indian Journal of Medical Research. 

Opinions expressed are those of the authors and do not necessarily reflect the opinions or policies of their  affiliated journal or its hosting agency, the Council of Science Editors, or the Editorial Board of Science Editor.