Developing an instrument for assessing determinants of data use in evidence-based decision making: a principal component analysis at public primary health centres in Haryana, India

INTRODUCTION

Information holds high value in a health system by facilitating evidence-based decisions which culminate to strengthening of the health care delivery system and providing improved health services ultimately leading to the achievement of national and sub-national health objectives.¹ Evidence-based decisions are strongly linked to better health outcomes leading to effectively detecting problems, defining priorities, identifying innovative solutions, rational resource allocation, improved transparency, ensuring accountability, better health planning and efficient management of programs.² Hence, evidence-based decisions based on the harnessed information are the pressing need of the hour specially in low- and middle-income countries (LMICs).³

According to performance of routine information system management (PRISM) framework, factors govern health information systems performance and are determinants of data informed decisions. These comprise of ‘technical factors’ pertaining to quality of data, design, technologies, methodologies, procedures, tools, and instruments of the system; ‘organizational factors’ comprising of structure, roles, functions, and responsibilities, as well as the culture of information of key actors and users at each level of the health system and ‘behavioural factors’ which include knowledge, skills, attitudes, values, and motivation of the individuals that collect and use data. The concept of individual constraints was introduced in place of behavioural factors wherein both behaviour and skills of individuals to use information were separately considered. Hence, individual factors can be divided into two subgroups – ‘individual behavioural’ - “the behaviors of data users and how data are used for problem-solving and program improvement” and ‘individual technical’ - “knowledge, skills, attitudes, values, and motivation of the individuals that collect and use data”.⁴

In India, primary health centres (PHCs), managed by frontline manager - a medical officer in charge (MOIC), form the foundational tier of public healthcare delivery in rural areas, providing essential primary care services at the grassroots level. Apart from the clinical duties, the administrative responsibilities of the MOIC of a PHC include supervision of staff, scrutinizing of programs, holding of monthly meetings to evaluate progress to deliberate upon steps for improvement, and capturing health information.⁵ National guidelines emphasizes, “A PHC medical officer should ensure timely submission of updated monthly reports and records for program monitoring and strategic planning; utilize records to undertake population-based analytics, and planning of activities for the primary health care team”.⁶

Hence, to promote evidence-based decision making at the grassroots level, it is imperative to consider various organizational, technical, and individual factors that determine data use. Specific technical, individual, and organizational activities can then be implemented to improve demand for, analysis, review, and use of routine health data in decision-making. An investigative tool was utilized in a previous study of three Indian states (Rajasthan, Maharashtra and Uttar Pradesh) which included numerous variables under each factor and hence was challenging for further use.⁷ The factors should be elaborate, encompass all possible aspects and yet should be available in concise, comprehensive, and manageable form. The current study and analysis aims to formulate an instrument/tool with comprehensive factors to assess determinants of data use for evidence-based decisions at public primary health centres in Haryana, India. We applied a dimension-reduction technique - principal component analysis (PCA) to reduce the large set of variables to a concrete workable set of correlated determinants/factors that still contained most of the information of the large set. The analysis describes the reduction process and the components/factors generated thereafter.

METHODS

Design and setting

The cross-sectional and analytical study was conducted in six selected districts of Haryana, India based on maternal and child health indicators - full ante natal checkup rate and immunization rate.^8,9 Maximum variation/ heterogenous purposive sampling technique was used wherein MOICs at all peripheral primary PHCs in the district were selected. Only those medical officers were included in the study who were ‘in charges’, medical officers who were not ‘in charges’ at PHC level or were posted at any other facility were excluded. Data was collected from 120 MOICs at the facilities where they were posted by trained investigators through a pretested semi-structured interview schedule adapted from the PRISM framework and a previous study.⁷ The interview schedule comprised of several elements under organizational, technical, and individual domain of determinants and each interview lasted for about 45-60 minutes. The data collection and analysis for the study were conducted over the period from December 2021 to May 2023.

Analysis

Initially organizational, technical, and individual factors were elaborate and comprised of numerous elements, therefore, to reduce and transform the large set of these independent variables into manageable, comprehensible components/factors, PCA was performed to extract components/factors with varimax orthogonal rotation. PCA has been defined as “a technique used to emphasize interpretability, commonly applied to systematically reduce the number of dimensions needed to describe datasets, but at the same time minimizing the loss of information”.¹⁰ Varimax rotation was used which generated factors/components uncorrelated to each other and facilitates the factors to better fit the data and has been defined as “varimax rotation is an orthogonal rotation method usually used for making the construct factors simple and explainable”.¹¹ The data suitability was checked by testing assumptions. The sample size should be more than 100 and in the current study was 120, hence further assumptions were tested.¹² Sampling adequacy was checked by Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy (>0.6) and factorability of the correlation matrix was assessed by Bartlett's test of sphericity (significance levels< 0.05).¹³ The cut off value for factor loading was 0.5, scree plot was examined, and factors were generated for organizational, technical, individual behavioural and individual technical domain.⁸ Since the data elements for individual technical factor were less, only three factors were generated. All the components generated had eigen values of more than one and the components/factors generated were named based on the interpretation of the variables/data elements they comprised of. The factor scores were generated by using regression method for maximum validity. Lastly, all the factors generated were tested for internal consistency by generating the reliability factor (Cronbach’s alpha). All analyses, including PCA and reliability testing, were performed using statistical package for the social sciences (SPSS) version 22.0.

RESULTS

Descriptive characteristics

The sample comprised of 120 MOICs positioned at the PHC level in 6 districts and 28 blocks. Most respondents (75%) were based in rural facilities, while the rest (25%) were stationed in urban facilities. Most MOICs (80%) were doctors in medicine, with the remainder being dental surgeons. Only 18 respondents (15%) had pursued higher qualifications or postgraduate degrees. The average work experience at the current position of MOICs was 4.8 years and in the health system was 7.6 years.

Reliability and validity of the instrument

Kaiser-Meyer-Olkin measure of sampling adequacy for organizational, technical, individual behavioural and individual technical factors was greater than 0.6 indicating that the sample size was large enough to assess the factor structure and the items were compact and correlated. Bartlett's test of sphericity had significance levels less than 0.05 for all factors, indicating that the data were sufficient to proceed for the factor analysis and factors were generated. The reliability factor – Cronbach’s alpha was greater than 0.7 for all factors as illustrated in Table 1.

Extraction of factors for the instrument

The final instrument comprised of three categories of determinants –organizational factors, technical factors, and individual factors. The 54, 45, 45, and 10 variables under organizational, technical, individual-behavioral, and individual-technical domains were reduced by using 0.5 as a cut-off value. The scree plot was examined, and eight organizational, eight technical, eight individual-behavioral, and three individual-technical factors, each having eigenvalues greater than one, were generated. The correlated factors were titled based on the comprehension of the consisting variables/data elements.

Organizational factors

For organizational factors, eight factors explained 60.5% of the total variance. The number of variables loading on to the factor, details of the variables and their loading value is explained in Table 2.

Technical factors

Eight technical factors were generated and accounted for 59.8% of the total variance. Table 3 explains the number of variables loading on to the factor, details of the variables and their loading value.

Individual factors

A total variance of 57.7% was explained by eight individual behavioural factors and 68% of variance was explained by three individual technical factors. The number of variables loading on to the factor, details of the variables and their loading value is explained in Table 4.

DISCUSSION

Main findings

The present study developed an instrument comprising of comprehensive, manageable, inclusive, and workable factors for each of the three organizational, technical, individual (behavioural and technical) determinants of health information systems performance and data use which can be used as a tool for future assessments. Through principal component analysis 154 correlated variables were clustered and transformed into a 27 uncorrelated variables/factors called ‘principal components’. Our method was rigorous as we used 0.5 as the cut off value for factor loading and conducted visual inspection of scree plot suggesting of strong correlation within a factor and generation of relevant factors.^14,15 Reliability factor of 0.70 was assumed good as all domains had more than ten items and 88.9% factors had value of reliability factor more than 0.70.¹⁶

Comparison with previous studies

Our study introduces a novel instrument with elaborate, yet comprehensive factors designed to assess the determinants of data utilization in public health decision making. To our knowledge, this is the first research to generate these factors and develop such an instrument, filling a critical gap and providing a foundation for future studies to build upon in assessing and enhancing data-driven practices in public health.

Our instrument is robust not just because of the methods used for generating it but because of the inclusive and expansive variables which capture vital information and yet in a manageable form. The clustering of variables and the generated factors are relevant as they measure diverse and extensive dimensions of data use. Amongst organizational factors, essential factors like external stakeholder’s influence captured the extent of influence of external stakeholders comprising of international, national and subnational stakeholders. This factor holds relevance as external stakeholder influence leads to collective decision making and designing of effective public health interventions.¹⁷

Organizational factor generated around management meetings whether with superiors, subordinates or follow up mechanisms include data-based review of performance and data presentation during meetings and serves as opportunity for staff at different level of hierarchies to congregate for common goals and programmatic actions. The factor exhibits applicability since previous studies emphasized the importance of a meeting platform for two-way communication, data use and sharing of information, dissemination of knowledge, discussion on ideas, evidence-based review of targets and performance, consensus on future plans and gap analysis.¹⁸ Exploring data oriented and conducive culture incorporated organizational culture of looking at outputs and outcomes, feedback mechanism, timeliness of data, and perception of data quality as evidence suggests that a conducive and supportive environment is as an immutable enabler for evidence-based decision making.^4,19 Additionally, while decision making rests with the individual policy-makers, program managers, and other implementers, these individuals operate within the context of a system/organization with processes that directly impact the ability and extent to which evidence can be used as part of the decision making process.²⁰ Organizational processes for data use can be strengthened by leadership adopting and supporting evidence-based decisions and hence this factor also exhibits applicability and relevance.²¹

The technical factors generated in the instrument related to training and skill building for using routine health information systems, basic software and advanced analytical packages are pertinent since training received on the information systems adds to knowledge and abilities for accessing and utilizing data.²² Our findings also emphasize measuring data quality as focusing on data quality is crucial, without which there is reluctance to use information for decision making.^23,24 Studies also reveal that if data quality is perceived to be satisfactory, it leads to increased data use for decision making and adds to further improvement in the quality of data, this creates a cycle of improved information, demand of data and information use eventually leading to improved health programs and policies.²⁵

Technical robustness, availability of IT infrastructure and procedure for maintenance including ensuring the availability of reports at all levels, implementation of simple and comprehensible software at all levels, availability and access of computer hardware, establishment of uniform data reporting and feedback mechanism, troubleshooting and maintenance procedure are vital factors for functioning of information systems and data utilization for decisions.^4,26 Information adequacy as a factor needs to be explored to assess sufficiency of information and absence of data duplication. Duplicity of information has been described as a deterrent to data use by authors in Indian and global public health context.^22,27

The individual factors encompassing both behavioural and technical factors were apt and relevant as it explored the individual aspect of data use for decisions. The health personnel using data are often involved in implementation of multiple health programs, dispensing their clinical and administrative services and thus have an expanded list of roles and responsibilities.⁵ This calls for support of data for quick and efficient evidence-based decision making enabling simultaneous management of multiple programs and hence indicates towards the appropriateness of the engendered factor. Factors on training needs on ensuring good quality data, understanding routine data, using data base software, applying statistical analysis techniques and communication and presentation of data is an indication towards being motivated and positive towards capacity building for data understanding and utilization. In absence of such motivation for training, evidence-based programmatic decision making is less likely and routine data is not used for planning by grass root workers as has been asserted by authors in Indian context.^25,28,29

Individual factors on performance evaluation and incentivization point towards a need for a structured system of incentives. This gives the much required boost to the sluggish momentum of motivation and accountability, furthermore leading to a transformative behaviour change with acceptance and enhancement of data use for programmatic decision making.⁷ The factors generated on incentivization displays applicability since incentivization as an effective strategy for promoting data based decision making has been emphasized in previous studies.^7,22,29 Consequently, Ministry of Health and Family Welfare, Government of India has introduced incentivization in the form of performance based incentives (PBIs).⁶ The individual technical factors on software knowledge and skills are vital as competency with analytic tasks and interpretation skills positively influence data use. It enables decision-makers to interpret and apply data, be able to distinguish which data they need to use and generate summaries of raw data.^22,30-32 This eventually will be very beneficial for the rural area PHCs and the population residing in nearby areas.³³

Limitations

The study has few limitations. The sampling method was non-probability - purposive and the actual sample was 80% of the expected sample, hence this might have implications on generalizability of results. We were unable to test the instrument on a larger sample size and in diverse geographic contexts. To further validate the instrument, future studies should test it with more representative and larger samples and consider modifying items in each category of determinants.

CONCLUSION

Our study generated an instrument consisting of workable categories of organizational, technical, and individual determinants of data use with correlated variables which captures multifaceted information and yet was clustered into correlated and practicable factors. The comprehensive factors generated in the instrument aptly represent the information system utilization landscape of the public health care system in India and similar LMICs. Our study contributes to the limited research conducted in LMICs particularly on identification and extraction of such inclusive factors which can be used as a tool/instrument in future research in India and similar LMICs.

The generated factors were valid, reliable and exhibit applicability in further analysis, research and programmatic contexts. These can be used as a ready and simplified tool or adapted in similar settings across diverse geographies to assess performance of routine health information systems and factors affecting data utilization for evidence-based decision making. Using this instrument in different contexts can yield policy recommendations and program strategies to enhance evidence-based decision-making at the peripheral level, hence promoting a culture of evidence-based decision-making right at the grassroot level.

ACKNOWLEDGEMENTS

The authors would like to acknowledge the contribution of the respondents and support of the officials at National Health Mission, Government of Haryana.

Funding: No funding sources

Conflict of interest: None declared

Ethical approval: The study was approved by the Institutional Ethics Committee

REFERENCES

1. 1. Royston G, Pakenham-Walsh N, Zielinski C. Universal access to essential health information: accelerating progress towards universal health coverage and other SDG health targets. BMJ Glob Health. 2020;5(5):e002475.
2. 2. Brownson RC, Fielding JE, Green LW. Building capacity for evidence-based public health: reconciling the pulls of practice and the push of research. Annu Rev Public Health. 2018;39:27-53.
3. 3. Witter S, Sheikh K, Schleiff M. Learning health systems in low-income and middle-income countries: exploring evidence and expert insights. BMJ Glob Health. 2022;7(sl7):e008115.
4. 4. Aqil A, Lippeveld T, Hozumi D. PRISM framework: a paradigm shift for designing, strengthening and evaluating routine health information systems. Health Policy Plan. 2009;24(3):217-28.
5. 5. PHC IPHS. 2022. Guideline. Available at: https://nhsrcindia.org/sites/default/files/PHC%20IPHS_2022_Guideline_pdf.pdf. Accessed on 30 June 2024.
6. 6. Ayshman Bharat Comprehensive Primary Health Care through Health and Wellness Centers Operational Guidelines. Available at: https://ab-hwc.nhp.gov.in/download/document/45a4ab64b74ab124cfd853ec9a0127e4.pdf. Accessed on 07 May 2024.
7. 7. Belay T. Data utilization and evidence-based decision-making in the health sector : survey of three Indian states. World Bank Group. 2009. Available at: https://documents.worldbank.org/en/publication/documents-reports/documentdetail/541461468267327228/data-utilization-and-evidence-based-decision-making-in-the-health-sector-survey-of-three-indian-states. Accessed on 07 May 2024.
8. 8. Sahota R, Das A, Afzal F. Determinants of data use for programmatic evidence-based decision-making at peripheral public health care centres in Haryana, India. Clin Epidemiol Glob Health. 2024;29:101713.
9. 9. Prinja S, Balasubramanian D, Sharma A, Gupta R, Rana SK, Kumar R. Geographic inequities in coverage of maternal and child health services in Haryana state of India. Matern Child Health J. 2019;23(8):1025-35.
10. 10. Jolliffe IT, Cadima J. Principal component analysis: a review and recent developments. Philos Trans R Soc Math Phys Eng Sci. 2016;374(2065):20150202.
11. 11. Hamzehgardeshi Z, Shahhosseini Z. Psychometric properties of an instrument to measure facilitators and barriers to nurses’ participation in continuing education programs. Glob J Health Sci. 2014;6(5):219.
12. 12. Hair JF, Anderson RE, Babin BJ, Black WC. Overview of multivariate analysis in Multivariate data analysis: a global perspective. 7th Edition. Upper Saddle River, NJ: Pearson. 2010;30-40.
13. 13. Shrestha N. Factor Analysis as a tool for survey analysis. Am J Appl Math Stat. 2021;20(9):4-11.
14. 14. Nunnally JC. Psychometric Theory 3E. Tata McGraw-Hill Education. 1994;784-90.
15. 15. Ledesma RD, Valero-Mora P, Macbeth G. The Scree Test and the Number of Factors: a Dynamic Graphics Approach. Span J Psychol. 2015;18(11):1-10.
16. 16. Pallant JF. Development and validation of a scale to measure perceived control of internal states. J Pers Assess. 2000;75(2):308-37.
17. 17. Leask CF, Sandlund M, Skelton DA, Altenburg TM, Cardon G, Chinapaw MJM, et al. Framework, principles and recommendations for utilizing participatory methodologies in the co-creation and evaluation of public health interventions. Res Involv Engagem. 2019;5(1):1-16.
18. 18. Laverack G. Public Health: Power, Empowerment and Professional Practice. Bloomsbury Publishing. 2019;180-4.
19. 19. Sriram C, Pichaandy C. Multi-level Analysis of Health Management Information System (HMIS) Adoption by Healthcare Professionals in the ESIC Main Hospital and Dispensaries in the Tirunelveli Sub-region in Tamil Nadu, India. J Educ Innov Commun. 2023;4(2):61-80.
20. 20. Gao X, Yu J. Public governance mechanism in the prevention and control of the COVID-19: information, decision-making and execution. J Chin Gov. 2020;5(2):178-97.
21. 21. Hoffmann T, Bennett S, Mar CD. Evidence-Based Practice Across the Health Professions. Elsevier Health Sciences. 2023;384-9.
22. 22. Neelima S, Govindaraj M, Subramani K, Alkhayyat A, Mohan C. Factors influencing data utilization and performance of health management information systems: a case study. Indian J Inf Sources Serv. 2024;14:146-52.
23. 23. Haftu B, Taye G, Ayele W, Habtamu T, Biruk E. A mixed-methods assessment of routine health information system (RHIS) data quality and factors affecting it, Addis Ababa City administration, Ethiopia, 2020. Ethiopian J Health Dev. 2021;35(1):15-24.
24. 24. Thomas MP, Kozikott S, Kamateeka M, Abdu-Aguye R, Agogo E, Bello BG, et al. Development of a simple and effective online training for health workers: results from a pilot in Nigeria. BMC Public Health. 2022;1;22(1):551-9.
25. 25. Meghani A, Rodríguez DC, Peters DH, Bennett S. Understanding reasons for and strategic responses to administrative health data misreporting in an Indian state. Health Policy Plan. 2023;38(2):150-60.
26. 26. Dehnavieh R, Haghdoost A, Khosravi A, Hoseinabadi F, Rahimi H, Poursheikhali A, et al. The District Health Information System (DHIS2): A literature review and meta-synthesis of its strengths and operational challenges based on the experiences of 11 countries. Health Inf Manag J. 2019;48(2):62-75.
27. 27. Meghani A, Tripathi AB, Bilal H, Gupta S, Prakash R, Namasivayam V, et al. Optimizing the Health Management Information System in Uttar Pradesh, India: Implementation Insights and Key Learnings. Glob Health Sci Pract. 2022;10(4):e2100632.
28. 28. Mishra A, Mokashi T, Nair A, Chokshi M. Mapping Healthcare Data Sources in India. J Health Manag. 2022;24(1):146-59.
29. 29. Afzal F, Mehra S, Mishra HK. A Dire need to incorporate hospital information system in paramedics’ curriculum: evidence from private hospitals’ paramedics of Delhi-NCR, India. Indian J Sci Technol. 2022;15(16):742-9.
30. 30. Afzal F, Ahmad AA, Ali QA, Joshi S, Mehra S. Fulfilling the need of hour: Systematic review of challenges associated with electronic medical record (EMR) implementation-SBEA model. Vidyabharati Int Interdiscip Res J. 2021;13(8):649-62.
31. 31. Car LT, Kyaw BM, Panday RSN, Kleij R van der, Chavannes N, Majeed A, et al. Digital Health Training Programs for Medical Students: Scoping Review. JMIR Med Educ. 2021;7(3):e28275.
32. 32. Meghani A, Tripathi AB, Bilal H, Gupta S, Prakash R, Namasivayam V, et al. Optimizing the health management information system in Uttar Pradesh, India: implementation insights and key learnings. Global Health: Science and Practice. 2022;10(4):e2100632.
33. 33. Afzal F, Raychaudhuri PS, Afzal MA, Ahmad AA. Challenges faced by BPL population in availing public healthcare–analysing government initiatives, technology and cultural barriers in Aligarh district, UP. South Asian J Soc Sci Humanities. 2021;2(5):1-9.