Development of an instrument for measuring adolescent health compromising behaviours among rural students: a Delphi study

INTRODUCTION

There are 253 million adolescents in the age group 10-19 years in India. This age group comprises of individuals in a transient phase of life requiring nutrition, education, counseling and guidance to ensure their development into healthy adults. They are susceptible to several preventable and treatable health problems, like early and unintended pregnancy, unsafe sex leading to sexually transmitted infections/human immunodeficiency virus/acquired immunodeficiency syndrome (STI/HIV/AIDS) nutritional disorders like malnutrition, anemia and overweight, alcohol, tobacco and drug abuse, mental health concerns, injuries and violence.¹

Of 56.4 million global deaths in 2015, 39.5 million, or 70%, were due to non-communicable diseases (NCDs).² The leading risk factors responsible for NCDs are behavioral factors such as tobacco use, physical inactivity, and metabolic factors such as overweight/obesity, high blood pressure (BP), high cholesterol level, and high blood glucose level.³ Most of the risk factors are behaviorally acquired and are due to change in lifestyle during adolescent age group.

Lifestyle-associated risk factors among adolescent are rampantly increasing throughout the world. In framing of cost-effective strategies for prevention and treatment, identification of the risk factors and their quantification is of great importance. India with >243 million adolescents and being the capital of NCDs.^4-5

Several studies in western societies have found that multiple etiological factors, including individual, socio-cultural and environmental factors are responsible.⁶ A range of community, family and individual characteristics affect substance use among adolescents.^7-10

The root causes of mortality among adolescent are not diseases, but are primarily related to preventable factors like social, environmental and behavioral factors. Many different studies on health-compromising behaviors among adolescents indicate that these behaviors tend to co-occur, leading to a health compromising life-style. To formulate and implement effective adolescent health programs at regional level, it is essential to identify different health compromising behaviors and their multidimensional correlates and for that it is also needed to develop an instrument for measuring the same.

In order to develop an instrument with good content validity for measuring adolescent health compromising behavior, a modified Delphi procedure was conducted, involving an interactive process designed to establish consensus on specific questions or criteria through systematic collection of informed judgments from professionals in the field.¹¹ This type of procedure was aimed at achieving consensus among experts in a systematic manner and consists of multiple consultation rounds in which experts indicate their (dis) agreement with statements or concepts.¹² Research tells us that the inclusion of different stakeholders in a Delphi procedure promotes acceptance of feedback and effective implementation of the instrument.¹³ Therefore many groups of stakeholders: doctors, nurses, educational experts, counsellors, and parents included. The modified Delphi procedure had been shown to provide adequate evidence for the content validity of an instrument and it was used because it enables effective consensus building in a situation where published information is inadequate.^14,15

Modified Delphi method is really a useful technique to combine both quantitative (can be statistically processed) and qualitative (anonymous written explanations in addition with controlled feedback) research approaches in designing a suitable instrument.¹⁶

METHODS

Delphi study design was adopted for the development of an instrument for measuring adolescent health compromising behaviors among rural students during January 2018 to January 2019. Following series of steps were followed to carry out the Delphi study.

Framing of research questions

Development of research questions is the initial step in all health research. Here following research questions were outlined for Delphi study to encourage the advancement of information assortment instrument to measure adolescent health compromising behaviors and it correlates among adolescent students at rural schools: what are the common health compromising behaviors found among adolescent students?, in which age group of adolescents, health compromising behaviors are more prevalent?, what are the predisposing factors related to different health compromising behaviors?, what are the enabling factors related to different health compromising behaviors?, and what are the reinforcing factors related to different health compromising behaviors?

Selection of experts of panels

The panels were composed of experts who had knowledge and experience, involvement with the issue under scrutiny: capacity and willingness to partake, sufficient time to take an interest and participate, and effective communication skills and commitment to participate in multiple rounds.

Panelists or experts of the panel were selected by non-probability purposive sampling technique. Here participants (experts) were not selected randomly, so representativeness is not assured. Rather, they were chosen for a reason, to apply their insight to a specific area. Here 15 experts from heterogeneous groups were included; they were from medical and nursing, psychologist, sociologist, education experts, and parents.

Criteria for inclusion of items in the instrument

The following criteria were included: if any expert suggested an additional item, an additional Delphi round would be conducted, a standard deviation of <1 was deemed to indicate consensus and considered for inclusion in the instrument, and the determinate criterion (cut-off point) of I-CVI was 0.78 (78%) and 0.90 (90%) for the S-CVI/Ave.17-18

Ethical considerations

Within the Delphi, experts didn’t meet with each other face to face and therefore they could present and react to ideas unbiased way. So as to keep up the meticulousness of this strategy the researcher (facilitator) knew the identity of experts.

Therefore, the pursuing of true anonymity was not possible. Respondents/experts were known to the researcher/facilitator and even to another, yet their remarks, proposal and decisions stayed anonymous.

Development of tool

Phase 1: planning of the instruments

Consideration of the background

In the initial steps of development of instruments, the research objectives, questions and hypothesis of the proposed research were examined. Attention was also given to the background of the participants especially their educational/readability levels. A thorough understanding of the problems through literature search was done.

Conceptualizations of the instruments: review of literature and framework of instrument

Firstly, literature related to the topic were reviewed to identify existing surveys related to health compromising behavior with the special focus on physical inactivity, sedentary behavior, unhealthy eating, poor dental hygiene, tobacco use and alcohol use.

In this step, the following issues were addressed and enquired from the literature: what are the different health compromising behaviors are prevalent among the adolescent students of selected area?, which age group of adolescents are high risk for developing health compromising behaviors?, how could different health compromising behaviors be explained?, what were the probable correlates or independent variables could be assessed?, and what were the different domains of those correlates?

In the previous research studies it was identified that data were collected by the following instruments developed for measuring the public health issues and which were not copyrighted: youth risk behavior surveillance (YRBS), health behavior in school aged children (HBSC), global school based student health survey (GSHS), and adolescent health wellbeing questionnaire (AHWQ).

Secondly, theory related to adolescent health compromising behaviors and related factors were reviewed to identify the probable correlates. A conceptual framework for the tool has been developed based on the following theories and models and presented in Figure 1: Jessor’s problem behaviour theory, health belief model, and Green’s proceed and precede model.

Phase 2: defining variables and determining the domain - item selection and generation

In this phase, a list of outcome variables and their probable correlates were prepared. Next conceptual definitions & operational definitions of those variables were decided based on the research findings. Further it was decided which variables could be measured by pre-existing tool and for which variables, items to be generated by self. Blueprint of the tool with items examples was developed with a strong understanding of the current literatures, existing scales and levels of data required to answer the research questions.

Most of the variables were intended to measure with pre-existing tool. Only when the tools found not feasible or not suitable for the target samples, and not culturally appropriate, then only efforts were made to generate the items based on the literature.

Phase 3: face validity: evaluation of blueprint of tool by expert panel review

In this phase, workshop was done with the expert health professionals to establish the face validity of the tool. Presentation done on following areas: WHO and centers for disease control and prevention (CDC) guidelines and recommendations for adolescent health promoting behaviors, surveys used in the past to measure the different health compromising behaviors and their correlates, probable conceptual framework developed for the present study, and blueprint of the data collection instruments.

Discussion was done for their comments, additions, deletion to identify the important health compromising behaviors and their correlates.

Phase 4: content validity (Delphi round)

Developing Delphi-round one questionnaire and administration

After the face validity, necessary modifications were made and 1st draft of the tool was prepared. Delphi panel members were given that 1st draft of questionnaire along with other existed survey proforma and related literature supportive of correlates, and research questions.

The aim was to include items which cover the complete domain of possibilities in the related areas. Panel members were being asked to judge and give comments on their professional or personal experience, i.e. practice-based evidence as well as literature-based evidence. They were also given 4 points criteria checklist to judge the questionnaire.

Analyzing Delphi round one

A quantitative approach of the Delphi method was used to identify the content validity by calculating I-CVI (the value of the I-CVI is the summation of agreement divided by the total number of experts) to determinate which question need to be removed from the proposed instrument, and the content validity index for S-CVI (adopting the average proportion of items which were approved by the experts and calculating this as the average of the I-CVI values) to recognize the proportion of agreement within the instrument. Out of 15 experts, all the experts responded and rated the items and given comments and suggestion.

Experts of panel rated all the items of the instruments on 4 point ordinal scale advocated by Davis to avoid having a neutral and ambivalent midpoint (1=relevant, 2=somewhat relevant, 3=quite relevant, and 4=highly relevant).

Then for each item, dichotomizing the ordinal scale was done in relevant (3 or 4) and not relevant (1 or 2) and the I-CVI was computed by no of agreement in relevancy and divided by no of experts. After that S–CVI was calculated by summing all the I-CVI and divided by no of items. Two health compromising behaviors: unprotected sex, substance use removed from the instrument and one behavior was added i.e. poor dental hygiene as suggested. After addition, deletion and modification, 2nd draft of the tool was developed.

Cognitive interviewing

The 2nd draft of the tool was translated to Bengali language and cognitive interviewing done with 5 girls and 5 boys students to identify and analyze sources of response errors in survey questionnaires.

Specifically, the main objective of this was to understand whether the samples were understating the questions in the ways intended by the researcher. “Think aloud” method was adopted in which questions were read to the students, then observed and recorded as the students “thinks aloud” about the questions and answered it and students were asked to verbalize his or her thoughts about the questions asked. In this phase also, modification done in the physical activities areas, as the students were facing difficulties in recapitulating the items and was taking huge time to record day wise time spent for each types of activities. So instead of global physical activity questionnaire (GPAQ) and adolescent sedentary behaviour questionnaire (ASBQ) tools, physical activity and sedentary behaviour were assessed by the items from youth risk behaviour surveillance (YRBS) and global school based student health survey (GSHS). So 3rd draft of the tool was developed which contained 100 questions.

Delphi round two

In this round feedback to the experts was provided: calculated I-CVI and S-CVI along with anonymous comments, suggestion on 1st draft of the tool by different experts along with result of cognitive interview, 3rd draft of the tool were sent to the experts. Out of 15 experts, 11 experts again rated the same items as well as new items on 4 point scale.

Analyzing Delphi round two

In this round, received the responses from 11 experts out of 15 experts. I-CVI and S-CVI were calculated. Three items related to factors of health compromising behaviors were deleted as not fulfilling the CVI accepted criteria. Few items were modified as per the suggestion in the area of parental monitoring, family influence, and peer influence, frequency of fast food, junk food, and cold drinks consumptions 4th draft of the tool was developed.

Language validity of the tool

The Bengali version of the tool was prepared with the help of a language expert. Language validity was established by translating in Bengali and retranslating it into English with the help of another language experts.

Delphi pilot testing of the tool

Pilot testing of the tool was done among 100 students. No students expressed that they take alcohol. So use of alcohol was not assessed in final study. And 12 questions related to alcohol use were deleted from the 4th draft of the tool, 5th draft of the tool was developed.

Delphi round three

Fourth draft of the instrument along with summary of round one and round two and result of pilot testing of the instrument were given to the experts. Here the experts were given the opportunity to change their answers and to comment on the same items related to different health compromising behaviors and their associated factors. In this round, received responses from nine experts out of 11 experts. No further round done as research questions were answered, consensus was reached, theoretical saturation was achieved, and sufficient information had been exchanged. As panelists proposed no additional items and made no other negative comments other than the suggestion to eliminate item related to alcohol. So it was concluded that consensus was reached. Thus 85 questions-based 5th draft of the instruments was finalized to identify the adolescent health compromising behaviors and its correlates.

Phase 5: testing of reliability

The Bengali version of the final questionnaires was administered to 100 students of coeducational higher secondary school after obtaining informed consent from the samples to establish the reliability of the tool. Stability of the tool was tested by test – retest method. Pearson product moment correlation coefficient (r) was calculated for the scoring questions and ranged from 0.72 to 0.84 and Cohen’s kappa coefficient (k) was calculated for nominal data and ranged from 0.8-1. Hence the tool was stable for meeting the purpose of the tool. Summary of the development of the instrument is represented in Figure 2.

RESULTS

I-CVI (item-level content validity index) was calculated for all single item questions and S-CVI/Ave (scale-level content validity index based on the average method) was calculated for all matrix table questions.

In Delphi round one, I–CVI of 12 questions were found less than 73.33%; S–CVI/Ave of two matrix table questions were 69.99 and 67.86. Those questions were excluded due to poor CVI. I–CVI were ranged between 67.86–100 % and S–CVI/Ave were ranged between 67.86–100%. Nine questions were modified, and five questions were included as per suggestions.

In Delphi round two, I–CVI of 2 questions were 63.33 and S–CVI/Ave of one matrix table question was 72.22; those questions were excluded due to below cut off CVI. I–CVI were ranged between 63.33–100% and S–CVI/Ave were ranged between 72.22–100%.

In Delphi round three, calculated I–CVI for all single item-based questions of part–I, part II, part III, part IV and part V of the instrument were 100%. S–CVI/Ave of two items of part I were 92.58 and 94.44; three items of part–II was 95.55; two items of part–III were 95.55 and 96.29. S–CVI/Ave for rest part of the instrument was 100%.

So in summary, I–CVI of the all parts of the instrument having single item questions was 100% and S–CVI/Ave were 97.83%, 97.33%, 99.18%, 100%, 100% in part–I, part–II, part–III, part–IV, and part–V respectively.

DISCUSSION

The final data collection instrument were having five parts. Part I consisted of the items related to demographic information as well as individual psychological factors, family and parental factors. Part II consisted of physical inactivity and sedentary behaviour, part III consisted of eating practices, part IV consisted of practice of dental hygiene and part V consisted tobacco use. The list of the factors were measured presented in Table 2 and sources of the items in the tool presented in Table 3.

During the development of the instrument, anonymity maintained, Delphi round conducted to get the responses of the same questions until the consensus reached. Controlled feedback was provided. All inappropriate questions were removed by calculating each content validity index for items (I-CVI) and the overall content validity index for scales (S-CVI) of the instrument. Through Delphi, individual viewpoint was converted to group viewpoint. Limitation of the study is low number of panelists. For Delphi studies different numbers of panelists have been reported.¹⁹ It is also recommended that the panel should not be too large so as to avoid drop-out. In this study, the response rates of the first and second rounds were 100% (15/15), 73% (11/15) and 82% (9/11) respectively.

CONCLUSION

The aim of this study was to develop an instrument with good content validity for adolescent health compromising behaviors and its correlates among adolescent students of rural school of West Bengal. This is the first instrument of its kind to be designed and customized for rural school students of south 24 Parganas of West Bengal. The instrument has similarities and differences compared with instruments of Western origin i.e. adolescent health development questionnaire and instruments developed for Indian adolescents’ i.e. adolescent wellbeing questionnaire. Furthermore, the findings suggest that the development of the instrument including content validity of instruments for measuring adolescent health compromising behaviors and its correlatesis influenced by socio-cultural aspects.

Funding: No funding sources

Conflict of interest: None declared

Ethical approval: The study was approved by the Institutional Ethics Committee

REFERENCES

1. 1. Adolescent health (RKSK), NHM. Adolescent Health (RKSK): National Health Mission. Available at: https://nhm.gov.in/index1.php?lang=1&level=2&sublinkid. Accessed on: 24 May 2019.
2. 2. World Health Organization. NCD Mortality & Morbidity. 2015. Available at: http://www.who.int/gho/ncd/mortality_morbidity/e/. Accessed on: 30 January 2018.
3. 3. Esmailnasab N, Moradi G, Delaveri A. Risk factors of non-communicable diseases and metabolic syndrome. Iran J Public Health. 2012;41:77-85.
4. 4. Singh AP, Misra G. Adolescent lifestyle in India: Prevalence of risk and promotive factors of health. Psychol Develop Soc. 2012;24:145-60.
5. 5. Hazreen MA, Su TT, Jalaludin MY, Dahlui M, Chinna K, Ismail M, et al. An exploratory study on risk factors for chronic non-communicable diseases among adolescents in Malaysia: BMC Public Health. 2014;14(3):6.
6. 6. Poland B, Frohlich K, Haines RJ, Mykhalovskiy E, Rock M, Sparks RJ. The social context of smoking: the next frontier in tobacco control? Tobacco Control. 2006;15:59-63.
7. 7. Neumark-Sztainer D. Patterns of health-compromising behaviors among Minnesota adolescents: socio-demographic variations, Am J Public Health. 1996;86:1599-606.
8. 8. Blum RW, Rinehart PM. Reducing the Risk: Connections that Make a Difference in the Lives of Youth, Minneapolis: Division of General Pediatrics and Adolescent Health, University of Minnesota, United States. 1997.
9. 9. Resnick MD. Protecting adolescents from harm. J Am Med Assoc. 1997;278:823-32.
10. 10. Jessor R, Turban MS, Costa FM. Protective factors in adolescent health behavior. J Personal Soc Psychol. 1998;75:788-800.
11. 11. Newman LR, Lown BA, Jones RN, Johansson A, Schwartzstein RM. Developing a peer assessment of lecturing instrument: lessons learned. Acad Med. 2009;84(8):1104-10.
12. 12. Boor K, Van Der Vleuten C, Teunissen P, Scherpbier A, Scheele F. Development and analysis of D-RECT, an instrument measuring residents’ learning climate. Med Teach. 2011;33:820-7.
13. 13. Bowden J, Marton F. The University of Learning: Beyond Quality and Competence. London: Kogan. 1998.
14. 14. Keeney S, Hasson F, McKenna HP. A critical review of the Delphi technique as a research methodology for nursing. Int J Nurs Stud. 2001;38:195-200.
15. 15. Palisano RJ, Rosenbaum P, Bartlett D, Livingston MH. Content validity of the expanded and revised gross motor function classification system. Dev Med Child Neurol. 2008;50:744-50.
16. 16. Jones J, Hunter D. Qualitative research: consensus methods for medical and health services research. BMJ. 1995;311:376-80.
17. 17. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30(4):459-67.
18. 18. Polit DF, Beck CT. The content validity index: are you sure you know what’s being reported? Critique and recommendations. Res Nurs Health. 2006;29(5):489-97.
19. 19. Cantrill J, Sibbald B, Buetow S. The Delphi and nominal group techniques in health services research. Int J Pharm Pract. 1996;4:67-74.