Assessment of suitability of pediatric index of mortality 2 score for monitoring the outcome of pediatric intensive care unit patients and associated risk factors of mortality in a tertiary care center in South India

INTRODUCTION

Severity scoring systems form an integral part of providing ICU care. There are several scoring systems being used for assessing severity of morbidity and predicting the risk of mortality. It is useful for objectively predicting the outcomes and assessing the prognosis of PICU patients. They are also used as performance indicators of ICUs. They are also important for comparing groups of patients in research trials.

There are various scoring systems that have been used since 1980, like the pediatric risk of mortality (PRISM), pediatric logistic organ dysfunction (PELOD), pediatric index of mortality (PIM). Of these, the simplicity, of PIM makes it easier to collect data from large number of sick children point of care data collection and risk stratification.^4,7

The first version of PIM was developed in 1997 in Australia and UK, by Shann et al, to predict the outcome in children admitted to PICU. It was revised in 2003, using data collected from 13 PICU in 1997 and 1999 in Australia, New Zealand and the UK. PIM 2 scoring is supposedly better than the earlier version for outcome predictability.³

Pediatric index of mortality 2 score uses logistic regression models to obtain an equation to describe the relationship between predictor variables and probability of death. To make use of these scores, both at clinical and policy level, it is important to know if it is relevant and valid in a given population.

PIM 2 score⁴

PIM model was revised in 2003, by Slate and Shann, to improve the predictability of outcome. The PIM variable of ‘specific diagnoses was replaced by ‘high risk diagnosis’ and ‘low risk diagnoses.

High risk diagnosis variable includes cardiac arrest preceding intensive care unit admission, severe combined immune deficiency, leukaemia or lymphoma after first induction, spontaneous cerebral haemorrhage, cardiomyopathy or myocarditis, hypoplastic left heart syndrome, HIV infection, liver failure is the main reason for intensive care unit admission, neuro-degenerative disorder.

Low-risk diagnosis include asthma is the main reason for ICU admission, bronchiolitis is the main reason for ICU admission, group is the main reason for ICU admission, obstructive sleep apnoea is the main reason for ICU admission, diabetic keto-acidosis is the main reason for ICU admission.

The post-procedure variable was further divided into ‘recovery post-procedure’ and post ‘cardiac bypass’.

In order to calculate the predicted risk of mortality, for each patient, logit was calculated by multiplying the fixed coefficients of the logistic regression by variables (X1, X2, ….X10) and adding them, to the pre-determined constant.¹⁰

That is, Logit =constant + AX1 + BX2 + CX3 + …

Thus, for the quantitative variables, absolute values and for the qualitative variables (dichotomy), either number zero or one was used. For example, if the child had a systolic blood pressure of 70 mmHg, 70 was multiplied by the related constant of 0.01395, or if there was no pupillary light reflex, one was multiplied by the related constant ratio of 3.0791. PaO₂ and FiO₂ were obtained according to laboratory findings. To calculate the predicted death rate, the logit value was obtained from the formula.¹⁰

P=e^logit / (1+e^logit), where e=2.7183.

The collected information was compiled with the related database and the data was analysed with chi-square test, logistic regression and Hosmer Lemeshow goodness of fit test. The SMR was calculated by taking ratio of observed to the predicted death rates and performance of the model was assessed.

So, in this study we aimed at assessing PIM 2 model for its discriminating ability, whether the model is calibrated to our setup and to compare the observed with the predicted death rates, so as to evaluate, whether the model predicted death well among PICU patients. So, by validating the PIM 2 score, we can stratify at risk patients and early intervention can be provided to improve the outcome. This also helps in monitoring the quality of care being provided.

There are many associated factors affecting the mortality in critically ill children admitted in PICU like dilated and fixed pupils of size >3 mm, presence of shock in the 1^st of hour of admission to PICU, mechanically ventilated within 1 hour of PICU admission, low GCS<8 all contributing to independently to the risk of mortality. These factors association are also assessed in the present study. This helps in prognosticating and counsel the parents and care givers accordingly.

METHODS

This was a prospective observational study done in pediatrics intensive care unit, department of pediatrics, Rajarajeswari medical college and hospital. It is a six-bedded intensive care unit, catering to both medical and post-surgical patients, mainly to medical illness. Our pediatrics intensive care unit is equipped with facilities for continuous monitoring, mechanical ventilation, CPAP, blood gas analysis, X-ray, ultrasonography and the echocardiography facilities.

The study was conducted over a period of 18 months from January 2019 to June 2020. The sample size of 120 PICU patients were taken up for the study, aged from 1 month to 18 years. Institutional ethical committee approval was taken prior to undertaking the study. Informed written consent was taken from the parents or guardians of all the participating children. All consecutive patients admitted to pediatrics intensive care unit, who had met the study criteria were enrolled for the study.

Demographic data such as name, age, gender was collected. Other data like clinical diagnosis, length of PICU stay and other clinical and laboratory data required for PIM 2 scoring were recorded. All the data were recorded within 1 hour of admission to PICU, using a structured data collection form. Arterial blood gas analysis was done within 1 hour of admission and base excess and PaO₂ were recorded.

PIM 2 score consists of ten variables.^4,25 The variables include elective admission to PICU, recovery post-procedure, cardiac bypass procedures, high risk diagnosis, low risk diagnosis, no response of pupils to bright light (>3 mm and both fixed), mechanical ventilation (at any time during the first hour in PICU), SBP (mmHg), base excess (mmHg) arterial or capillary blood and FiO₂* 100/PaO₂ (mmHg).

High-risk diagnosis includes cardiac arrest preceding ICU admission, severe combined immune deficiency, leukemia or lymphoma after first induction, spontaneous cerebral hemorrhage, cardiomyopathy or myocarditis, hypopllastic left heart syndrome, HIV infection, liver failure is the main of reason for PICU admission, neurodegenerative disorder.

Low risk diagnosis include asthma is the main reason for ICU admission, bronchiolitis is the main reason for ICU admission, croup is the main of ICU admission, obstructive sleep apnoea is the main cause of ICU admission, diabetic ketoacidosis is the main cause of ICU admission.

Yes or no response of the variables was recorded as 1 or 0 respectively. All the variables were entered into the system, www.sfar.org/scores2/pim22.html, for the calculation of predicted death rate, by using logistic regression equation.

Logit=(-4.8841) + (values * beta) + (0.01395 * absolute( SBP-120))) + (0.1040 * (absolute base excess)) + (0.2888 * (100 * FiO₂/ PaO₂)).

Predicted death rate = e^Logit /(1+ e^Logit )

Systolic blood pressure was recorded as ‘0’, if the patient was in cardiac arrest, 30 if the patient was in shock and BP was so low that it cannot be measured, if SBP of patient was not known, then it was recorded as 120 as per the website recommendations. Elective admission included admission after elective surgery or admission for an elective procedure (e.g., -insertion of a central line) or elective monitoring or review of home ventilation. An admission is considered elective if it could be the postponed for more than the six hours without adverse effect.

The associated factors in predicting mortality like dilated (>3 mm) and fixed pupils, presence of shock, mechanical ventilation within 1 hour of PICU admission, low GCS <8 were also, recorded. Any patient with tachycardia, poor pulse volume, prolonged capillary refill time, decreased urine output, altered sensorium, cool extremities were considered to have “presence of shock”. The patients were followed up throughout their stay in PICU and the final outcome was recorded as ‘discharged’ or ‘death’. For the purpose of analysis, the outcome of patients who were discharged against medical advice was recorded as ‘death’.

Statistical analysis

The statistical analysis was performed using SPSS version 19.0. After completing data collection, data were recorded in Microsoft excel 2007. The discriminating ability of PIM 2 scoring system between survivors and non-survivors was analysed by area under receiver operator characteristic (ROC) curve, using PIM 2 values. The ROC curve represents the relationship between sensitivity and specificity of a given test and thus expresses how well the model distinguishes between the survivors and non-survivors. The area under ROC curve of 0.75 or more, is considered to be clinically useful and have good discriminating ability.

The calibration of PIM 2 scoring system was analysed by Hosmer-Lemeshow goodness-of-fit (GOF) test.²⁶ It is a logistic regression model. The GOF test p were done for calibration across deciles of risk for age and diagnostic subgroups. It describes the performance of the scoring system in the present unit where the study is done, in comparison with the original unit where the scoring system was formulated.

The standardised mortality ratio (SMR), is the ratio of observed to expected mortality rates. It is a well-recognised and valid measure for comparing the risk adjusted mortality between different centers. The SMR values <1, imply good performance of the model which in turn indicates good quality of care being provided in the study unit when compared to the unit where the scoring system was formulated. If the upper limit of 95% confidence interval of the calculated SMR is<1, then observed mortality is regarded as being lower than the expected. In this present study, SMR was calculated with 95% confidence interval, for the whole group using OpenEpi statistical calculator online.

RESULTS

A total of 120 consecutive patients admitted to PICU, aged from 1 month to 18 years, who met the study criteria, were taken up for the study. The mean age of admission to our PICU was 2.2 years±3.8 (Table 2). Among the admitted, 58% were males and 42% females (Table 3). Mean heart rate among the admitted patients was 123±25 bpm. Mean systolic BP was 93±24 mmHg. Respiratory disease was the most common cause for PICU admission, with 55% of admissions and sepsis being the second most common cause, 9%. Mortality was maximum in sepsis group, with 40% observed mortality rate (Table 4). Age and gender of the patient had no significant association with the outcome (p=0.321). The length of stay in PICU had shown significant association with the mortality with p<0.001. The predicted death rate by PIM 2 score had statistically significant association with the mortality with p<0.001 (Table 5). The mortality rate was found to be 12.5%.

PIM 2 score has good discriminating ability between the survivors and dead. The area under ROC curve is 0.867, which is high as shown in the Figure 1. This indicates that the model shows increased predicted death rate, increases mortality (p<0.001) as shown in the Table 8). PIM 2 score can be used in predicting the mortality in PICU patients.

PIM 2 had good calibration across all age groups with goodness-of-fit p>0.05, with maximum calibration across 1-4 years age group. PIM 2 had good calibration across respiratory disease subgroup with p=0.27. PIM 2 had poor calibration in cardiovascular and sepsis diagnostic subgroups with p=0.005 and p=0.002 respectively (Table 6). Overall, PIM 2 is well calibrated to our setup with p=0.961 (>0.05). Standardized mortality ratio in the study is equal to 1, as observed deaths are equal to predicted death rates. So, PIM 2 score predicted well the deaths. Among the risk factors studied, dilated (>3 mm) and fixed pupils was seen in 2.5% cases, mechanical ventilation in 1^st hour in 15.8% cases. 14.2% cases presented in shock and 10% had low GCS (<8). All the above factors are significantly associated with mortality (p<0.001) as shown in the Table 7. Heart rate variations has not statistically significant association with the mortality.

DISCUSSION

PIM 2 scoring system is an illness severity scoring system, that can be used to predict the mortality in PICU patients. It is scored within 1 hour of PICU admission, of the patient. It is simple, easier to collect data, it can be done even in set-up with minimal resources and is not affected by treatment given.

In the present study, the results has been evaluated for discrimination, calibration and SMR, by PIM 2 scoring system in tertiary care PICU, in South India. The current study investigated 120 patients admitted to PICU, over 18 months.

The general characteristics of the studied subjects when compared to other studies showed that the mean age of admission being 2.2±3.8 years, while in a study done by Hariharan et al, in Barbodas, mean age was 4.9±4.4 years.¹³ The mean age of admitted cases in a study by Youssef et al in Egypt, was 14months and 4years in the study done by Sankar et al, in AIIMS, New Delhi.^9,18 In our study, there were more of male subjects, which was similar to study by Destiana et al.¹³ In other studies, there was nearly equal proportions of males and females being admitted.

There was no statistically significant association of gender with the outcome. In our study, the mean length of stay was 2.13±1.2 days and 3.7±2.3 days among dead and discharged groups respectively which was comparable to other similar studies like Gandhi et al, Youssef et al, Aroor et al, investigating the validity of PIM 2 score in PICUs in developing countries.^23,9,32 As like previous studies done in other developing countries, respiratory morbidity (both infective and non-infective) was the most common diagnosis at admission. In Roshani et al study, cardiovascular involvement was the most common and in Sankar et al study, sepsis and malignancy in Destiana et al study done in Indonesia.^21,18,11 The mean predicted death rate was found to be statistically significant with the outcome as with other studies mentioned.

Higher predicted death rate was associated with increased risk of mortality. In our study, PIM 2 scoring system had good discriminating ability between survivors and non-survivors, with area under ROC curve of 0.867 with 95% CI of 0.729-0.980. This was comparable with results of, Slater et al with AUC of 0.9, where the PIM model was revised to PIM 2.⁴ In other studies like Tibby et al AUC was 0.83-0.87, in Hariharan et al, it was 0.82.^14,13 A study done in UK in 5 PICU settings, proved that PIM 2 scoring system had good discrimination, with AUC of 0.84. A study in Hong Kong, showed AUC of 0.843.³³ Few Indian studies like study by Gandhi et al also had similar values, which was all comparable to our present study.²³ Other studies like Antonio et al, Youssef et al , Payman et al, showed fair discrimination with AUC between 0.70-0.80.^31,9,10

PIM 2 scoring system had good calibration across all age groups with GOF test p>0.05 with maximum calibration between 1-4 years of age. There was also good calibration in respiratory diseases subgroup and poor calibration was observed in cardiovascular and sepsis subgroups. In Youssef et al study, PIM 2 had good calibration with GOF p=0.943.⁹ A multicentric study in Italy also reported, good calibration of PIM 2 with GOF p=0.308.⁸ Other studies by Antonio et al, Tibby et al, also showed good calibration compared to PRISM.^31,14 Study done in AIIMS, New Delhi, by Sankar et al, also recorded good calibration across all age groups and among all diagnostic subgroups, except post-operative cases, where poor calibration was noted.^18,20 In another Indian study by Roshani et al study, in Mumbai tertiary care centre showed poor calibration with GOF p=0.028.²¹

The calculated SMR in our study was 1, across deciles of risk, which implied that observed death rates were equal to predicted death rate. which was similar to studies by Antonio et al and Sankar et al.^31,18 Studies like Youssef et al, found SMR >1, that implied PIM 2 scoring under-predicted the death or poor quality of care of the study unit according to Pearson et al.^9,7 This, they have attributed to confounders like differences in the population under study when compared with the original population, where PIM 2 was developed, or due to small number of study sample or due to inadequate resources in the study setup. Another study by Payman et al, done in Indonesia also recorded SMR>1 (1.8).¹⁰ Study by Hariharan et al, another multicentric study to validate PIM 2 performance done in Italy in 3 PICUs showed SMR<1.¹³ Few other studies in Argentina and Hong Kong also showed similar results (SMR<1), which implied that observed deaths are lesser than predicted deaths.^12,33 These studies concluded that PIM 2 score over-predicted the death rate.

In our study factors like dilated and fixed pupils, mechanical ventilation within 1^st hour of admission, presence of shock and low GCS, were all significantly associated with the mortality (p<0.001). All these factors, independently were associated with increasing the risk of mortality among PICU patients. Thus, presence of any of these risk factors warrants attention to improve the outcome. In study done by Gandhi et al, shock within 1 hour had shown significant association with the mortality.²³ Another study done by Aroor et al, found that hypoalbuminemia was significantly associated with the mortality, in addition to the above factors.³²

Limitations

Due to financial constraints, few patients took DAMA, whom were considered dead in the study for statistical purposes, that could result in variations in the result when the study is reproduced in other settings. The statistical analysis like Goodness-of-Fit test is more reliable with larger sample size. The smaller the variable size, the lesser is the sensitivity and specificity. So, we recommend further research to confirm the results of our present study, from a large multicentric study.

CONCLUSION

From this study, it can be concluded that PIM 2 scoring system can be used in minimal manpower resource setup, because of ease of collection of data, scoring not affected by treatment and freely available software, so economical. PIM 2 score can be used in predicting the mortality of ICU patients. The PIM 2 predicted death rates were equal to the observed death rates, SMR being equal to 1. Thus, the quality of care provided in our PICU is comparable to the unit where the score was developed and validated. PIM 2 score is well calibrated to our unit and can be used for hospital audit. As the PIM 2 can be scored at the earliest, it helps in counselling the parents. As it gives objective and measurable value to evaluate and monitor the quality of care being provided. Thus, helps in periodic reviewing, improving and providing good quality care in PICUs, with the available resources to achieve the best outcome.

ACKNOWLEDGEMENTS

Author would like to thanks to all professors, associative and assistant professors, senior residents of department of pediatrics, Rajarajeswari medical college and hospital Bengaluru and to all the children and their parents who were the subjects of this study.

Funding: No funding sources

Conflict of interest: None declared

Ethical approval: The study was approved by the Institutional Ethics Committee

REFERENCES

1. 1. Nelson Textbook of Pediatrics, 21^st edition.
2. 2. Rogers Pediatric Intensive, Care, 4^th edition.
3. 3. Shann F, Pearson G, Slater A, Wilkinson K. Paediatric index of mortality (PIM): a mortality prediction model for children in intensive care. Intensive Care Med. 1997;23(2):201-7.
4. 4. Slater A, Shann F, Pearson G. Paediatric Index of Mortality (PIM) Study Group. PIM2: A revised version of the Paediatric Index of Mortality. Intensive Care Med. 2003;29:278-85.
5. 5. Slater A, Shann F, ANZICS Paediatric Study Group. The suitability of the pediatric index of mortality (PIM), PIM2, the pediatric risk of mortality (PRISM), and PRISM III for monitoring the quality of pediatric intensive care in Australia and New Zealand”. Pediatric Critical Care Med. 2004;5(5):447-54.
6. 6. Shann F. Are we doing a good job: PRISM, PIM and all that. Intensive Care Med. 2002;28(2):105-7.
7. 7. Pearson GA, Stickley J, Shann F. Calibration of the paediatric index of mortality in UK paediatric intensive care units. Arch Disease Childhood. 2001;84(2):125-8.
8. 8. Wolfler A, Silvani P, Musicco M, Salvo I. Pediatric index of mortality 2 score in Italy: a multicenter, prospective, observational study. Intensive Care Med. 2007;33(8):1407-13.
9. 9. Youssef MRL, Mosleh H, Labib JR. Assessment of the performance of the Pediatric Index of Mortality 2 (PIM2) among Egyptian pediatric patients admitted to the intensive care. Egyptian Pediatric Association Gazette. 2014;62:65-71.
10. 10. Salamati P, Talaee S, Eghbalkhah A, Chaman R, Mokhtari Z, Azarshahin M. Validation of Pediatric Index of Mortality-2 Scoring System in a Single Pediatric Intensive Care Unit in Iran. Iran J Pediatr. 2012;22(4):481-6.
11. 11. Sari DSP, Saputra I, Triratna S, Saleh I. The pediatric index of mortality 3 score to predict mortality in a pediatric intensive care unit in Palembang, South Sumatera, Indonesia. Paediatr Indones. 2017;57:164-70.
12. 12. Eulmesekian PG, Perez A, Minces PG, Ferrero H. Validation of pediatric index of mortality 2 (PIM2) in a single pediatric intensive care unit of Argentina. Pediatr Crit Care Med. 2007;8:54-7.
13. 13. Hariharan S, Krishnamurthy K, Grannum D. Validation of Pediatric Index of Mortality-2 scoring system in a pediatric intensive care unit, Barbados. J Trop Pediatr. 2011;57:9-13.
14. 14. Tibby SM, Taylor D, Festa M, Hanna S, Hatherill M, Jones G et al. A comparison of three scoring systems for mortality risk among retrieved intensive care patients. Arch Dis Child. 2002;87:421-5.
15. 15. Taori RN, Lahiri KR, Tullu MS. Performance of PRISM (Pediatric Risk of Mortality) score and PIM (Pediatric Index of Mortality) score in a tertiary care pediatric ICU. Indian J Pediatr. 2010;77:267-71.
16. 16. Thukral A, Lodha R, Irshad M, Arora NK. Performance of Pediatric Risk of Mortality (PRISM), Pediatric Index of Mortality (PIM), and PIM-2 in a pediatric intensive care unit in a developing country. Pediatr Crit Care Med. 2006;7:356-61.
17. 17. Qureshi AU, Ali AS, Ahmad TM. Comparison of three prognostic scores (PRISM, PELOD and PIM-2) at pediatric intensive care unit under Pakistani circumstances. J Ayub Med Coll Abbottabad. 2007;19:49-53.
18. 18. Sankar J, Chandel A, Dubey NK, Sreenivas V, Sankar MJ. Do interventions in an ICU affect the predictive ability of pediatric index of mortality and pediatric index of mortality2 scores in a tertiary care hospital? Pediatric Cri Care Med. 2013;14(2):e70-6.
19. 19. Hariharan S, Chen D, Merritt-Charles L. Risk-adjusted outcome evaluation in a multidisciplinary intensive care unit. West Indian Med J. 2007;56(3):240-5.
20. 20. Sankar J, Singh J, Sankar MJ, Joghee S, Dewangan S, Dubey N. Pediatric Index of Mortality and PIM2 Scores Have Good Calibration in a Large Cohort of Children from a Developing Country. BioMed Res Int. 2014;907871.
21. 21. Taori RN, Lahiri KR, Tullu MS. Performance of PRISM (Pediatric Risk of Mortality) Score and PIM (Pediatric Index of Mortality) Score in a Tertiary Care Pediatric ICU. Indian J Pediatr. 2014;77.
22. 22. Sankar J, Chandel A, Dubey NK, Sreenivas V, Sankar MJ. Do interventions in an ICU affect the predictive ability of Pediatric index of mortality and Pediatric index of mortality-2 scores in a tertiary care hospital? Pediatr Crit Care Med. 2013;14:e70-6.
23. 23. Gandhi J, Sangareddi S, Varadarajan P, Suresh S. Paediatric index of mortality 2 score as an outcome predictor in pediatric Intensive Care Unit in India. Indian J Crit Care Med. 2013;17(5):288-91.
24. 24. Pollak MM. Pediatric risk of mortality (PRISM) score. Crit Care Med. 1988;16(11):1110-6.
25. 25. Societe Francaise d’ Anesthesia et de Reanimation (SFAR). Scoring systems for ICU and surgical patients; PIM 2 (Paediatric Index of Mortality). 1997;23:201-7.
26. 26. Lemeshow S, Hosmer DW. A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol. 1982;115:92-106.
27. 27. OpenEpi. Standardized Mortality Ratio and 95 Confidence Interval. Available at http://www.openepi.com/oe2.3/menu/openepimenu.htm. Accessed on 12/1/2014.
28. 28. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiol. 1982;143:29-36.
29. 29. Shukla VV, Nimbalkar SM, Phatak AG, Ganjiwale JD. Critical Analysis of PIM2 Score Applicability in a Tertiary Care PICU in Western India. Int J Pediatr. 2014;703942.
30. 30. Taro Y. Statistics, An Introductory Analysis, 2^nd Edition., New York: Harper and Row. 1976.
31. 31. Netto AL, Muniz VM, Zandonade E, Maciel ELN, Bortolozzo RN, Costa NF et al. Performance of the Pediatric Index of Mortality 2 in a pediatric intensive care unit. Rev Bras Ter Intensiva. 2017;29(4):453-9.
32. 32. Aroor S, Kumar S, Kini P, Mundkur S. Applicability of Paediatric Index of Mortality 2 Score to Predict Outcome in Children Admitted to Paediatric Intensive Care Unit. Paediatric Intensive Care Unit. J Nepal Paediatr Soc. 2018;38(3):149-52.
33. 33. Ng DK, Miu T, Chiu W, Hui N, Chan C. Validation of pediatric index of mortality 2 in three pediatric intensive care units in Hong Kong. Indian J Pediatr. 2011;78(2):1491-4.