Remote temperature monitoring system for strengthening cold chain management: Belize experience

INTRODUCTION

Immunization has been recognized as one of the most successful public health interventions and effective cold chain management is essential in ensuing vaccine potency and the quality of the program.¹ The ability to monitor vaccines for optimal temperatures throughout the supply chain is a cornerstone of successful immunization programmes. The continuous monitoring of vaccine temperatures is imperative with all vaccines required to be maintained within a specific temperature range.² Effective, well-managed temperature monitoring is essential for ensuring vaccine quality throughout the supply chain and for preventing wastage of vaccines due to heat or freezing. The WHO guidelines recommend storing vaccines between+2°C and+8°C at all levels of the cold chain because exposure to heat or cold outside that range can adversely affect the immunological properties of the vaccines and thus reduce their potency.³ The global vaccine action plan (2011-2020) recommends the development of innovative vaccine supply chain technologies to ensure that potent vaccines reach their target population.⁴

Remote temperature monitoring (RTM) devices continuously monitor the temperature and allows for real-time vaccine cold chain equipment temperature monitoring.⁵ The data provided can detect cold chain equipment that need services or replacement.⁶ This enables better monitoring of performance of the cold chain equipment, allows a more rapid reaction from health workers to prevent damage to vaccines and regular analysis of data collected to inform appropriate decision making and action.⁷ The introduction of COVID-19 vaccines following the pandemic in 2019 brought more complexity to cold chain operations and therefore to temperature monitoring.⁸

Learning from the polio legacy on the use of polio investment to strengthen primary health care, Belize leveraged the COVID-19 vaccination rollout as a transformative opportunity for building resilient immunization programmes.⁹ With UNICEF support, government installed remote temperature monitoring devices to all the cold chain equipment in the country as part of the enhancement of the cold chain capacity for the integration of COVID 19 vaccine into the routine immunization system.¹⁰ This study aimed to describe the process of deployment of remote temperature monitoring devices to strengthen cold chain system and assess the effectiveness of the use of the technology.

METHODS

Belize with a population of about 400,000 is divided into four health regions and 6 health districts with each district providing primary and secondary care services. There is a National/ referral hospital, 3 Regional and 3 Community Hospital and 36 primary health care facilities and Immunization services are provided in all primary health care facilities and through mobile clinics. There is a national vaccine store and sub-national vaccine stores located at each of the 6 districts from where vaccines are deployed to all the primary health care facilities at urban and rural communities. The cold chain equipment includes 4 walking in cooler and 2 ultra-low temperature units at the national vaccine stores while ice-line biomedical refrigerators and combine biomedical refrigerators are used at the subnational and health facility level.

Study design

An exploratory study and used mix of qualitative and quantitative methods. The qualitative data was extracted from the project reports that described the process involved in the deployment of the Remote Temperature Monitoring Devices (RTMDs). Key Informant interviews (KII) were conducted for eight health workers responsible for immunization at the national, district and health facility levels. The quantitative method used retrospective review of records of temperature alarms extracted from the periodic report from the online RTM (remote temperature monitoring) dashboards between November 2023 and October 2024.

Data collection and analysis

The KII was done using a semi-structured interview guide conducted through phone calls to gather information on the health workers’ perception and experiences on use of the RTM to improve cold chain management. The interviews transcribed verbatim and analyzed using thematic analysis and the result organized into themes. The temperature alarm data from all the RTMDs was extracted using a form designed by one of the authors to capture temperature excursions (<+2^oC, >+8^oC), sites, duration of alarm, frequency of alarm and was analysed and presented in graphs.

The result is organized into two parts, the description of the process for the deployment of the RTM system and, program result which is both quantitative from the analysis of the temperature alarm reports and qualitative from the KII done with the HCWs.

The description of the process for installation of the remote temperature monitoring system. This is divided into 3 phases. Technical screening of the cold chain system. Training and installation of the RTM. Supportive supervision and monitoring.

Phase 1: Technical screening

This was done in 2022 as part of the process for strengthening of the cold chain network in the country for the integration of COVID 19 vaccine into the routine immunization system.

The process for the introduction of RTM started with technical screening of the cold chain equipment (CCE) including the temperature monitoring devices being used at the national and the health facility level and reliability on internet connectivity at the health facility to identify the types of RTM device that will be suitable. The assessment was done using WHO cold chain equipment inventory and gap analysis tool, a web based technical screening tool which is a planning tool for conducting and analyzing cold chain equipment inventory.¹¹

Some of the findings from the screening showed that most of cold chain equipment (ice-line refrigerators) at the health facilities used stem cell thermometers while the national vaccine store had an alarm temperature monitors from Beyond wireless (old model). The health facilities in the rural areas have poor internet connectivity. Following the screening, it was decided that remote temperature monitors be used to replace all temperature monitoring devices at the national and health facility levels. This nationwide use of RTMDs was aimed to improve and monitor cold chain equipment performance to ensure vaccine potency, reduce wastage and tracking of cold chain maintenance to inform long-term equipment planning needs.

In addition, 2 types of RTM devices from different manufacturers (Beyond wireless Limited and Haier Limited) were agreed to be used to replace the temperature monitors based on the specific features of the 2 devices.

The RTMD manufactured by beyond wireless has 4 sensors but uses local sim card for internet connectivity and so can only be used where there is reliability of internet connectivity. It was recommended to be used for the national walk-in coolers and the subnational cold rooms where a device can be used to connect four refrigerators. The walk-in coolers, have a device that receive input from the 4 sensors: 2 sensors (1 on top shelf and 1 on bottom shelf recorded the refrigerator interior temperatures; 1 sensor (attached to door jamb) monitored the open-closed door status; and 1 sensor (on wall outside refrigerator) monitored the ambient room temperature. At the subnational cold room, the device with 4 sensors is connected to 4 refrigerators.

The U-cool haier RTM manufactured by haier limited has only one sensor and can be only used for one refrigerator and can be used in the rural health facilities where there is limited/poor internet connectivity because it uses global SIM which works in areas where there is no or limited local internet connectivity.

The RTMDs are primarily powered by the electrical grid but they have built-in batteries that will power the devices for up to four hours in case of a power outage.

The technical screening dashboard shows the number of sites being monitored, the type of hardware and accessories used (e.g. type of main power available at the site, main power plug, spare sensors available), type of sim card used (local or global), APN settings for local sim cards, subscription period and dashboard language, installation and training done (see Figure 1).

Phase 2: Training and installation of RTMDs

The required number of cold chain equipment at the national and health facilities following the technical screening was used to determine the number of the RTMD to be procured so that all the cold chain equipment have the RTMD connected to them. The procurement process was supported by the UNICEF supply division immunization technology center that sourced the device from the suppliers (Beyond Wireless Technology and Haier Biomedical companies) after the technical screening recommendations.

Series of trainings were done following the procurement of the devices for local contractors and the MOHW’s national engineering and maintenance center (NEMC) team and cold chain manager who did the installation of the devices with virtual support from the technical team from the suppliers. At this session, the detailed function, installation and settings were explained by the suppliers.

The beyond wireless RTMDs were installed in the 4 WIC at the national vaccine store and 6 subnational vaccine stores located in the 6 districts. The U-Cool Haier RTMDs were installed in 28 refrigerators in 27 primary health care facilities in the country.

The RTM system consisted of 2 major parts, the hardware and the dashboard. The hardware is a global system for mobile communication (GSM) that is, connected to a cellular network with temperature. The sensor probe(s) were placed inside a vaccine refrigerator, with the main body of the device positioned nearby and mounted on a wall. The system uploaded temperature and grid power availability data to a server using cellular networks.

The second part of the RTM system, the dashboard, organized and displayed the collected data through various visualizations and analytics to inform decision making for technicians and managers.¹²

The process for the installation of all the devices on all the cold chain equipment took between 6-8 months. In addition, training was also done by the manufacturers for the HC who manage the cold chain and immunization services in all the health facilities about the devices and Standard Operating Procedure developed to guide both the installation and other details required to get familiarize with the functions and management of the devices including how to respond to temperature excursion alarms from the RTM system, maintain cold chain equipment and escalate unresolved cold chain issues to the national levels to be addressed when appropriate

Remote temperature recording

The most important feature of the RTM system is the capability to notify recipients of alarm conditions, alarms which are often referred to as excursions.

Beyond wireless RTM

There are 3 layers of alarm system set for the sensors based on the temperature change and in line with the WHO regulation.¹³

The 1st layer alarm is raised when the temperature is more than 8 degrees Celsius (>8^oC), but the alarm is set with a delayed time of one hour before sending alert via SMS, email or App to the recipient.

The 2nd layer alarm is raised when the temperature is less than +2^oCelsius (<+2^oC), but the alarm is set with a delayed time of 2 hours before sending alert via SMS, email or App to the recipient. The 3rd layer alarm is raised when the temperature is less than minus 0.5 degree Celsius, but the alarm is set with a delayed time of 30 mins before sending alert via SMS, email or app to the recipient. Alarm recipients are in 3 levels. 1st level recipient is the cold chain focal person at the health facility level who first receive the alert via SMS, while other individuals/group at the same health system level are also notified of the alert via email. If there is no response from the 1st level recipient who received the alert after 15 mins, the alarm is sent to the second level recipient who usually is at the next health system level (district level). Similarly, if after 15 mins, there is no response the alert is sent to the 3^rd level recipient who is at the regional or national level so that actions are taken timely. However, if the alarm is resolved (i.e. sensor goes back into range), by the 1st level recipient then Stage 2 or 3 may never be reached and their recipients will not be notified at all and the first recipient will receive a notification that the alarm is resolved.

For the U-Cool Haier, (Figure 2) it is attached to the refrigerators, the sensor is attached to only a device and the alarm is raised when the temperature is more than +8 degrees Celsius (>+8^oC), but the alarm is set with a delayed time of 20 minutes before sending alert via SMS, email or App to the recipients- the officer managing the cold chain equipment at the facilities levels and other officers at the district, regional and national level depending on the set up of recipients in the systems. Initially only the HCW who manage the cold chain equipment at the health facility level received the alert, however, to improve accountability and timely response to the alert, the supervisor at the district and national level were included as recipients.

Phase 3: Supervision

Regular planned/schedule monitoring visits were conducted by the cold chain manager and the technicians to all health facilities to reinforce their knowledge and skills and to provide hands on support to the HCW. The manager also used the visits to assess all the cold chain equipment, trouble shoot for any problem and conduct preventive maintenance of the cold chain equipment and replace any parts of the cold chain equipment or RTMD when needed.

Figure 3 show the temperature monitoring dashboard for a health facility device for the month of October 2024. It shows the temperature ranges during the month with temperature of more than +8 degree Celsius (>+8^oC) reported some days in the month. Figure 4 shows the sample of SMS alerts received from the Remote temperature monitoring devices by recipients of the temperature alarm when there is temperature excursions from any of the devices.

PROGRAM RESULTS

Analysis of the alarm report from the remote temperature monitors

The data on temperate pattern was extracted from the periodic report from the online RTM dashboards between November 2023 and October 2024. The data was collected from all the 8 Beyond wireless devices and the 28 Haier devices on temperature excursions (<+2^oC, >+8^oC), site, duration and frequency analysed and presented in graphs.

Figure 5 shows the temperature alarm reports from the Beyond wireless Remote Temperature devices connected to the 4 national WIC and 6 districts cold chain equipment. During the one-year study period, a total of 18 alarms were reported in 8 months from only four of the eight devices (1national and 3 hospitals) which ranged from one alarm per month to 6 alarms per month with average of 2.6 alarms per month. The duration of each alarm was from 0.6 hours to 31.3 hours with average of 9.6 hours with 75% of the alerts resolved within 24 hours. All the alarms occurred when the temperature was more than +8℃ and no case of alarm dure to temperature less than 2 degree was reported.

The major cause of alarm was failure of the electricity supply and doors not properly closed and faulty thermostat. Most of the alarms occurred during the week and few cases during public holidays or weekends. Seventy-eight percent (78%) of the alerts were resolved by the 1st recipient – closure of the doors and removal of vaccine to other areas especially when there was power failure or fault in the electricity or generators and resettling of the thermostat. Only 22% of the alerts were escalated to the 2nd or 3rd level recipient before being resolved.

Figure 6 shows the report of alarms from 28 Haier RTM devices installed on 28 refrigerators in 27 health facilities. During the one-year study period, a total of 428 alarms were reported in 12 months from between 2-7 devices which ranged from 23 alarms per month to 56 alarms per month with average of 35.6 alarms per month.

The duration of each alarm was from 7.3 hours to 28.2 hours with average of 12.2 hours and 91% of the alarms resolved within 24 hours. All the alarms occurred when the temperature was more than +8^oCelsius and no case of alarm dure to temperature less than 2 degree was reported. The major cause of alarm was the doors of the refrigerators not properly closed and failure of the electricity supply usually for a short period of a few hours. Most of the alarms occurred during the week and few cases during public holidays or weekends.

Qualitative data from the key informat interview done with the health workers

The result is organized into two themes: (1) Perception and experience of health workers on the use of the devices in improving temperature monitoring and (2) Health system support in ensuring timely response to temperature excursions.

Perception and experience of Health workers on the use of the devices in improving temperature monitoring

All the respondents generally agreed that having the RTM system and receiving alarms helped them to be more aware of and responsive to temperature excursions. All the respondents reported the remote temperature control devices provided them confidence and assurance of the safety of the vaccines in the refrigerators and made their work much easier.

‘The RTM has made life very easy for us to monitor the temperature of the cold chain. Before when we were expected to be recording the temperature twice daily, but this was difficult and we sometime forgot because we are always very busy’ KI1.

‘The RTMD give me a lot of assurance about the safety of my vaccines, I don’t get worried since I know if anything goes wrong it will send an alarm to me and I will act on time or even take the vaccines to somewhere else safe if the problem cannot be resolved on time ‘KI2.

‘It is very easy to use and understand as it is user friendly. It provides surety that the vaccine in my refrigerators is maintained within the required temperature as I can review the history of the temperature over time It has made life easy for us’ KI3.

‘We have not experienced any issues since the RTMD was installed, however about 3 weeks ago when there was power outage at the health center, I received an alarm alert and I removed all the vaccines from the refrigerator using cold boxes and transferred them to the district urban health center where the district store for vaccines is located which helped in preventing the vaccines from being damaged due to temperature excursion.’ KI4

Health system support in ensuring timely response to temperature excursions

Despite the understanding of the health workers about how the devices works and the process for alert reporting when there are temperature excursions, sometimes the response was very slow either because the health workers do not have the capacity to respond or attitudinal /behavioral issues of not being responsive. Since the RTM were introduced to identify excursion for timely response, the MOHW adopted some measures which included regular training and supervision and inclusion of the supervisors of the 1st recipients and supervisors at national level of the alerts as secondary and tertiary recipients of the alerts.

‘Initially there was delayed response to alarm by the health workers because of not usually checking their phones or email for alerts sent especially during the weekend, however, since their supervisors were included as recipients of the alerts the health workers at health facility level are now responding better to alerts’. KII6

‘The technology is still very new and so some health workers are still getting used to it especially what to do when they receive the alert. However, the regular training and supervision by the cold chain manager has been very useful in ensuring better and timely response.’ KII 7

‘Anytime there is an alarm alert and we have tried all we can do at the health facility to correct the alarm, we usually call the cold chain manager who is always available to guide us in trouble shooting the problem. This has improved our skills and capacity to resolve most of the alerts we receive’ KII 8.

DISCUSSION

The study described the deployment of the Remote Temperature Monitoring system to improve the vaccine cold chain performance. The alarm received through short message services (SMS) and email when there is temperature excursions alerted the health workers at all levels to respond promptly. The heath workers in the study generally agreed that the use of the Remote temperature monitoring devices to replace the stem cell thermometers and receiving alarms helped them to be more aware of and responsive to temperature excursions. This is like the studies on RTM where HCW agreed that having the RTM devices made their jobs much easier and beneficial to their ability to monitor the state of the vaccines.^5,7

The response time to alerts by the first recipients who are the focal persons for immunization at the health facilities in the study improved with the inclusion of their supervisors as recipients of the alerts. The inclusion of the supervisors as recipients of the alarm alerts is a form of accountability framework established to improve the response to temperature excursions.

This form of accountability mechanism has been advocated in various studies which found that despite the access to data on continuous temperatures there were still lack of responses to documented excursions by health workers.^14,15 The purpose of investing in the technology is defeated if there is no timely response to temperature alerts due to the attitude or behaviors of the health workers.^14,16 A study in Albania reported that procedural compliance by health workers increased due to the knowledge that supervisors are automatically informed of temperature excursions.¹⁷

In addition, the inclusion of the supervisors in the study as recipients of the temperature alerts also ensured that all alerts are followed up by the supervisors and all support needed is provided to fix the problems, some of which may be beyond the capacity of the first recipients, like changing, repair or regulation of thermostat or electrical problems. Similarly, the cold chain officer who received all the alerts followed up with the health facilities to support by phone in addressing the concerns and when these can’t be fixed via virtual guidance, the cold chain manager would give priority to the matter and go to the health facilities to support physically at the health facility level. This is similar to the finding in study in Tanzania and Albani which reported that the awareness and involvement of supervisors reinforced the process of seeking solutions through tracking of cold chain equipment performance remotely and following up with facility-level health workers to address immediate concerns and provide the needed support including the replacement of inadequate equipment.^17,18

There is still the need to improve on the response to alerts especially when it happens over the weekend or public holidays when the staff members live far from the health facilities and may have challenges with transportation. This can be addressed by the hospital management providing transportation for the health workers or ensuring staff who are on duty or live close to the health facilities are trained on what to do and have access to the cold chain equipment and can be asked to respond timely. Lack of transportation and poor coordination of resources are some of the factors found to be responsible for lack of response to documented excursions.^14,15

Monthly monitoring and supervisory visits by the cold chain officer to reinforce the knowledge and the skills of the health workers on the causes of temperature excursion and what to do including resettling of thermostat contributed to improved management of alarm alerts as part of the health system support initiated in the study. The need for continuous training of all health workers involved in immunization services on device maintenance and data interpretation and developing standard operating procedures for responding to alarms has been identified as an initiative to improve knowledge and skills to be implemented as a complement to RTM system implementation and use.^5,6,17

CONCLUSION

The study contributes to existing knowledge on the use of remote temperature monitoring system to improve cold chain management. The alarm alerts received by the health workers when there are temperature excursions ensures prompt response to maintain the cold chain and vaccine potency even when the health facilities are closed. However, the deployment of RTMD alone is not sufficient to ensure effective cold chain management, the attitudes and practices of the health workers are essential in ensuring the investment in the RTM system is not waste.

There is need to include the monthly report of the RTM as part of the monthly immunization report and be discussed during Immunization review meetings at the health facility, district and national levels to promote the use of the data for action.

Funding: No funding sources

Conflict of interest: None declared

Ethical approval: Not required

REFERENCES

1. 1. World Health Organization (WHO). Immunization coverage. WHO. Available at: http://www.who.int/news-room. Accessed on 14 October, 2024
2. 2. World Health Organization (WHO). WHO Vaccine Management Handbook. Module VMH-E2-0.11. How to monitor temperatures in the vaccine supply chain. Geneva: WHO. 2015. http://apps.who.int/iris. Accessed on 18 October 2024.
3. 3. World Health Organization (WHO). How To Monitor Temperatures in the Vaccine Supply Chain. Geneva, Switzerland: WHO. 2015. https://iris.who.int/bitstream. Accessed on 22 November 2024.
4. 4. MacDonald N, Mohsni E, Al-Mazrou Y, Kim Andrus J, Arora N, Elden S, et al. Global vaccine action plan lessons learned I: Recommendations for the next decade. Vaccine. 2020;14(33):5364-71.
5. 5. Lutukai M, Bunde EA, Hatch B, Mohamed Z, Yavari S, Some E, et al. Using data to keep vaccines cold in Kenya: remote temperature monitoring with data review teams for vaccine management. Glob Health Sci Pract. 2019;23;7(4):585-97.
6. 6. Cavallaro KF, Francois J, Jacques R, Mentor D, Yalcouye I, Wilkins K, et al. Demonstration of the Use of Remote Temperature Monitoring Devices in Vaccine Refrigerators in Haiti. Public Health Rep. 2018;133(1):39-44.
7. 7. WHO and UNICEF 2017: Perception of health workers on the use of remote temperature monitoring systems. Evidence Brief https://iris.who.int/bitstream. Accessed on 16 December 2024.
8. 8. Pambudi NA, Sarifudin A, Gandidi IM, Romadhon R. Vaccine cold chain management and cold storage technology to address the challenges of vaccination programs. Energy Reports. 2022;8:955–72.
9. 9. Cochi SL, Freeman A, Guirguis S, Jafari H, Aylward B. Global polio eradication initiative: lessons learned and legacy. J Infect Dis. 2014 Nov 1;210(1):540-6.
10. 10. Oladeji O, Largaespada BN, Baitwabusa A. COVID-19 vaccination: opportunity to strengthen immunization system in Belize. European J of Med and Health Sc. 2023;5(4):30–4.
11. 11. WHO. Cold chain equipment inventory and gap analysis tool. 2021. Available at: https://www.who.int/publications. Accessed on 14 September 2024.
12. 12. Remote Temperature Monitoring Devices. Available at: https://supply.unicef.org/s0002548.html. Accessed on 22 December 2024.
13. 13. World Health Organization. Category E006: temperature monitoring devices. Available at: https://apps.who.int/immunization_standards/vaccine_quality/pqs_catalogue/categorypage. Accessed on 22 September 2024.
14. 14. Lennon P, Atuhaire B, Yavari S. Root cause analysis underscores the importance of understanding, addressing and communicating cold chain equipment failures to improve equipment performance. Vaccine. 2017;35(17):2198-202.
15. 15. Brison M, LeTallec Y. Transforming cold chain performance and management in lower-income countries. Vaccine. 2017;35(17):2107-9.
16. 16. Ashok A, Brison M, LeTallec Y. Improving cold chain systems: challenges and solutions. Vaccine. 2017;35(17):2217-23.
17. 17. PATH. Assessment of a remote alarm system for vaccine storage in Albania. Seattle: PATH. 2011: 1-30.
18. 18. Remote Temperature Monitoring of Tanzania's Vaccine Cold Chain Improves Visibility into Equipment Performance. 2022. Available at: https://www.jsi.com/resource/remote-temperature-monitoring-of-tanzanias-vaccine. Accessed on 1st December 2024.