Perceptive evaluation of the severity of dysphonia among Lebanese bilingual subjects

INTRODUCTION

Dysphonia is the most frequently encountered vocal dysfunction.¹ In general; it is considered a manifestation of a biomechanical, physiological or dysfunctional problem.²

This vocal dysfunction can be easily diagnosed. However, the evaluation must be multidisciplinary, including a phoniatric assessment and a speech-language evaluation.¹ In this context, perceptual judgment remains essential. It reflects the awareness that the patients have of the changes in vocal characteristics that prompt them to consult.³ The dysphonic voice is generally rough, breathy and hoarse.⁴ These characteristics are well defined in the Hirano GRBAS scale,³ known for its reliability and validity. This scale reduces the subjective character of perceptual evaluation of voice.^5-6 It is an available resource that allows quantification of the alteration of voice characteristics and is used worldwide. ^7-8 It is concise, easy to use and gives reproducible results. Moreover, it remains valid in the presence of most laryngeal abnormalities.⁹ However, several studies suggest a biased assessment affected by several factors, such as the language used by the patient and the language skills of the evaluator.³

The Lebanese population is deeply concerned by language as the Lebanese live in a multilingual society. In Lebanon, the use of at least two languages, including Lebanese Arabic, French and English, is an integral part of the culture and the educational system.

Bilinguals use different phonetic systems. In fact, French is considered as an anterior language because of the large number of lower vowels compared to the reduced number of posterior consonants.¹⁰ English has a high density vocal system and a complex index system for the characterization of vowels (duration, accentuation).¹¹ Lebanese Arabic is a language with poor vocalism. It is rich in consonants, most of which are posterior.¹⁰ The exposition of the different vocal systems accounts for the articulatory antagonism that bilingualism illustrates. These observations led us to suppose that if the articulatory constraints have an influence on the vocal production (fundamental, usual frequency and timbre), then audible differences would be noticed in the perception of the severity of dysphonia.¹²

Since the Lebanese dysphonic patient is part of this multicultural environment, it would seem reasonable to assume that the perception of dysphonia might change depending on the spoken language. It is from this postulate that this research topic emerged.

This study aims to search for relevant information to characterize dysphonia depending on the spoken language. It attempts to explore potential links between the perceptual criteria, bilingualism in Lebanon, and their possible effects on the voice in the context of dysphonia. Its hypothesis was that voice seems more dysphonic when the Lebanese bilingual patient speaks Lebanese Arabic.⁴

METHODS

The study was conducted from September 2015 till May 2016.

Population

Thirty dysphonic patients were invited to participate. They were all existing patients at ENT (Ear, Nose and Throat) or speech therapy clinics. The etiology of their dysphonia was not taken into consideration. However, the clinical diagnosis had to be documented through a laryngofiberoptic examination.

The age of the patients varied from twenty to sixty years. Candidates were recruited from different regions in Lebanon, knowing that the regional accent does not have an influence on the perception of dysphonic voices.³

The participants had to speak and read two of these three languages fluently: Lebanese Arabic and French or Lebanese Arabic and English. The level of knowledge of the language was determined through the completion of a questionnaire specifically developed for the study. It was inspired by the QPEB (Parental Questionnaire for Bilingual Children, QPEB, COST ISO 804) and the Quebec scale of French proficiency levels for adult immigrants.^13,14

Description of the population

The initial sample consisted of twenty-eight Lebanese dysphonic patients, twenty-six of whom were women and two men. However, men were eliminated in order to have a more homogeneous sample. Four women were later excluded as they did not fulfill all the criteria necessary for language knowledge. The final sample consisted of twenty-two women. They had different laryngeal disorders. The anonymity of patients was maintained in the collection of information and the coding of sound files.

Table 1 shows the distribution of patients according to their laryngeal pathology.

Among the twenty-two patients speaking Lebanese Arabic fluently, thirteen could speak and read French very well. The nine remaining patients were able to speak and read English appropriately.

Presentation of the phonetic material

Phonetic research clearly indicates the need for standardized language material in voice assessment.¹⁵ As such standardization is non-existant in Lebanese Arabic, we set out to select a standardized corpus for the Lebanese Arabic language before proceeding with the study.

Corpus in French

A passage from the story of "Mr. Seguin’s goat” by Alphonse Daudet was chosen. It had been reported as the most used phonetic material in the assessment of voice in the French language.⁸ The sentences chosen for the perceptual analysis “He left them all (…) retained them” were selected because they include a succession of voiced/unvoiced transitions and vocalic continuums. Moreover, their prosodic structures induce a natural accentuation, especially on the word "all”.¹⁶

Corpus in English

The "Rainbow Passage" is the most used text by native English speakers for the evaluation of voice.¹⁸ It was authored by Fairbanks G. in 1960. The selected passage goes from “the rainbow (…) horizon” since it follows the same criteria as in the French selection.

Corpus in Lebanese Arabic

In Lebanon, there is no consensus on the use of a reference text for voice assessment. Therefore, we wanted to propose a text with the same linguistic and prosodic characteristics as those which are mainly used in French and English. However, finding the appropriate one was a real challenge for two main reasons:

(a) Lebanese is considered a language that cannot be written.¹⁸ Thus; the use of a text written in literary Arabic does not give the right clues as the patient does not practice it on a daily basis unless there is an academic or official obligation to do so.

(b) The selection or development of such phonetic material is essential as it will be used systematically as a reference for the evaluation of voice quality; allowing a more rigorous assessment.

Different criteria were taken into consideration in selecting the corpus. As a first step, the passages presented in French and English were analyzed. The following criteria were found to be common to both selected texts:

• • They do not have an affective burden.
• • They are characterized by a situation of departure and a problem situation.
• • They contain long sentences.
• • There is a moral in both the French and English texts.
• • The intonation is well marked in both the French and English texts. This led to foresee a question in the corpus in Lebanese Arabic.

One text seemed to fit expectations. It was taken from the book entitled “dot after dot (…) make a sea” by Nadine Touma.¹⁹

We wanted the Lebanese Corpus to be phonetically balanced so it can be used as a reference corpus to evaluate the language of people who stutter or people having dyspraxia for example.

Following the advice of the first phonetician consulted, the whole story was phonetically transcribed. Then, the number of voiceless consonants (such as [t] and [p]) was counted as well as the number of consonants and the number of vowels. When a text is phonetically balanced, the number of sound consonants added to the number of vowels has to be about twice the number of deaf consonants.²⁰ The presence of deaf consonants is of great importance as these consonants are difficult to vocalize by dysphonic patients.²¹ Indeed, a study on the analysis of the different phonemes related to several languages, including French, emphasizes the role of deaf consonants in the perception of dysphonia.

This phonetic analysis allowed us to identify the part of the story that was the most phonetically balanced. The passage chosen in Lebanese Arabic had two hundred seventy-five deaf consonants, four hundred fifty-one sound consonants and four hundred ninety-three vowels.

Two phoneticians validated the selected text in Lebanese spoken Arabic.

Four non-dysphonic adults (two men and two women) read the text in order to test its flow and the time needed to complete the reading.

After these factors were checked, the part that had to be read by the patients was chosen. It was neither at the beginning nor at the end of the story. It was composed of fifty syllables and could be read in about two minutes. It stretched from “One of these days (…) accompanies him”. Moreover, the selected sentence meets the same criteria as the French sentence and the English sentence (Annex 1).

Table 2 summarizes the characteristics of the three extracts:

Recordings

The same instructions were given to all patients by the same speech therapist (researcher). Every text had to be read at a comfortable speed and intensity as if it was spoken orally.

Before starting the recording, the speakers were able to get familiarized with the text as long as they wanted. Moreover, the order of the presentation of the texts was randomized.

The patient was seated. The voices were recorded using a Logitech Headset H600 (wireless and noise-filtering) to keep the mouth-to-microphone distance at three centimeters for all speakers.

Voices were recorded in real-life speech therapy practice. Recordings were made at non-extreme times, as morning or evening hours are known to have an effect on the voice.²² A short break (not more than five minutes) was offered on demand following the reading of a text.

Goldwave software was used for data recording as it partially isolated background noise (www.goldwave.com).

The data was collected and arranged on the Praat software as Wav files per patient and per language. This software was chosen because of its availability and its wide use in phonetic and acoustic studies.²³

Listening panel

Composition of the panel

The listening panel consisting of four expert listeners performed the perceptual evaluation of voice and speech. Kreiman et al. (1990) have shown that expert jury possesses a rich palette of internal referents and auditory perception. This led us to select a group of expert listeners who were speech-language pathologists with at least three years of experience in voice rehabilitation.¹ Moreover; the listeners were fluent in the three targeted languages: Lebanese Arabic, French and English.²

Role of the jury

Each member of the jury completed the evaluation grid and judged the quality of voices separately. The jury had to analyze the recorded voices by quoting the criteria G, R and B of the GRBAS scale.²⁴

Listening protocol

The listening sessions were carried out individually in order to eliminate the chance of influence between the auditors. These were held in a quiet room. The jury was informed that forty-four recordings will be presented randomly in three different languages: French, Lebanese Arabic and English.

Each listener was allowed to rerun the recording that was just listened to, but could not play back any previous recording. Two listening sessions were scheduled one week apart to avoid memorization. However, members of the jury were not informed in advance of the existence of two separate listening sessions. This allowed better evaluation of intra-judge reliability. To be consistent, the procedure for each listening session was strictly standardized, including instructions given to members of the jury. It was presented to them in the following way: “You are invited to listen to different voices presented in three different languages, French, Lebanese Arabic and English in order to make a perceptual evaluation of voices. Recordings are separated by ten seconds. This time is reserved for the quotation of the G, R and B criteria of the GRBAS scale. We remind you that criterion G refers to the overall grade of dysphonia. Criterion R grades roughness of the voice. Criterion B refers to the breathy character of the voice. These three parameters are graded in four levels (from zero to three where zero corresponds to a normal voice and three corresponds to severe alteration). You can play back the recording if it was not well heard the first time.”

Editing voice samples

The speech samples were digitized and mounted on iTunes (www.itunes.com). The presentation of sound stimuli was randomly performed to avoid possible bias generated by a logical link between voices.²⁵ The order and the patient code were modified at each listening session.

Evaluation scale

Choice of scale

The HIRANO GRBAS (1981) was chosen as a perceptual evaluation scale as it is the most used tool, regardless of language. The criteria G, R and B were selected because of their high reliability.¹

The European Laryngological Society (ELS) indicated that the three parameters G, R, and B were sufficiently reliable to assess dysphonia in a clinical setting.²⁶

Evaluation grid

Each listener had the same computer with an Excel sheet containing the forty-four recordings that had been anonymized according to the order and the parameters of the used scale. No auditor was able to use the previous evaluation sheet.

Presentation of the statistical tools used

For statistical analysis, the PASW version 22 software was used. Since the scores obtained do not have a normal distribution and the population is small, nonparametric tests were used.

In order to verify the hypotheses related to the study, the Kappa coefficient was used to compare the answers given by the different judges. The Pearson correlation was used to estimate the consistency of each individual judge. The Wilcoxon or Mann Whitney U test was used to determine statistical significance (p˂0.05).

RESULTS

The results showed that the voice of the dysphonic patients tended to appear more dysphonic, rougher and breathier when they speak French compared to English and Lebanese Arabic.

Table 3 unveils the G, R and B scores according to languages: Lebanese Arabic, French and English.

The results show that the voice appears more dysphonic with a more pronounced roughness when reading in French for bilingual French/Lebanese Arabic. That, however, was not the case for bilingual English/Lebanese Arabic. These differences were not significant which suggests that the findings need to be corroborated with a study involving a larger number of participants in order to verify this hypothesis.

In order to better understand the preliminary results obtained, the sample was divided into two groups. Group 1 included the Lebanese Arabic speakers who also speak French fluently (n=13). Group 2 included the Lebanese Arabic speakers who are also fluent in English (n=9). A comparison of the Lebanese Arabic language was made between group 1 and group 2 as any differences could potentially impact interpretation of the results.

Table 4 shows the results obtained for each criterion G, R and B.

Group 1=French speaking group ; Group 2=English speaking group.

There were no statistically significant differences between the two groups. However, the p value for the overall score was close to 0.05. This suggests that group 1 has an overall Arabic language score that is higher than group 2. Coding was set as follows: 0 for absent, 1 for light, 2 for medium and 3 for severe. The score thus increases with severity. By comparing two means on criterion G, 1.88 (table 3) and 1.77 (table 4), we found that dysphonia tended to be more severe among women speaking French and Lebanese Arabic than those speaking English and Lebanese Arabic. This was supported by comparing the two averages on criterion G 1.39 (table 3) and 1.25 (Table 4).

As the coherence difference was not significant, a separate inter- and intra-judge coherence study was suggested by the experts.

Table 5 shows agreement of the expert auditors on the evaluation of the G, R and B criteria by language.

*Kappa statistically significant with alpha = 0.05 bilateral. Note: non-significant concordance = no concordance or discordance (as if the choice made by the judge was random).

Concordance for each expert judge (all languages):

For judge a: the significant Kappa agreement fluctuated between 0.44 and 0.64 which indicates moderate concordance. It should be noted that for the French language, concordance was very low, almost non-existent (especially for the G and the R).

For judge b: the significant Kappa agreement fluctuated between 0.39 and 0.53 which indicates moderate concordance. However, this judge had five items out of nine where concordance was not statistically significant (all R and B in Arabic and English).

For judge c: the significant Kappa agreement fluctuated between 0.35 and 0.67 which indicates moderate concordance. This judge had six out of nine items where concordance was not statistically significant.

For judge d: the significant Kappa agreement fluctuated between 0.44 and 0.8 which indicates moderate to good concordance. This judge had only one item that was not significant (the R in French). He was considered the most consistent.

Concordance for each expert judge according to each language

For Lebanese Arabic

Kappa fluctuated moderately between 0.36 and 0.56. Three measurements out of twelve did not have a significant kappa; two of these three measurements were from judge b. This suggests that for the Arabic language, a judge could be moderately concordant with himself.

For the French language

Kappa fluctuated between 0.35 and 0.69. French was the language with the most measurements not reaching significance (7 out of 12); especially for judge c who practically showed no concordance at all as if his score was random.

For the English language

kappa fluctuated between 0.52 and 0.8 suggesting moderate to good concordance. The number of non-significant measurements was four. So, overall, it could be implied that English was the language where the experts tended to be most consistent with themselves.

A comparison of the choices between the judges is presented in table 6. It shows the correlation coefficients obtained following the application of the Spearman correlation test.

The correlation coefficient for the judges’ choices fluctuated between 0.513 and 0.930. Such correlation ranged from moderate to very good (especially judges a and d). The English language had the highest coefficients, followed by Lebanese Arabic and lastly French. The number of measurements with no significant correlation totalled eighteen: six for Lebanese Arabic, eight for French and nine for English. While it was difficult to conclude with certainty in which language the expert judges’ choices correlated the most, it seemed that such correlation was best for the English language where almost all of them agreed on the judgement of voice quality.

As the results were not statistically significant, and because the number of patients and the number of judges were small, we decided to focus on three specific cases. We chose three patients who speak Lebanese Arabic and French and who have three different levels of dysphonia: mild, moderate and severe. We selected Lebanese Arabic and French because their correlation coefficient ranges were similar. We compared the results given by the experts who assessed the voices of these three patients.

The patient with mild dysphonia appeared to be more dysphonic when speaking Lebanese Arabic according to all experts except one who believed that the voice tended to be more dysphonic in French. The patient with moderate dysphonia seemed more dysphonic in French with unanimous agreement by the judges. The patient with severe dysphonia was more dysphonic in Lebanese Arabic according to all expert judges except one.

*Statistically significant correlation with alpha= 0.05 bilateral.

DISCUSSION

The perceptual assessment of dysphonia remains the gold standard for evaluating voice quality, and the GRBAS scale by Hirano is one of the most well-known methods used for this purpose.¹

Continuous speech recording is a method that reliably reproduces the natural use of voice. Compared to sustained vowels, it is more suitable for the perceptual evaluation of dysphonia. However, the evaluation of dysphonic voices in continuous speech could be influenced by the linguistic features of both speaker and examiner.³

The results revealed that voice tended to be more dysphonic, rougher and more breathy in the French language (Table 3). However, and according to the literature, a bilingual French/Lebanese Arabic subject would appear more dysphonic when reading Lebanese Arabic. Indeed, Al-Makarem et al. in 2007 showed that Arabic speakers had a tendency to speak with a higher intensity.²⁷ This would therefore affect the fundamental frequency that would increase in Arabic when compared to French for example.²⁸ Nevertheless, that study was conducted on a text in standard Arabic and not in Lebanese Arabic. It could then be assumed that the assessment of dysphonia might not be the same when patients express themselves in standard Arabic as compared to Lebanese Arabic.

In order to shed more light on this result, patients were divided into two groups (Table 4). Patients in group 1 who speak Lebanese Arabic and French appeared to be more dysphonic when speaking Lebanese Arabic than the patients in group 2 who speak Lebanese Arabic and English. This finding seems to correlate with the results showed in table 3 where patients appeared to be more dysphonic in French, and could be related to the choice of sample. It is possible that patients from group 1 might have a more pronounced dysphonia than patients in group 2. This leads us to suggest repeating the same evaluation on a more homogeneous sample with patients having the same severity level of dysphonia. Doing so would ensure a more objective judgment of the severity of dysphonia as it relates to the subject’s spoken language.

As the results between Lebanese Arabic and French were very close for criteria G, R and B, we attempted to understand why voice tended to appear more dysphonic in French contrary to expectations. We tried to analyze the results given by experts on six different corpuses relative to three dysphonic patients each having a different degree of severity. We noted that the expert jury was concordant in cases of mild and severe dysphonia. However, the experts were more discordant when the dysphonia was moderate. This observation is in agreement with the literature that describes the ease of rating such voice categories.¹

These results underscore the need to further study the effect of change of language on the characteristics of dysphonia in bilingual patients. It would first be necessary to obtain a greater variety of speech samples in order to standardize the perceptual evaluation of voice. Additional studies on the objective evaluation of voice while using the same phonetic material would also be needed. More specifically, voice intensity should be considered in French and Lebanese Arabic for each bilingual Lebanese subject in this population. This would allow us to factor in the intensity aspect which, according to research, seems to be higher in Arabic than in French.²⁷ It would also be interesting to study Lebanese Arabic in two contexts, reading and spontaneous speech, or according to the specific pathology underlying the dysphonia as the results may also differ.²⁹

CONCLUSION

The idea for this preliminary study emerged from a specific need to evaluate voice in the context of society as is the case for Lebanon.

The results we have gathered have allowed us to suspect that patients tend to appear more dysphonic when speaking the French language. However, a firm conclusion is not possible at this stage due to the study limitations and the small sample size.

The suggested corpus in Lebanese Arabic seems to be adapted for the evaluation of voice quality as well as other types of language and communication disorders such as stuttering.

This work raises various suggestions that could have important clinical implications in a speech therapy practice. In fact, it would be important to prove to what extent the voice reflects a cultural marker. It would also be useful for the voice therapist to understand and evaluate the patient’s perception of their own voice quality according to the languages used in their daily life. This would help the clinician set appropriate objectives in voice evaluation. It will also help him judge which language should he prioritizes in vocal therapy when it comes to a multilingual patient.

These suggestions provide starting points for designing additional research projects that address cultural and multilingual concerns that impact the delivery of care.

Funding: No funding sources

Conflict of interest: None declared

Ethical approval: The study was approved by the Institutional Ethics Committee

REFERENCES

1. 1. Giovanni A, Aumelas E. Anatomie et physiologie de la phonation. In: Giovanni A. Le bilan d’une dysphonie. Etat actuel et perspectives. Collection Voix Parole Langage, Solal; 2004: 7-44.
2. 2. Velsvik Bele I. Reliability in Perceptual Analysis of Voice Quality. J Voice. 2006;19:555-73.
3. 3. Ghio A, Cantarella G, Weisz F, Robert D, Woisard V, Fussi F, et al. Is the perception of dysphonia severity language – dependent? A comparison of French and Italian voice assessments. Logoped Phoniatr Vocol. 2015;40(1):36-43.
4. 4. Dejonckere PH, Bradley P, Clemente P, Cornut G, Crevier-Buchman L, Friedrich G. A basic protocol for functional assessment of voice pathology, especially for investigating the efficacy of (phonosurgical) treatments and evaluating new assessment techniques. Guideline elaborated by the Committee on Phoniatrics of the European Laryngological Society. Eur Arch. Otolaryngol. 2001; 258:77-82.
5. 5. Dejonckere PH, Remacle M, Fresnel-Elbaz E, Woisard V, Crevier-Buchman L, Millet B. Reliability and clinical relevance of perceptual evaluation on pathological voices. Revue de laryngologie otologie et rhinol. 1998;119(4):247-8.
6. 6. Karnell MP, Melton SD, Childes JM, Coleman TC, Dailey SA, Hoffman HT. Reliability of clinician-based (GRBAS and CAPE-V) and patient-based (V-RQOL and IPVI) documentation of voice disorders. J Voice. 2007;21(5):576-90.
7. 7. Revis J, Giovanni A, Wuyts F, Triglia J. Comparison of different voice samples for perceptual analysis. Folia Phoniatr Logop. 1998;51(3):108-16.
8. 8. Ghio A, Giovanni A, Teston B, Révis J, Yu P, Ouaknine M, Legou T. Bilan et perspectives de quinze ans d’évaluation vocale par méthodes instrumentales et perceptives. Journées d’Etude sur la Parole. 2008;309-12.
9. 9. Wuyts FL, De Bodt M, Van de Heyning P. Is the Reliability of a visual Analog Scale Higher Than An Ordinal Scale? An Experiment with the GRBAS Scale for the Perceptual Evaluation. J Voice. 1999;13:508-17.
10. 10. Benzaquen Y. Retrouver, comprendre et maîtriser sa voix en toutes circonstances. SOS voix. Frison – Roche, 2000.
11. 11. Meunier C, Frenck-Mestre C, Le Besnerais M, Lelokov-Boissard T. La perception des systèmes vocaliques étrangers: une étude inter-langues. Laboratoire Parole et Langage 2004; C.N.R.S. U.MR.6057.
12. 12. Benoist-Lucy A, Pillot-Loiseau C. Qualité et ressenti vocaux dans le cas du bilinguisme franco-arabe. Glossa. 2012;111:78–97.
13. 13. QPEB. Parental Questionnaire for Bilingual children. Project realized at the Institute of Speech and Language Pathology at St. Joseph University in Beirut with the collaboration of COST team 2009-2013.
14. 14. Desbiens D, Laurier M, Leroux J. Echelle Québécoise des niveaux de compétence en français des personnes immigrantes adultes. Bibliothèque et Archive nationales du Québec. 2011.
15. 15. Hammarberg G, Gauffin J. Perceptual and acoustic characteristic of quality differences in pathological voices as related to physiological aspects. In: Vocal Fold Physiology: voice Quality Control. 1995;19:283-303.
16. 16. Ghio A, Dufour S, Rouaze M, Bokanowski V, Pouchoulin G, Revis J, et al. Mise au point et évaluation d’un protocole d’apprentissage de jugement perceptif de la sévérité de dysphonies sur de la parole naturelle. Rev Laryngol Otol Rhinol. 2011;132:19 –27.
17. 17. Dibazar A, Narayanan S, Berger TW. Feature analysis for automatic detection of pathological speech. Biomedical Engineering Department and Electrical Engeneering Department. University of Southern California, 2002.
18. 18. Choueiri L, Aoun J, Benmamoun E. The Syntax of Arabic. Cambridge University Press, 2010.
19. 19. Touma N. Dot after dot (…) make a sea. Dar Ounbouz press; 2009: 2-32.
20. 20. Villaseňor-Pineda L, Montes-y-Gomez M, Vaufreydaz D, Serignat J-F. Experiments on the construction of a phonetically balanced corpus from the web. In: Gelbukh, A. (eds). Computational Linguistics and Intelligent Text Processing. CICLing. Lecture Notes in Computer Science. Springer; 2004.
21. 21. Révis J, Galant C, Fredouille C, Ghio A, Giovanni A. Influence du contexte phonétique sur la manifestation dysphonique: Apport des nouvelles méthodologies d’analyse de la voix pathologique. In. La voix dans tous ses maux, sous la direction de Peguy Gatignol. Ortho Edition – fno – UNADREO, 2009.
22. 22. Ormezzano Y. Le résultat acoustique final. In: Le guide de la voix. Odile Jacob. 2000:125-151.
23. 23. Christian S. L’analyse objective de la voix en libéral avec PRAAT. In. La voix dans tous ses maux, sous la direction de Peguy Gatignol. Ortho Edition – fno – UNADREO, 2009.
24. 24. Dupessey M, Coulombeau B. A l’écoute des voix pathologiques. Symétrie. 2005.
25. 25. Kreiman J, Gerratt BR, Precoda K, Berke G. Individual differences in voice quality perception, J Speech Hear Res. 1992;35(3):512–20.
26. 26. Yamaguchi H, Shrivastav R, Andrews ML, Niimi S. A comparison of voice quality ratings made by Japanese and American listeners using the GRBAS Scale. Folia phoniatrica et logopaedica. 2003;55(3):147-57.
27. 27. Al-Makarem AA, Petrosino L. Reading and spontaneous speaking fundamental frequency of young Arabic men for Arabic and English languages: a comparative study. Perceptual and motor skills. 2007;105:572-80.
28. 28. Pillot-Loiseau C, Benoist-Lucy A, Vaissière J. Fréquence fondamentale moyenne, qualité vocale et bilinguisme: quelles implications pour la rééducation vocale? In: “bilinguisme et culture: nouveaux défits”. Chapter 3. Ortho-Edition. 2012: 37-78.
29. 29. Kreiman J, Gerratt BR, Kempster GB, Erman A, Berke GS. Perceptual evaluation of voice quality: review, tutorial, and a framework for future fresearch. J Speech Lang Hearing Res. 1993;36(1):21-40.