This article has Open Peer Review reports available.
Evaluation of chemiluminescence, toluidine blue and histopathology for detection of high risk oral precancerous lesions: A cross-sectional study
© Ujaoney et al; licensee BioMed Central Ltd. 2012
Received: 8 July 2011
Accepted: 12 March 2012
Published: 12 March 2012
Early detection holds the key to an effective control of cancers in general and of oral cancers in particular. However, screening procedures for oral cancer are not straightforward due to procedural requirements as well as feasibility issues, especially in resource-limited countries.
We conducted a cross-sectional study to compare the performance of chemiluminescence, toluidine blue and histopathology for detection of high-risk precancerous oral lesions. We evaluated 99 lesions from 55 patients who underwent chemiluminescence and toluidine blue tests along with biopsy and histopathological examination. We studied inter-as well as intra-rater agreement in the histopathological evaluation and then using latent class modeling, we estimated the operating characteristics of these tests in the absence of a reference standard test.
There was a weak inter-rater agreement (kappa < 0.15) as well as a weak intra-rater reproducibility (Pearson's r = 0.28, intra-class correlation rho = 0.03) in the histopathological evaluation of potentially high-risk precancerous lesions. When compared to histopathology, chemiluminescence and toluidine blue retention had a sensitivity of 1.00 and 0.59, respectively and a specificity of 0.01 and 0.79, respectively. However, latent class analysis indicated a low sensitivity (0.37) and high specificity (0.90) of histopathological evaluation. Toluidine blue had a near perfect high sensitivity and specificity for detection of high-risk lesions.
In our study, there was variability in the histopathological evaluation of oral precancerous lesions. Our results indicate that toluidine blue retention test may be better suited than chemiluminescence to detect high-risk oral precancerous lesions in a high-prevalence and low-resource setting like India.
Oral malignancies continue to burden the clinical and economic dimensions of health care around the world [1, 2]. In India, for example, oral cancers constitute 40% of all cancers and rank as the most common cancer in men and third most common cancer in women [3, 4]. The reason why oral cavity cancers occupy a strategic position in the health care systems is that an early detection of these lesions is theoretically possible and practically useful [5–8]. Such early detection is generally associated with a high expectation of prevention of deformity, relapse and mortality [3, 9].
Early detection of oral cavity carcinoma is, however, far from straightforward. Presence of precancerous lesions is not easy to detect due to a high likelihood of false-positivity. Histopathology continues to be used as the reference standard test . However the difficulties in detecting early lesions with confidence  combined with the possible interrater variations of histopathological evaluations  compound the diagnostic challenges. For this reason, light-based methods [9, 13, 14] that visually highlight lesions are becoming popular as an adjunct for detection of precancerous lesions. Despite the expected theoretical benefit of these tests, Mehrotra et al  recently reported that these measures may not add a meaningful value to the simple diagnostic protocol of a detailed visual examination in a high prevalence setting. It has been argued [15, 16] that the light-based methods are designed for screening rather than as a diagnostic aid in a tertiary care setting. However, in our experience and in conjunction with those reported by Mehrotra et al , these tests are currently used as diagnostic aids in tertiary care centers in India.
A possible explanation to the contested use of the light-based protocols for the diagnosis of precancerous lesions in high prevalence settings could be the variability in the histopathological evaluation. Current evaluation of the diagnostic/screening utility of these tests is contingent upon the assumption that histopathological evaluation is the reference standard. Arguably, however, if the histopathological evaluation is itself subject to errors then the estimates of the sensitivity and specificity of the light-based protocols can be expected to be biased. In this study, we considered the diagnostic performance of the light-based protocols without treating histopathological evaluation as a gold standard.
This study was conducted at the Oral Diagnosis, Medicine and Radiology Department of the Sharad Pawar Dental College, Sawangi, Maharashtra, India. Consecutive outpatients who visited the study center and who clinically presented with at least one precancerous lesion were recruited into this study. The exclusion criteria were: presence of frank malignancy (class I lesions based on Sciubba's  definition); known hypersensitivity to any ingredient or their analogues used during chemiluminescent light examination; any systemic disease that could obscure the true clinical presentation and interfere with or are contraindications to biopsy procedure; and any dental conditions such as orthodontic appliances or prostheses that may interfere with the examination.
We then conducted two diagnostic tests and documented the results using one of the following three diagnostic protocols: chemiluminescent illumination system (CHEM, obtained from Vizilite®, Zila, Inc. Fort Collins, CO), toluidine blue retention test (TBLU) and a combination of chemiluminescence and toluidine blue retention test (CHTB, obtained from Vizilite PLus®, Zila, Inc. Fort Collins, CO). For CHEM protocol, we used The Vizilite® light stick comprising an outer flexible capsule and a retractor (Figure 1B). Upon activation, the emanating light radiation (wavelength 430-580nm) was used to examine the oral cavity after dimming the room lights. The lesions that reflected the blue-white light were considered CHEM-positive. Any new lesion, not visible during conventional visual examination under incandescent light, but visible after chemiluminescent illumination test was noted and documented.
For TBLU protocol, the entire oral cavity was swabbed with 1% acetic acid solution and a pre-soaked swab of pharmaceutical grade toluidine blue was applied. Excess toluidine blue was removed using 1% acetic acid. Visual examination was then repeated under standard incandescent light to identify toluidine blue retention (Figure 1C) for each previously identified lesion and/or any new lesions subsequently found. Dark staining lesions were considered positive; faint lesions were considered equivocal; and those which did not take up the stain were considered negative. Using these categories, lesions were classified as TBLU-positive if it was observed to be positive and TBLU-negative if the result was either equivocal or negative. Finally, to classify using the CHTB protocol (Figure 1D), we considered a lesion to be CHTB-positive if it was both CHEM-positive and TBLU-positive; otherwise the lesion was considered to be CHTB-negative. Finally, incisional biopsy was performed on all lesions. All procedures were conducted during a single patient visit.
Biopsy specimens were collected in 10% formalin solution and processed. Histopathologic evaluation was done by two senior Oral Pathologists blinded to the clinical findings. The first pathologist evaluated each specimen at two time points. The average interval between the two evaluations was 3 months. For all evaluations, the histopathologists used Smith and Pindborg's  scoring system which was based on 13 histopathological features. The total score ranged from 0 to 75 and, based on this total score, the histopathological grading was given as follows: no dysplasia (score 0-10, Figure 1E), mild dysplasia (score 11-25, Figure 1F), moderate dysplasia (score 26-45, Figure 1G) and severe dysplasia (score > 45, Figure 1H). We further reduced these evaluations to a binary classification scheme as high risk/low risk in accordance with the criteria set by the World Health Organization (WHO) classification .
We studied the intra-and inter-rater agreement using Siegel and Castellan's fixed-marginal multi-rater kappa statistic, Bland-Altman plot and Pitman's variance ratio test for paired observations. The Siegel and Castellan's method of kappa estimation permits the estimation of per category kappa statistic (using the kap command in the Stata software package). To estimate the diagnostic performance of histopathological evaluations along with the three test protocols (CHEM, TBLU and CHTB) we did not make any a priori assumption about the reference standard. Such a representation of the data is amenable to latent class analysis (LCA) [19–22]. We used Hui and Walter's multinomial latent class model, the details of which are described elsewhere . Briefly, if there are n dichotomous diagnostic tests, then there exist 2n + 1 unknown parameters to be estimated (n sensitivities, n specificities and prevalence) from a total of 2n diagnostic combinations. The degrees of freedom for estimation of the parameters are, thus, 2n-1. Therefore this model can be used only if there are at least three tests (number of parameters to be estimated = 7 and degrees of freedom = 7). When the degrees of freedom exceed the number of parameters to be estimated the excess degrees of freedom can be used to test the goodness-of-fit of the latent class model. For latent class analyses, we used the latent1.exe program (Walter and Cook, personal communication). Other statistical analyses were conducted using the Stata 10.0 (Stata Corp, College Station, TX) statistical software package. Statistical significance was assessed at a type I error rate of 0.05.
Characteristics of the study subjects and samples
N† or Mean*
%† or SD*
Tobacco + lime
Number of samples
Location of lesion
Variability in reference standard evaluation
The majority of the specimens were rated as mild by both the histopathologists (Figure 2B, code 1). We examined the inter-evaluation agreement for each category of the classification. In general, the kappa statistic was low (< 0.15) for all categories. However, the kappa statistic reached statistical significance for the mild, moderate, or severe categories (codes 1, 2, 3, respectively; Figure 2B). The overall agreement among the three evaluations was also low but statistically significant (kappa = 0.1126, p = 0.005). Together, these findings indicated a substantial intra-and inter-rater variability in the histopathological evaluations of the study specimens.
Composite histopathological evaluation
Thus, we reasoned that the true histopathological evaluation for a given specimen would remain unknown. To use LCA, we needed to binarize the histopathological classification as shown in Figure 2C. Using this binarization scheme, we constructed eight combinatorial categories based on each histopathological evaluation (Figure 2D). The results of the LCA indicated that the estimated prevalence of the latent trait of a high-risk lesion was 20.8%. LCA predicted that the sensitivities of the three evaluations were 95.4%, 87.2%, and 71.4%, respectively, while the respective specificities were 50.4%, 63.1%, and 59.6%. Using these predictions, LCA estimated that the probability of a high-risk lesion was lowest when all the histopathological evaluations classified a specimen as a low-risk lesion, and highest when all the evaluations classified it as a high-risk lesion (bar graph in Figure 2D).
We then proceeded to evaluate the validity of a composite histopathological outcome. For this, we first generated the sum of codes ascribed to each specimen by all the three evaluations with the expectation that specimens with higher sums of scores (range 0-9) will have a higher likelihood of high-risk lesions. That indeed was the case (Figure 2E). One-way analysis of variance indicated that the total score explained 87.1% of the variability in the estimated probability of a high-risk lesion based on LCA. We then generated the majority vote from the three histopathological evaluations as follows: a lesion received as the histopathological majority vote (HPMV) the risk score seen in two or three evaluations. If all three evaluations yielded a different risk score for the same lesion then average risk score was taken as the HPMV. Using this composite measure, we observed that 17% specimens had a high-risk lesion (Figure 2F). This number corroborated the estimated prevalence of latent high-risk lesion trait using LCA.
Comparison of diagnostic performance
Diagnostic performance of the tests for high-risk lesions
Compared to HPMV
Prevalence of HRL
Discussion and Conclusions
We made three cardinal observations. First, for detection of precancerous lesions, there exists substantial intra-rater and inter-rater variation in the histopathological evaluation. Our results suggest that histopathology may be useful as a diagnostic test in demonstrably high degree of dysplasia or frank neoplasia but its value as a reference standard for diagnosis of low-risk precancerous lesions is questionable. Consequently, the use of histopathology as a reference standard against light-based assistance for diagnosis of high-risk lesions may lead to biased estimates of the diagnostic performance of these measures.
Second, we observed widely differing estimates of the sensitivity and specificity of the studied diagnostic protocols. However, caution needs to be exercised when reading and interpreting the results of latent class modeling [24–27]. A substantially different estimate of sensitivity (or specificity) for a test from that for the other tests can result from two scenarios: a) if the test is diagnostically inferior as compared to the rest; and b) if the test is using different criteria for classification of the disease state. In our case, the results do not necessarily imply that TBLU and CHTB are diagnostically superior to histopathology - rather it is possible that these tests use totally different criteria that do not compare with those used by histopathology. Nevertheless, our results clearly demonstrate (Table 2) that one of the main reasons for the controversial estimates of the diagnostic performance of light-based aids may be the classification method employed for the reference standard.
Third, a comparison of the diagnostic performance of TBLU and CHTB consistently indicated that use of CHEM may be somewhat redundant. From a primary health care perspective this finding is important since it will reduce the cost of diagnostic evaluation considerably by restricting the use of the more expensive component. Indeed the estimates of sensitivity and specificity of TBLU observed in this study are comparable with or better than those of other more expensive protocols like autofluorescence [28, 29], photodynamic diagnosis , and chemiluminescence . Our results are in agreement with the findings of Epstein et al which show that toluidine blue retention test holds promise as a screening tool for high-risk oral precancerous lesions since it can reduce a large number of unnecessary biopsies . Concurring with other studies [33, 34], our results encourage consideration of TBLU as a viable and feasible screening method in high-prevalence and low-resource scenarios like India.
There are important limitations of this study. First, as with the Mehrotra et al  study, our study recruited patients with a suspicion of a precancerous lesion for the reasons of feasibility as observed elsewhere . However, the protocol did preclude visually negative patients that could have been later detected by at least one of the diagnostic methods. Our estimates of high sensitivity may also partially reflect this spectrum bias thereby limiting a ready generalization of the results. Second, the study sample had an a priori high likelihood of a precancerous lesion. Therefore our study design does not permit a full evaluation of the screening performance of these tests but rather considers them in the more practical scenario of a tertiary care setting as a diagnostic aid.
In summary, our findings support those of Mehrotra et al  and demonstrate that improvements are needed for histopathological evaluation of precancerous lesions - especially, low risk lesions. Our findings also suggest that toluidine blue retention may be considered as a diagnostic strategy for oral cancers in countries like India. More robust and larger studies are required to assertively and definitively answer questions related to the screening use of these tools in high prevalence settings.
- Mignogna MD, Fedele S, Lo Russo L: The World Cancer Report and the burden of oral cancer. Eur J Cancer Prev. 2004, 13 (2): 139-142.View ArticlePubMedGoogle Scholar
- Petersen PE: Oral cancer prevention and control-the approach of the World Health Organization. Oral Oncol. 2009, 45 (4-5): 454-460. 10.1016/j.oraloncology.2008.05.023.View ArticlePubMedGoogle Scholar
- Mehrotra R, Singh M, Thomas S, Nair P, Pandya S, Nigam NS, Shukla P: A cross-sectional study evaluating chemiluminescence and autofluorescence in the detection of clinically innocuous precancerous and cancerous oral lesions. J Am Dent Assoc. 2010, 141 (2): 151-156.View ArticlePubMedGoogle Scholar
- Yeole BB, Sankaranarayanan R, Sunny MSL, Swaminathan R, Parkin DM: Survival from head and neck cancer in Mumbai (Bombay), India. Cancer. 2000, 89 (2): 437-444. 10.1002/1097-0142(20000715)89:2<437::AID-CNCR32>3.0.CO;2-R.View ArticlePubMedGoogle Scholar
- Sankaranarayanan R: Screening for cervical and oral cancers in India is feasible and effective. Natl Med J India. 2005, 18 (6): 281-284.PubMedGoogle Scholar
- Sankaranarayanan R, Boffetta P: Research on cancer prevention, detection and management in low- and medium-income countries. Ann Oncol. 2010, 21 (10): 1935-1943. 10.1093/annonc/mdq049.View ArticlePubMedGoogle Scholar
- Sankaranarayanan R, Dinshaw K, Nene BM, Ramadas K, Esmy PO, Jayant K, Somanathan T, Shastri S: Cervical and oral cancer screening in India. J Med Screen. 2006, 13 (Suppl 1): S35-S38.PubMedGoogle Scholar
- Sankaranarayanan R, Mathew B, Jacob BJ, Thomas G, Somanathan T, Pisani P, Pandey M, Ramadas K, Najeeb K, Abraham E: Early findings from a community-based, cluster-randomized, controlled oral cancer screening trial in Kerala, India. The Trivandrum Oral Cancer Screening Study Group. Cancer. 2000, 88 (3): 664-673. 10.1002/(SICI)1097-0142(20000201)88:3<664::AID-CNCR25>3.0.CO;2-V.View ArticlePubMedGoogle Scholar
- Trullenque-Eriksson A, Munoz-Corcuera M, Campo-Trapero J, Cano-Sanchez J, Bascones-Martinez A: Analysis of new diagnostic methods in suspicious lesions of the oral mucosa. Med Oral Patol Oral Cir Bucal. 2009, 14 (5): E210-E216.PubMedGoogle Scholar
- Patton LL, Epstein JB, Kerr AR: Adjunctive techniques for oral cancer examination and lesion diagnosis: a systematic review of the literature. J Am Dent Assoc. 2008, 139 (7): 896-905. quiz 993-894View ArticlePubMedGoogle Scholar
- Sciubba JJ: Improving detection of precancerous and cancerous oral lesions. Computer-assisted analysis of the oral brush biopsy. U.S. Collaborative OralCDx Study Group. J Am Dent Assoc. 1999, 130 (10): 1445-1457.View ArticlePubMedGoogle Scholar
- Brandwein-Gensler M, Smith RV, Wang B, Penner C, Theilken A, Broughel D, Schiff B, Owen RP, Smith J, Sarta C, et al: Validation of the histologic risk model in a new cohort of patients with head and neck squamous cell carcinoma. Am J Surg Pathol. 2010, 34 (5): 676-688.PubMedGoogle Scholar
- Kerr AR, Sirois DA, Epstein JB: Clinical evaluation of chemiluminescent lighting: an adjunct for oral mucosal examinations. J Clin Dent. 2006, 17 (3): 59-63.PubMedGoogle Scholar
- Ram S, Siar CH: Chemiluminescence as a diagnostic aid in the detection of oral cancer and potentially malignant epithelial lesions. Int J Oral Maxillofac Surg. 2005, 34 (5): 521-527. 10.1016/j.ijom.2004.10.008.View ArticlePubMedGoogle Scholar
- Huff KD: More about cancer detection. J Am Dent Assoc. 2010, 141 (6): 626-628. author reply 628, 630View ArticlePubMedGoogle Scholar
- Truelove EL: Detecting oral cancer. J Am Dent Assoc. 2010, 141 (6): 626-author reply 628, 630View ArticlePubMedGoogle Scholar
- Oliver RJ, MacDonald DG, Felix DH: Aspects of cell proliferation in oral epithelial dysplastic lesions. J Oral Pathol Med. 2000, 29 (2): 49-55. 10.1034/j.1600-0714.2000.290201.x.View ArticlePubMedGoogle Scholar
- Warnakulasuriya S, Reibel J, Bouquot J, Dabelsteen E: Oral epithelial dysplasia classification systems: predictive value, utility, weaknesses and scope for improvement. J Oral Pathol Med. 2008, 37 (3): 127-133. 10.1111/j.1600-0714.2007.00584.x.View ArticlePubMedGoogle Scholar
- Dendukuri N, Hadgu A, Wang L: Modeling conditional dependence between diagnostic tests: a multiple latent variable model. Stat Med. 2009, 28 (3): 441-461. 10.1002/sim.3470.View ArticlePubMedGoogle Scholar
- Ihorst G, Forster J, Petersen G, Werchau H, Rohwedder A, Schumacher M: The use of imperfect diagnostic tests had an impact on prevalence estimation. J Clin Epidemiol. 2007, 60 (9): 902-910. 10.1016/j.jclinepi.2006.11.016.View ArticlePubMedGoogle Scholar
- Koukounari A, Webster JP, Donnelly CA, Bray BC, Naples J, Bosompem K, Shiff C: Sensitivities and specificities of diagnostic tests and infection prevalence of Schistosoma haematobium estimated from data on adults in villages northwest of Accra, Ghana. Am J Trop Med Hyg. 2009, 80 (3): 435-441.PubMedPubMed CentralGoogle Scholar
- Yang I, Becker MP: Latent variable modeling of diagnostic accuracy. Biometrics. 1997, 53 (3): 948-958. 10.2307/2533555.View ArticlePubMedGoogle Scholar
- Bertrand P, Benichou J, Grenier P, Chastang C: Hui and Walter's latent-class reference-free approach may be more useful in assessing agreement than diagnostic performance. J Clin Epidemiol. 2005, 58 (7): 688-700. 10.1016/j.jclinepi.2004.10.021.View ArticlePubMedGoogle Scholar
- Albert PS, Dodd LE: On Estimating Diagnostic Accuracy From Studies With Multiple Raters and Partial Gold Standard Evaluation. J Am Stat Assoc. 2008, 103 (481): 61-73. 10.1198/016214507000000329.View ArticlePubMedPubMed CentralGoogle Scholar
- Baughman AL, Bisgard KM, Cortese MM, Thompson WW, Sanden GN, Strebel PM: Utility of composite reference standards and latent class analysis in evaluating the clinical accuracy of diagnostic tests for pertussis. Clin Vaccine Immunol. 2008, 15 (1): 106-114. 10.1128/CVI.00223-07.View ArticlePubMedGoogle Scholar
- Chu H, Zhou Y, Cole SR, Ibrahim JG: On the estimation of disease prevalence by latent class models for screening studies using two screening tests with categorical disease status verified in test positives only. Stat Med. 2010, 29 (11): 1206-1218.PubMedPubMed CentralGoogle Scholar
- Toft N, Jorgensen E, Hojsgaard S: Diagnosing diagnostic tests: evaluating the assumptions underlying the estimation of sensitivity and specificity in the absence of a gold standard. Prev Vet Med. 2005, 68 (1): 19-33. 10.1016/j.prevetmed.2005.01.006.View ArticlePubMedGoogle Scholar
- Rana M, Zapf A, Kuehle M, Gellrich NC, Eckardt AM: Clinical evaluation of an autofluorescence diagnostic device for oral cancer detection: a prospective randomized diagnostic study. Eur J Cancer Prev. 2012, doi: 10.1097/CEJ.0b013e32834fdb6dGoogle Scholar
- Awan KH, Morgan PR, Warnakulasuriya S: Evaluation of an autofluorescence based imaging system (VELscope) in the detection of oral potentially malignant disorders and benign keratoses. Oral Oncol. 2011, 47 (4): 274-277. 10.1016/j.oraloncology.2011.02.001.View ArticlePubMedGoogle Scholar
- Driemel O, Kunkel M, Hullmann M, von Eggeling F, Muller-Richter U, Kosmehl H, Reichert TE: Diagnosis of oral squamous cell carcinoma and its precursor lesions. Journal der Deutschen Dermatologischen Gesellschaft = Journal of the German Society of Dermatology: JDDG. 2007, 5 (12): 1095-1100. 10.1111/j.1610-0387.2007.06397.x.View ArticlePubMedGoogle Scholar
- Seoane Leston J, Diz Dios P: Diagnostic clinical aids in oral cancer. Oral Oncol. 2010, 46 (6): 418-422. 10.1016/j.oraloncology.2010.03.006.View ArticlePubMedGoogle Scholar
- Epstein JB, Silverman S, Epstein JD, Lonky SA, Bride MA: Analysis of oral lesion biopsies identified and evaluated by visual examination, chemiluminescence and toluidine blue. Oral Oncol. 2008, 44 (6): 538-544. 10.1016/j.oraloncology.2007.08.011.View ArticlePubMedGoogle Scholar
- Guneri P, Epstein JB, Kaya A, Veral A, Kazandi A, Boyacioglu H: The utility of toluidine blue staining and brush cytology as adjuncts in clinical examination of suspicious oral mucosal lesions. Int J Oral Maxillofac Surg. 2011, 40 (2): 155-161. 10.1016/j.ijom.2010.10.022.View ArticlePubMedGoogle Scholar
- Epstein JB, Guneri P: The adjunctive role of toluidine blue in detection of oral premalignant and malignant lesions. Current opinion in otolaryngology & head and neck surgery. 2009, 17 (2): 79-87. 10.1097/MOO.0b013e32832771da.View ArticleGoogle Scholar
- Patton LL: The effectiveness of community-based visual screening and utility of adjunctive diagnostic aids in the early detection of oral cancer. Oral Oncol. 2003, 39 (7): 708-723. 10.1016/S1368-8375(03)00083-6.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6890/12/6/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.