|Year : 2015 | Volume
| Issue : 2 | Page : 201-204
Comparison of the diagnostic accuracy and validity of a short version of teen screen questionnaire-mental health (TSQ -M-short) for use in community
M. K. C. Nair1, Venkateswaran Rajaraman2, Deepa Chacko1, Sushila Russell2, Babu George1, Leena Sumaraj1, Paul Swamidhas Sudhakar Russell2
1 Child Development Centre, Government Medical College, Thiruvananthapuram, Kerala, India
2 Department of Psychiatry, Child and Adolescent Psychiatry Unit, Christian Medical College, Vellore, Tamil Nadu, India
|Date of Web Publication||22-Apr-2015|
Prof. Paul Swamidhas Sudhakar Russell
Department of Psychiatry, Professor of Psychiatry, Child and Adolescent Psychiatry Unit, Christian Medical College, Vellore - 632 002, Tamil Nadu
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Background: A few self-administered questionnaires are available for assessing mental health among adolescents in primary-care settings. Brief measures are desirable for use in big-data, epidemiological studies. Objectives: To evaluate a 7-item version, of the Teen Screen Questionnaire-Mental Health (TSQ-M), the TSQ-M-Short. Materials and Methods: In this prospective cross-sectional study of 140 adolescents, recruited from 6 rural or urban schools, the newly developed TSQ-M-Short as the measure for validation and General Health Questionnaire-12 item (GHQ-12) as the gold standard measure were administered by independent trained raters. Tests for diagnostic accuracy and validity were conducted. Results: A total TSQ-M-Short score of ≥ 6 had a sensitivity of 76%, specificity of 74%, positive likelihood ratio of 2.99, negative likelihood ratio of 0.33, positive predictive value of 6% and a negative predictive value of 82.1%. The area under curve (AUC) in the Receiver Operating Characteristic (ROC) for the TSQ-M-Short version was 0.84 (95% cumulative incidence (CI) = 0.76-0.89). The AUC for the TSQ-M-Short version was higher than the AUC for the original version, and the difference between the areas was 0.10 (95% CI = 0.02-0.19), which was statistically significant (z = 2.49; P = 0.01). The internal consistency of TSQ-M-Short, as measured by chronbach's α, was 0.34 (95% CI = 0.15-0.48). The construct validity demonstrated a 3-factor structure, which explained 55% of the variance. Conclusion: The TSQ-M-Short has an overall diagnostic accuracy which is better than the original TSQ-M. Although the original version includes symptoms for more mental health disorders, providing a wider screen. This short version will prove useful in big-data studies.
Keywords: Adolescents, diagnostic accuracy, mental-health, primary-care, questionnaire, validation
|How to cite this article:|
Nair M, Rajaraman V, Chacko D, Russell S, George B, Sumaraj L, Russell PS. Comparison of the diagnostic accuracy and validity of a short version of teen screen questionnaire-mental health (TSQ -M-short) for use in community. Indian J Psychol Med 2015;37:201-4
|How to cite this URL:|
Nair M, Rajaraman V, Chacko D, Russell S, George B, Sumaraj L, Russell PS. Comparison of the diagnostic accuracy and validity of a short version of teen screen questionnaire-mental health (TSQ -M-short) for use in community. Indian J Psychol Med [serial online] 2015 [cited 2020 Jan 21];37:201-4. Available from: http://www.ijpm.info/text.asp?2015/37/2/201/155621
| Introduction|| |
Despite the adolescent mental health problems being widely prevalent,  the diagnostic methods are quite disorganized in primary-care settings,  resulting in poor identification in the community.  Among different methods available to improve the diagnostic accuracy in mental health, using questionnaires are relatively inexpensive, can be used by health as well as non-health professionals,  and thus forms the central pillar for mental health research among adolescents in primary-care. However, brief questionnaires have been documented to reduce respondent burden,  improve the response rates  and eventually the quality of data collected.  Thus, short versions of existing measures are invaluable in conducting big-data studies in primary-care settings. Recently, we had validated the original 21-question version of Teen Screen Questionnaire-Mental Health.  In this study, we have developed and validated a self-administered, brief version (7-question) of the questionnaire named Teen Screen Questionnaire-Mental Health- Short (TSQ-M-Short), which identifies mental ill health over the previous 1-year for big-data epidemiological studies.
| Materials and Methods|| |
This study was conducted as part of the Adolescent Health District Plan (AHDP) Project with the support of National Rural Health Mission (NRHM) Kerala state. Data was collected from 3 schools from Thiruvananthapuram city and 3 schools from the rural taluks of Thiruvananthapuram district (Chirayinkeezhu, Nedumangad and Neyyattinkara). Schools from both these settings were randomly selected and students, of both genders, from 9 th to 12 th standard were included in the study if they gave verbal assent to their participation. Prior to the data collection, written permission to conduct the study was obtained from the District Educational Office and the project was approved by the Institutional Review Board of the Child Development Centre, Thiruvanthapuram. The confidentiality of the data was protected with reversible anonymization and by limiting the availability of the data to only the primary investigator and who did the statistical analysis.
Teens Symptoms Questionnaire-Mental Health-7 (TSQ-M-7) is a 7-item version of Teens Symptoms Questionnaire-Mental Health (TSQ-M).  The seven items, of TSQ-M-Short, had the highest factor endorsement TSQ-M or clinical relevance and were culled to form this short version of the measure. Like in the original version, the endorsement pattern for the TSQ-M short version was also a 3-point likert scale of 'Never', 'Sometimes' and 'Often' to minimize endorsement related error in big data studies. This abridged version (TSQ-M-short) was the index measure for studying the diagnostic accuracy and comparing with the original version in this study. General Health Questionnaire-12 item is a 12-item measure of current mental health among adults and adolescents. This gold standard, self-reported measure of psychological morbidity, detects psychiatric disorders in community settings and non-psychiatric settings. GHQ-12 cut-off of 2/3 (sensitivity = 87.4% and specificity = 79.2%) was used to define 'caseness' in this study as suggested for the Indian population. 
Data collection was done by qualified Clinical Child Developmental therapist trained in using the TSQ-M-Short and GHQ-12. The data was collected with face-to-face interviews with the adolescent by these trained auxiliary health professionals using TSQ-M-Short and GHQ-12, independently on the same day to minimize the rater bias as well as maximize the stability of the rating with time.
Sensitivity, specificity, likelihood ration and predictive values for various TSQ-M and TSQ-M-Short cut-off scores were calculated in order to determine the optimal screening threshold with Receiver Operating Characteristic (ROC) analyses, against the GHQ-12 cut-off of ≥2/3. We selected the optimal cut-off scores for screening that satisfied both sensitivity and specificity criteria (highest Youden index) for mental ill health using ROC analysis. To assess the diagnostic accuracy of each version of TSQ-M, we compared the Area Under the Curves (AUCs) of TSQ-M short version and TSQ-M original version by calculating the critical ratio 'z' (Hanley and McNeil, 1983). Multiple linear regression analysis with step-wise variable selection assessed the relative contributions of age, education and gender to TSQ-M short version and TSQ-M long version. The internal consistency of the measure was calculated using Chronbach's α and factor structure with Exploratory Factor analysis (principal component analysis with promax rotation). The items were excluded if they failed to load on any factor (loading <0.40) or had unacceptably high secondary loadings/cross loading (>0.40). All statistical analyses were performed using Statistical Package for the Social Sciences (SPSS) (version 19.0) and MedCalc (version 12.2.1).
| Results|| |
Among the 140 participants, 41% and 58% were boys and girls, respectively. The mean (SD) age of the teenagers was 15.60 (3.48) years. Most of the participants were from a nuclear family (66%) and others were from extended (20%) and joint families (14%). The TSQ score in the study population ranged from 5 to 16.
A total TSQ score of ≥6 had a sensitivity of 76%, specificity of 74%, positive likelihood ratio of 2.99, negative likelihood ratio of 0.33, positive predictive value of 6% and a negative predictive value of 82.1% making it appropriate for screening in the primary-care settings. The AUC in the ROC for the TSQ-M-Short version was 0.84 (95% CI = 0.76 to 0.89). The AUC for the TSQ-M-Original version was 0.73 (95% CI = 0.65 to 0.80). The AUC for the TSQ-M-Short version was higher than the AUC for the original version, and the difference between the areas was 0.10 (95% CI = 0.02-0.19), which was statistically significant (z = 2.49; P = 0.01) [Figure 1].
|Figure 1: The comparison of area under curve (AUC) for Teen Screen Questionnaire-Mental Health (TSQ-M) Short version and TSQ-M Original version against General Health Questionnaire-12 (GHQ-12) as the reference standard Difference between the AUCs = 0.10 (95% cumulative incidence (CI) = 0.02-0.19); z = 2.49; P = 0.01|
Click here to view
The internal consistency, as measured by chronbach's α, was 0.34 (95% CI = 0.15-0.48). For further investigation of the construct validity, we explored the factor structure of the items in the TSQ-M as the Kaiser-Meyer-Olkin measure of sampling adequacy was 0.53 and Barlett's test of sphericity was significant (P = 0.001). We extracted those factors with an eigen value of 1, and thus a 3-factor structure was derived explaining 55% of the variance [Table 1].
|Table 1: The factor structure of Teen Screen Questionnaire-Mental Health (TSQ-M-Short) version|
Click here to view
| Discussion|| |
This brief version of the scale (TSQ-M-Short) has an overall diagnostic accuracy (AUC = 0.84) that is statistically significantly better than the original version of the scale (AUC of 0.79) (Nair et al., 2014).  This improved diagnostic accuracy will enhance the identification of mental ill-health among adolescents in primary-care settings and in conducting big-data studies. Like the TSQ-M, Strength and Difficulties Questionnaire, Indian Adolescent Health Questionnaire and Patient Health Questionnaire the construct this brief version measures is mental ill health. However, the TSQ-M-Short encompasses a fewer number of Priority Mental Health Disorders of adolescence than its original version.
The diagnostic accuracy of TSQ-M-Short at the cut-off score of ≥6 has adequate sensitivity and specificity to be used as a screening instrument in the primary-care [Table 2]. The internal consistency of 0.34, which was lower then the original version of the measure and thus suggestive of multiple sub-constructs within the construct of mental ill-health as measured by TSQ-M-Short. This was theoretically anticipated because of inclusion of symptoms of mood and anxiety disorders, biological and impairment symptom clusters in the questionnaire. This was further proved by the 3-factor structure in the construct of mental ill-health by TSQ-M-Short. The strength of our study is that the validation methodology followed the guidelines as given COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) protocol  for validation and STAndards for the Reporting of Diagnostic accuracy studies (STARD) guidelines  for diagnostic accuracy respectively.
|Table 2: The diagnostic accuracy of Teen Screen Questionnaire-Mental Health (TSQ-M) short version|
Click here to view
| Conclusion|| |
In conclusion TSQ-M-Short is an easy to use measure with adequate psychometric properties for conducting big-data studies in the primary-care settings.
| Acknowledgment|| |
This study is supported by a grant from National Rural Health Mission (NRHM) Kerala state. We thank all the students and schools that participated in this study.
| References|| |
Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry 2005;62:593-602.
Kramer T, Garralda ME. Child and adolescent mental health problems in primary care. Adv Psychiatry Treat 2000;6:287-94.
Coyle JT, Pine DS, Charney DS, Lewis L, Nemeroff CB, Carlson GA, et al
. Depression and Bipolar Support Alliance Consensus Development Panel. Depression and bipolar support alliance consensus statement on the unmet needs in diagnosis and treatment of mood disorders in children and adolescents. J Am Acad Child Adolesc Psychiatry 2003;42:1494-503.
Committee on adolescent health care services and models of care for treatment, prevention, and healthy development, national research council and institute of medicine, board on children, youth, and families. In: Lawrence RS, Appleton Gootman J, SimL J, editors. Adolescent Health Services: Missing Opportunities. Washington, DC: The National Academies Press; 2009.
Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Public Health (Oxf) 2005;27:281-91.
McCluskey S, Topping AE. Increasing response rates to lifestyle surveys: A pragmatic evidence review. Perspect Public Health 2011;131:89-94.
Lund E, Gram IT. Response rate according to title and length of questionnaire. Scand J Soc Med 1998;26:154-60.
Nair M, Chacko D, Rajaraman V, George B, Samraj L, Russell PS. The diagnostic accuracy and validity of the teen screen questionnaire-mental health for clinical and epidemiological studies in primary-care settings. Indian J Psychol Med 2014;36:187-91.
Golderberg D, Williams P. A user's guide to the General Health questionnaire. Windsor: NFER-Nelson; 1988.
Kuruvilla A, Pothen M, Philip K, Braganza D, Joseph A, Jacob KS. The validation of the Tamil version of the 12 item general health questionnaire. Indian J Psychiatry 1999;41:217-21.
Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, et al
. Protocol of the COSMIN study: COnsensus-based Standards for the selection of health Measurement INstruments. BMC Med Res Methodol 2006;6:2.
Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al
. Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD initiative. Standards for Reporting of Diagnostic Accuracy. Clin Chem 2003;49:1-6.
[Table 1], [Table 2]