Indian Journal of Psychological Medicine
  Home | About Us | Editorial Board | Search | Ahead of print | Current Issue | Archives | Instructions | Contact | Advertise | Submission | Login 
Users Online: 875 
Wide layoutNarrow layoutFull screen layoutHome Print this page Email this page Small font sizeDefault font sizeIncrease font size


 
 Table of Contents    
ORIGINAL ARTICLE
Year : 2015  |  Volume : 37  |  Issue : 2  |  Page : 201-204  

Comparison of the diagnostic accuracy and validity of a short version of teen screen questionnaire-mental health (TSQ -M-short) for use in community


1 Child Development Centre, Government Medical College, Thiruvananthapuram, Kerala, India
2 Department of Psychiatry, Child and Adolescent Psychiatry Unit, Christian Medical College, Vellore, Tamil Nadu, India

Date of Web Publication22-Apr-2015

Correspondence Address:
Prof. Paul Swamidhas Sudhakar Russell
Department of Psychiatry, Professor of Psychiatry, Child and Adolescent Psychiatry Unit, Christian Medical College, Vellore - 632 002, Tamil Nadu
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/0253-7176.155621

Rights and Permissions
   Abstract 

Background: A few self-administered questionnaires are available for assessing mental health among adolescents in primary-care settings. Brief measures are desirable for use in big-data, epidemiological studies. Objectives: To evaluate a 7-item version, of the Teen Screen Questionnaire-Mental Health (TSQ-M), the TSQ-M-Short. Materials and Methods: In this prospective cross-sectional study of 140 adolescents, recruited from 6 rural or urban schools, the newly developed TSQ-M-Short as the measure for validation and General Health Questionnaire-12 item (GHQ-12) as the gold standard measure were administered by independent trained raters. Tests for diagnostic accuracy and validity were conducted. Results: A total TSQ-M-Short score of ≥ 6 had a sensitivity of 76%, specificity of 74%, positive likelihood ratio of 2.99, negative likelihood ratio of 0.33, positive predictive value of 6% and a negative predictive value of 82.1%. The area under curve (AUC) in the Receiver Operating Characteristic (ROC) for the TSQ-M-Short version was 0.84 (95% cumulative incidence (CI) = 0.76-0.89). The AUC for the TSQ-M-Short version was higher than the AUC for the original version, and the difference between the areas was 0.10 (95% CI = 0.02-0.19), which was statistically significant (z = 2.49; P = 0.01). The internal consistency of TSQ-M-Short, as measured by chronbach's α, was 0.34 (95% CI = 0.15-0.48). The construct validity demonstrated a 3-factor structure, which explained 55% of the variance. Conclusion: The TSQ-M-Short has an overall diagnostic accuracy which is better than the original TSQ-M. Although the original version includes symptoms for more mental health disorders, providing a wider screen. This short version will prove useful in big-data studies.

Keywords: Adolescents, diagnostic accuracy, mental-health, primary-care, questionnaire, validation


How to cite this article:
Nair M, Rajaraman V, Chacko D, Russell S, George B, Sumaraj L, Russell PS. Comparison of the diagnostic accuracy and validity of a short version of teen screen questionnaire-mental health (TSQ -M-short) for use in community. Indian J Psychol Med 2015;37:201-4

How to cite this URL:
Nair M, Rajaraman V, Chacko D, Russell S, George B, Sumaraj L, Russell PS. Comparison of the diagnostic accuracy and validity of a short version of teen screen questionnaire-mental health (TSQ -M-short) for use in community. Indian J Psychol Med [serial online] 2015 [cited 2019 Oct 22];37:201-4. Available from: http://www.ijpm.info/text.asp?2015/37/2/201/155621


   Introduction Top


Despite the adolescent mental health problems being widely prevalent, [1] the diagnostic methods are quite disorganized in primary-care settings, [2] resulting in poor identification in the community. [3] Among different methods available to improve the diagnostic accuracy in mental health, using questionnaires are relatively inexpensive, can be used by health as well as non-health professionals, [4] and thus forms the central pillar for mental health research among adolescents in primary-care. However, brief questionnaires have been documented to reduce respondent burden, [5] improve the response rates [6] and eventually the quality of data collected. [7] Thus, short versions of existing measures are invaluable in conducting big-data studies in primary-care settings. Recently, we had validated the original 21-question version of Teen Screen Questionnaire-Mental Health. [8] In this study, we have developed and validated a self-administered, brief version (7-question) of the questionnaire named Teen Screen Questionnaire-Mental Health- Short (TSQ-M-Short), which identifies mental ill health over the previous 1-year for big-data epidemiological studies.


   Materials and Methods Top


This study was conducted as part of the Adolescent Health District Plan (AHDP) Project with the support of National Rural Health Mission (NRHM) Kerala state. Data was collected from 3 schools from Thiruvananthapuram city and 3 schools from the rural taluks of Thiruvananthapuram district (Chirayinkeezhu, Nedumangad and Neyyattinkara). Schools from both these settings were randomly selected and students, of both genders, from 9 th to 12 th standard were included in the study if they gave verbal assent to their participation. Prior to the data collection, written permission to conduct the study was obtained from the District Educational Office and the project was approved by the Institutional Review Board of the Child Development Centre, Thiruvanthapuram. The confidentiality of the data was protected with reversible anonymization and by limiting the availability of the data to only the primary investigator and who did the statistical analysis.

Measures

Teens Symptoms Questionnaire-Mental Health-7 (TSQ-M-7) is a 7-item version of Teens Symptoms Questionnaire-Mental Health (TSQ-M). [8] The seven items, of TSQ-M-Short, had the highest factor endorsement TSQ-M or clinical relevance and were culled to form this short version of the measure. Like in the original version, the endorsement pattern for the TSQ-M short version was also a 3-point likert scale of 'Never', 'Sometimes' and 'Often' to minimize endorsement related error in big data studies. This abridged version (TSQ-M-short) was the index measure for studying the diagnostic accuracy and comparing with the original version in this study. General Health Questionnaire-12 item[9] is a 12-item measure of current mental health among adults and adolescents. This gold standard, self-reported measure of psychological morbidity, detects psychiatric disorders in community settings and non-psychiatric settings. GHQ-12 cut-off of 2/3 (sensitivity = 87.4% and specificity = 79.2%) was used to define 'caseness' in this study as suggested for the Indian population. [10]

Data collection was done by qualified Clinical Child Developmental therapist trained in using the TSQ-M-Short and GHQ-12. The data was collected with face-to-face interviews with the adolescent by these trained auxiliary health professionals using TSQ-M-Short and GHQ-12, independently on the same day to minimize the rater bias as well as maximize the stability of the rating with time.

Statistical analysis

Sensitivity, specificity, likelihood ration and predictive values for various TSQ-M and TSQ-M-Short cut-off scores were calculated in order to determine the optimal screening threshold with Receiver Operating Characteristic (ROC) analyses, against the GHQ-12 cut-off of ≥2/3. We selected the optimal cut-off scores for screening that satisfied both sensitivity and specificity criteria (highest Youden index) for mental ill health using ROC analysis. To assess the diagnostic accuracy of each version of TSQ-M, we compared the Area Under the Curves (AUCs) of TSQ-M short version and TSQ-M original version by calculating the critical ratio 'z' (Hanley and McNeil, 1983). Multiple linear regression analysis with step-wise variable selection assessed the relative contributions of age, education and gender to TSQ-M short version and TSQ-M long version. The internal consistency of the measure was calculated using Chronbach's α and factor structure with Exploratory Factor analysis (principal component analysis with promax rotation). The items were excluded if they failed to load on any factor (loading <0.40) or had unacceptably high secondary loadings/cross loading (>0.40). All statistical analyses were performed using Statistical Package for the Social Sciences (SPSS) (version 19.0) and MedCalc (version 12.2.1).


   Results Top


Among the 140 participants, 41% and 58% were boys and girls, respectively. The mean (SD) age of the teenagers was 15.60 (3.48) years. Most of the participants were from a nuclear family (66%) and others were from extended (20%) and joint families (14%). The TSQ score in the study population ranged from 5 to 16.

A total TSQ score of ≥6 had a sensitivity of 76%, specificity of 74%, positive likelihood ratio of 2.99, negative likelihood ratio of 0.33, positive predictive value of 6% and a negative predictive value of 82.1% making it appropriate for screening in the primary-care settings. The AUC in the ROC for the TSQ-M-Short version was 0.84 (95% CI = 0.76 to 0.89). The AUC for the TSQ-M-Original version was 0.73 (95% CI = 0.65 to 0.80). The AUC for the TSQ-M-Short version was higher than the AUC for the original version, and the difference between the areas was 0.10 (95% CI = 0.02-0.19), which was statistically significant (z = 2.49; P = 0.01) [Figure 1].
Figure 1: The comparison of area under curve (AUC) for Teen Screen Questionnaire-Mental Health (TSQ-M) Short version and TSQ-M Original version against General Health Questionnaire-12 (GHQ-12) as the reference standard Difference between the AUCs = 0.10 (95% cumulative incidence (CI) = 0.02-0.19); z = 2.49; P = 0.01

Click here to view


The internal consistency, as measured by chronbach's α, was 0.34 (95% CI = 0.15-0.48). For further investigation of the construct validity, we explored the factor structure of the items in the TSQ-M as the Kaiser-Meyer-Olkin measure of sampling adequacy was 0.53 and Barlett's test of sphericity was significant (P = 0.001). We extracted those factors with an eigen value of 1, and thus a 3-factor structure was derived explaining 55% of the variance [Table 1].
Table 1: The factor structure of Teen Screen Questionnaire-Mental Health (TSQ-M-Short) version

Click here to view



   Discussion Top


This brief version of the scale (TSQ-M-Short) has an overall diagnostic accuracy (AUC = 0.84) that is statistically significantly better than the original version of the scale (AUC of 0.79) (Nair et al., 2014). [8] This improved diagnostic accuracy will enhance the identification of mental ill-health among adolescents in primary-care settings and in conducting big-data studies. Like the TSQ-M, Strength and Difficulties Questionnaire, Indian Adolescent Health Questionnaire and Patient Health Questionnaire the construct this brief version measures is mental ill health. However, the TSQ-M-Short encompasses a fewer number of Priority Mental Health Disorders of adolescence than its original version.

The diagnostic accuracy of TSQ-M-Short at the cut-off score of ≥6 has adequate sensitivity and specificity to be used as a screening instrument in the primary-care [Table 2]. The internal consistency of 0.34, which was lower then the original version of the measure and thus suggestive of multiple sub-constructs within the construct of mental ill-health as measured by TSQ-M-Short. This was theoretically anticipated because of inclusion of symptoms of mood and anxiety disorders, biological and impairment symptom clusters in the questionnaire. This was further proved by the 3-factor structure in the construct of mental ill-health by TSQ-M-Short. The strength of our study is that the validation methodology followed the guidelines as given COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) protocol [11] for validation and STAndards for the Reporting of Diagnostic accuracy studies (STARD) guidelines [12] for diagnostic accuracy respectively.
Table 2: The diagnostic accuracy of Teen Screen Questionnaire-Mental Health (TSQ-M) short version

Click here to view



   Conclusion Top


In conclusion TSQ-M-Short is an easy to use measure with adequate psychometric properties for conducting big-data studies in the primary-care settings.


   Acknowledgment Top


This study is supported by a grant from National Rural Health Mission (NRHM) Kerala state. We thank all the students and schools that participated in this study.

 
   References Top

1.
Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry 2005;62:593-602.  Back to cited text no. 1
    
2.
Kramer T, Garralda ME. Child and adolescent mental health problems in primary care. Adv Psychiatry Treat 2000;6:287-94.  Back to cited text no. 2
    
3.
Coyle JT, Pine DS, Charney DS, Lewis L, Nemeroff CB, Carlson GA, et al. Depression and Bipolar Support Alliance Consensus Development Panel. Depression and bipolar support alliance consensus statement on the unmet needs in diagnosis and treatment of mood disorders in children and adolescents. J Am Acad Child Adolesc Psychiatry 2003;42:1494-503.  Back to cited text no. 3
[PUBMED]    
4.
Committee on adolescent health care services and models of care for treatment, prevention, and healthy development, national research council and institute of medicine, board on children, youth, and families. In: Lawrence RS, Appleton Gootman J, SimL J, editors. Adolescent Health Services: Missing Opportunities. Washington, DC: The National Academies Press; 2009.  Back to cited text no. 4
    
5.
Bowling A. Mode of questionnaire administration can have serious effects on data quality. J Public Health (Oxf) 2005;27:281-91.  Back to cited text no. 5
    
6.
McCluskey S, Topping AE. Increasing response rates to lifestyle surveys: A pragmatic evidence review. Perspect Public Health 2011;131:89-94.  Back to cited text no. 6
    
7.
Lund E, Gram IT. Response rate according to title and length of questionnaire. Scand J Soc Med 1998;26:154-60.  Back to cited text no. 7
    
8.
Nair M, Chacko D, Rajaraman V, George B, Samraj L, Russell PS. The diagnostic accuracy and validity of the teen screen questionnaire-mental health for clinical and epidemiological studies in primary-care settings. Indian J Psychol Med 2014;36:187-91.  Back to cited text no. 8
[PUBMED]  Medknow Journal  
9.
Golderberg D, Williams P. A user's guide to the General Health questionnaire. Windsor: NFER-Nelson; 1988.  Back to cited text no. 9
    
10.
Kuruvilla A, Pothen M, Philip K, Braganza D, Joseph A, Jacob KS. The validation of the Tamil version of the 12 item general health questionnaire. Indian J Psychiatry 1999;41:217-21.  Back to cited text no. 10
[PUBMED]  Medknow Journal  
11.
Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, et al. Protocol of the COSMIN study: COnsensus-based Standards for the selection of health Measurement INstruments. BMC Med Res Methodol 2006;6:2.  Back to cited text no. 11
    
12.
Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD initiative. Standards for Reporting of Diagnostic Accuracy. Clin Chem 2003;49:1-6.  Back to cited text no. 12
    


    Figures

  [Figure 1]
 
 
    Tables

  [Table 1], [Table 2]



 

Top
 
 
  Search
 
  
    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

 
  In this article
    Abstract
   Introduction
    Materials and Me...
   Results
   Discussion
   Conclusion
   Acknowledgment
    References
    Article Figures
    Article Tables

 Article Access Statistics
    Viewed1316    
    Printed15    
    Emailed0    
    PDF Downloaded64    
    Comments [Add]    

Recommend this journal