Comparison of factor structure and psychometric properties of original and abbreviated version of the Penn State Worry Questionnaire in a nonclinical sample:a cross-sectional psychometric study

Alireza Rashtbari,Hossein Malekizadeh,Omid Saed,2,＊

1 Department of Clinical Psychology,School of Medicine,Zanjan University of Medical Sciences,Zanjan,Iran

2 Social Determinants of Health Research Center,Zanjan University of Medical Sciences,Zanjan,Iran

Abstract Background and objectives:Worry as a main symptom of generalized anxiety disorder is a chain of repetitive and uncontrollable thoughts about possible negative events in the future.The Penn State Worry Questionnaire is one of the most widely used measures for assessing pathological worry.The purpose of the present study was to investigate the psychometric properties of the abbreviated version of the Penn State Worry Questionnaire-Abbreviated(PSWQ-A)and to compare it with the original version of the Penn State Worry Questionnaire(PSWQ)in a nonclinical sample.Participants and methods:The present study is a cross-sectional psychometric study.The statistical population of this study consisted of all students studying at Zanjan University of Medical Sciences(n=3500)from January to October 2018.A sample of 350 people was selected for the study.Research measures were the PSWQ-A,PSWQ,and the Generalized Anxiety Disorder 7-item(GAD-7)Scale.The present study was carried out after approval of Social Determinants of Health Research Center of Zanjan University of Medical Science with the project code of A-12-924-5 on October 7,2017.The project was also approved by the Ethics Committee of Zanjan University of Medical Sciences on October 17,2017 and the approval ID was IR.ZUMS.REC.1396.187.Results:Exploratory factor analysis,scree plot,and parallel analysis supported the single factor structure PSWQ-A.The total variance explained by the single-factor model of PSWQ-A was higher(53.1% versus 49.1%).Generally,fit indices for the PSWQ-A was better fitted than the PSWQ.Both measures had acceptable convergent validity(r=0.52 for both questionnaires)and satisfactory internal consistency(α=0.87 for both questionnaires).Conclusion:PSWQ-A has better psychometric properties compared to PSWQ,and it can be used for faster and more accurate assessment of worry in psychological studies and therapeutic settings.

Key words:abbreviated version of Penn State Worry Questionnaire;confirmatory factor analysis;cross-sectional study;factor structure;nonclinical sample;original version of the Penn State Worry Questionnaire;psychometric properties;worry

INTRODUCTION

Generalized anxiety disorder(GAD)is mainly characterized by the main symptom of chronic and uncontrollable worry about several issues at most days and for at least 6 months.Related physiological symptoms include fatigue,muscle tension,restlessness,irritability,and difficulty in concentrating that cause discomfort and dysfunction(American Psychiatric Association,2013).Approximately 25% of patients referring to clinics for anxiety disorders are diagnosed with GAD.The female-to-male ratio in GAD is about two to one,and the prevalence of this disorder is reported to be around 5%(Sadock and Sadock,2011;Bandelow et al.,2013).Numerous studies have shown that GAD is as debilitating as major depressive disorder(Kessler et al.,2001;Sanderson and Andrews,2002).Comorbid disorders are found in approximately 60% of patients with GAD,which is associated with more significant impairment in function,as well as lower remission rates and high rates of recurrence(Bruce et al.,2005;Newman et al.,2010).GAD is a risk factor for cardiovascular disease and premature mortality(Barger and Sydeman,2005;Todaro et al.,2007;Martens et al.,2010).Therefore,the treatment of GAD,as well as the development of precise measures for studying worry and monitoring the treatment process,is crucial.

Worry has been defined as a chain of repetitive and uncontrollable thoughts about possible negative events in the future(Roemer and Borkovec,1993).Worry is a transdiagnostic construct and the underlying mechanism in anxiety disorders and depression(McEvoy et al.,2013;Drost et al.,2014).Theoretical models of worry and anxiety disorders suggest that worry suppresses feared mental images(Borkovec et al.,2004),avoiding aversive emotional experiences(Mennin,2004),and avoiding negative emotional shifts(Newman and Llera,2011;Rashtbari et al.,2019).

Given the contribution of worry in the psychopathology of GAD as well as in other psychological disorders,the necessity of designing measures with desirable psychometric properties applicable to different cultures is increasingly felt.The Penn State Worry Questionnaire(PSWQ)is one of the most commonly used measures for assessing worry.The PSWQ measures extreme,excessive,uncontrollable,and pathological worry without focusing on specific areas of worry.The questionnaire was designed by Meyer et al.(1990).PSWQ has high internal consistency and test-retest reliability and acceptable convergent and divergent validity in general(Meyer et al.,1990),clinical(Brown et al.,1992;Webb et al.,2008)and student populations(Hazlett-Stevens et al.,2004).PSWQ is widely used in studies of worry and GAD.For example,PSWQ has been used to evaluate the efficacy of psychotherapy for GAD(Fisher,2006)as well as it has been used in etiological studies of worry and GAD(Via et al.,2018).

PSWQ consists of 16 items,of which 11 items are scored directly,and 5 items are scored as inverse.Initial studies on general populations(Meyer et al.,1990)and the clinical group(Brown et al.,1992),supported the single-factor structure of the questionnaire.However,subsequent studies have shown that the two-factor structure of this questionnaire fits better than the one-factor structure(Beck et al.,1995;Fresco et al.,2002;Olatunji et al.,2007).The first factor(general worry)contains 11 items with direct scoring,and the second factor(absence of worry)contains five items with inverse scoring.

However,the study by Olatunji et al.(2007)showed that the second factor(i.e.,items with inverse scoring)has no practical application despite its relation to psychological variables.Their study showed that 11 items with direct scoring had a stronger relationship with other psychological variables such as trait anxiety and negative affect than the absence of worry items.These findings suggest that the general worry factor is more useful in understanding the relationship between worry and other psychological constructs.Hopko et al.(2003)based on two modification criteria including significant factor loadings of all 16 PSWQ items on the single-factor model and standardized residual values above 2 in Lagrange analysis(indicating the significant measuring error of the item),designed a short version of PSWQ entitled Penn State Worry Questionnaire-Abbreviated(PSWQ-A).The researchers eliminated five reverse-scored items based on the modification criteria since they had a negative effect on the fitness of the one-factor model.Besides,although all 11 items had significant factor loadings with direct scoring,three of these items(14,15,and 16)had high standardized residual values and so excluded from the PSWQ single-factor model.Finally,an 8-item questionnaire loaded on one factor was designed.In the study by Hopko et al.(2003),the correlation between the total score of PSWQ-A and PSWQ was very high.The results also showed that PSWQ-A had good internal consistency.PSWQ-A has acceptable convergent and divergent validity with anxiety and depression questionnaires,respectively.The results of the study by Crittendon and Hopko(2006)also indicate the favorable psychometric properties of this measure in the European-American sample.A study on a clinical sample showed that PSWQ-A had high internal consistency and good convergent validity with anxiety questionnaires.DeLapp et al.(2016)examined the psychometric properties of this measure in a population of African-American and European-American students.Again,the results showed that PSWQ-A had good internal consistency in European-American and African-American students.The PSWQ-A also had good convergence validity with other anxiety measures,but the divergent validity of the measure was not acceptable.Taken together,these findings indicate that PSWQ-A has acceptable psychometric properties and can be used as a short worry assessment measure that reduces time wasting when completing a questionnaire in psychological studies and,also,in therapeutic settings.However,so far,no studies have investigated the psychometric properties of PSWQ-A in Iran.Therefore,the present study aimed to investigate the psychometric properties of PSWQ-A and compare it with the psychometric properties of PSWQ in the Iranian student population.

PARTICIPANTS AND METHODS

Ethics

The present study was carried out with approval from Social Determinants of Health Research Center of Zanjan University of Medical Science with the project code of A-12-924-5 on October 7,2017.The project also approved by the Ethics Committee of Zanjan University of Medical Sciences on October 17,2017 and the approval ID was IR.ZUMS.REC.1396.187 and performed in accordance of the.This study was reported in line with the STrengthening the Reporting of OBservational studies in Epidemiology(STROBE)statement(Additional file 1).The participants provided their written informed consent to take part in this study.

Participants

The statistical population of this cross-sectional study was all students of Zanjan University of Medical Sciences,Iran(=3500)who were studying at this university from January to October 2018.The standard criterion of minimum sample size for factor analysis is 5 or 10 times higher than measure items(Jung and Lee,2011).Accordingly,for this study,the minimum sample size should be 160.On the other hand,for confirmatory factor analysis,a sample size of 300 is described as “good” and 500 as “very good”(Weston and Gore Jr,2006).Thus,the sample size was set at 350 to reduce sampling error.The study questionnaires were implemented on 350 university students.Of these questionnaires,11 questionnaires(3.2%)were excluded due to problems such as duplicate responses and unanswered items.In the end,the data of 339 questionnaires(96.8%)were used for statistical analysis.

Procedure

First,the English version of 16-item PSWQ was translated into Farsi by an assistant professor of clinical psychology.Then,to ensure the accuracy of the translation,the Farsi version of PSWQ was translated into English by a specialist in English.At this point,the grammatical and editorial issues were resolved.In the next step,a pilot study was conducted on 81 students of Zanjan University of Medical Sciences(40 men,19-33 years old,mean age=23.5,SD=3.2,and 41 females,21-29 years old,mean age=23.7,SD=2.3),in order to examine the face validity and clarity of the items.Of these,15(18.5%)were undergraduate students,17(21%)were graduate students,40(49.4%)were Doctor of medicine students,and 9(11.1%)did not respond.Then,the conceptual defects of several items were corrected.Finally,a Farsi version of the PSWQ was completed after several revisions and correction steps.In the next step,the PSWQ and other questionnaires selected for this study were performed on 350 students of Zanjan University of Medical Sciences.

Outcome measures

Demographic data on age,gender,and educational level were collected using a researcher-made questionnaire.

PSWQ is a 16-item self-report questionnaire used to screen for GAD(Meyer et al.,1990;Davey and Wells,2006)and to measure severe,excessive,and uncontrollable worry.The PSWQ is scored on a 5-point Likert scale ranging from 1(not at all typical of me)to 5(very typical of me).The questionnaire consists of two factors.The first factor is the general worry factor with 11 items,and the second factor is the absence of worry,which contains 5 items.Internal consistency coefficients in the general population and students ranged from 0.90 to 0.91(Brown et al.,1992),and test-retest coefficients between clinical groups and students were reported as 0.74 to 0.92(Meyer et al.,1990;St?ber,1998).Internal consistency and test-retest coefficients(with 1-month interval)of this questionnaire were 0.79 and 0.88 in Iranian college students,respectively,and it had good validity(Dehshiri et al.,2009).

This is an abbreviated version of the PSWQ,consisting of eight items of the original questionnaire.The PSWQ-A is highly correlated with the original version.The questionnaire has also been shown to have good convergence and divergence validity in an older adult sample.However,test-retest reliability for this measure was moderate(=0.63,＜0.001;Hopko et al.,2003).Similar to the study by Hopko et al.(2003),in this study,instead of separately performing PSWQ and PSWQ-A,data from the full version of PSWQ was used to derive scores of both questionnaires.

This questionnaire is a short scale to diagnose cases of GAD and to measure the severity of clinical symptoms.The 7-item questionnaire has an additional question that measures impairment in individual,social,family,and occupational functioning.The questionnaire is scored on a 4-point Likert scale from 0(not at all)to 3(nearly every day).In the original version,the Cronbach’s alpha coefficient was 0.92,and with 2-weekinterval test-retest coefficient was 0.83(Spitzer et al.,2006).GAD-7 also has acceptable convergent validity(Spitzer et al.,2006).In an Iranian sample,the internal consistency coefficient of the questionnaire was calculated to be=0.85(Naeinian et al.,2011).The study also found that GAD-7 had satisfactory convergent validity(Naeinian et al.,2011).

Statistical analysis

Statistical analyses were performed using SPSS 25.0 and AMOS 24 softwares(IBM Corporation,Armonk,NY,USA).The Kaiser-Meyer-Olkin(KMO)test and Bartlett’s test of sphericity were used to evaluate the adequacy of the sampling.Statistical values of KMO＜0.70 are within an acceptable range.The Bartlett’s test of sphericity test must also be significant for exploratory factor analysis to be performed.The significance of Bartlett’s test of sphericity indicates that there is sufficient correlation in the item matrix.Exploratory factor analysis was performed on the data of 339 participants using Principal Component Analysis and Promax rotation with kappa coefficient 4.In confirmatory factor analysis,the following indices were used to evaluate the fitness of the model:(1)Chi-square to degree of freedom ratio(/df)- values＜3 indicate good model fit(Carmines and McIver,1981);(2)goodness of fit index(GFI);comparative fit index(CFI);(3)Tucker-Lewis index(TLI)- values＞0.90 for GFI,CFI,and TLI indices indicate good model fit(Hu and Bentler,1999);(4)Root Mean Square Error of Approximation(RMSEA)(5)Standardized Root Mean Square Residual(SRMR).Moreover,SRMR and RMSEA indices＜0.05 indicate good fit,and values between 0.05 to 0.08 indicate acceptable fit(Browne and Cudeck,1992).The internal consistency method was used to test the reliability of the questionnaires.Convergent validity of PSWQ-A was assessed using GAD-7 and PSWQ.

RESULTS

Demographic findings of the participants

Because 11 questionnaires(3.2%)were excluded due to problems such as duplicate responses and unanswered items,in the end,the data of 339 questionnaires(96.8%)were used for statistical analysis.Of the 339 data collected from Zanjan University of Medical Sciences,68.8%(227 students)were female.54.2%(184 students)were undergraduate students,11.8%(40 students)were graduate students,and 33.9%(115 students)were MD students.Furthermore,67.8%(230 students)of the study participants were in the age range of 18-21 years,20.0%(68 students)in the age range of 22-25 years,5.6%(19 students)were in the age range of 26 years and more,and 6.4%(22 students)did not answer this question.

Exploratory factor analysis results

The results showed that KMO statistics were within the acceptable range and were 0.91 and 0.89 for PSWQ and PSWQ-A,respectively.Bartlett’s test of sphericity was also significant(for PSWQ and PSWQ-A,=5360.693,＜0.001 and=1111.214,＜0.001,respectively).Exploratory factor analysis was then performed on the data of 339 participants.Initial exploratory factor analysis without determination of factors number revealed three factors for PSWQ and one factor for PSWQ-A with an eigenvalue higher than 1.Studies have suggested that in exploratory factor analysis,factors with an eigenvalue above,one should be retained.In addition,parallel analysis is referred to as the gold standard in determining the number of factors(Horn,1965;Hayton et al.,2004).For this reason,we used Monte Carlo software(Watkins,2000)for parallel analysis.A study in Iran has determined the twofactor structure of the PSWQ(Dehshiri et al.,2009).On the other hand,previous studies on the PSWQ-A also support its single-factor structure(Hopko et al.,2003;DeLapp et al.,2016).Finally,based on the results of the parallel analysis,the scree plot,as well as the results of previous studies,we conducted exploratory factor analysis for PSWQ and PSWQA,restricted to 2 and 1 factors,respectively.Figures 1 and 2 show the scree plot of these two questionnaires.In the PSWQ,factor 1 with an eigenvalue of 6.167 and factor 2 with a value of 1.689 respectively explained 38.60 and 10.55 percent of the total variance.The PSWQ-A is a one-factor questionnaire with an eigenvalue of 4.225,and explains 53.13% of the total variance.Factor loadings,the eigenvalues of each factor,communalities of each item,and the variance explained by the factors of the two questionnaires are presented in Table 1.

Confirmatory factor analysis results

Figure 1:Scree plot for Penn State Worry Questionnaire-Abbreviated.

Figure 2:Scree plot for Penn State Worry Questionnaire.

Table 1:Results of exploratory factor analysis

For the purpose of confirmatory factor analysis of the PSWQ and PSWQ-A,the item skewness and kurtosis were examined first.The skewness values above 0.2 and the kurtosis values above 0.7 indicate abnormal data distribution(Brown et al.,1992).The data were analyzed for these two criteria,and it was found that the data have a normal distribution.For this reason,the maximum likelihood estimation method - assuming that the distribution of data is normal - was used to test the goodness of fit of the models.At this stage,the default PSWQ model was tested with two factors.Fit indices for the model were at the desired level(RMSEA=0.06,GFI=0.90,CFI=0.92,TLI=0.91,SRMR=0.05)and this was determined as the final model for the PSWQ model.Figure 3 shows the confirmatory factor analysis of this questionnaire.Confirmatory factor analysis was performed for PSWQ-A.Default model fit indices show that this model does not have a good fit(RMSEA=0.10,GFI=0.92,GFI=0.92,CFI=0.89,SRMR=0.05).The uncorrelated items errors result in a more realistic representation of the data(Bentler and Chou,1987;Hu and Bentler,1999).Therefore,fit indices can be improved by modification indices in AMOS output.Modification indices show that by releasing covariance error between items 12 and 13,the model fit will be significantly improved.The results showed that goodness of fit indices of the modified model was improved and are in a good range(RMSEA=0.07,GFI=0.95,CFI=0.96,TLI=0.95,SRMR=0.03).For facilitating the comparison of the goodness of fit indices,the results of the confirmatory factor analysis of PSWQ are reported in Table 2.In the end,this model was selected as the final model for the PSWQ-A.The graph of the PSWQ-A confirmatory factor analysis is shown in Figure 4.

Figure 3:Standardized two-factor structure of the PSWQ-A model.

Figure 4:Standardized one-factor structure of the Penn State Worry Questionnaire-Abbreviated model.

Validity of PSWQ and PSWQ-A

These two questionnaires are designed to measure pathological worry.Since worry is the main symptom of GAD,therefore,GAD-7 was used to evaluate the convergent validity of PSWQ-A and PSWQ.The results showed that PSWQ and PSWQ-A have a positive and strong correlation with each other(=0.932,＜0.01).In addition,there was a significant positive correlation between the PSWQ and PSWQ-A questionnaires and the GAD-7 questionnaire=0.521,0.520,＜0.001,respectively).Overall,the results showed that the PSWQ-A had a good convergent validity similar to the original version,and there was no significant difference between the two instruments in terms of divergent validity.The correlation results of the variables under study and theirmean and standard deviation are reported in Table 3.

Table 2:Fit indices for PSWQ and PSWQ-A

Reliability of PSWQ and PSWQ-A

The reliability of the questionnaires was tested by the internal consistency method.The results of this study showed that PSWQ and PSWQ-A questionnaires have good reliability(=0.87 for both questionnaires).Cronbach’s alpha coefficient＞0.7 indicates the high internal consistency of the questionnaire.Therefore,PSWQ and PSWQ-A can be used for research and also in therapeutic settings.The internal consistency coefficients of the measures used in this study and the internal consistency coefficients of the PSWQ factors are reported in Table 3(numbers on diagonal).

DISCUSSION

GAD is one of the most common anxiety disorders,and it causes many damages on the individual and community(Kessler et al.,2005).The main characteristic of GAD is chronic and uncontrollable worry about possible future feared events.Pharmacological treatments have shown significant efficacy in the treatment of GAD,but the side effects of psychotropic medications and the need for prolonged use can lead to pa-tients being discontinued and recurring symptoms(Bandelow and Michaelis,2015).Psychological therapies,especially cognitive-behavioral therapies,have been developed in recent years,and studies have shown efficacy in the treatment of GAD.These therapies require measures that,in addition to high accuracy,reduce time wasting to monitor changes in clients’ worry during treatment.On the other hand,having favorable psychometric properties and shortness of worry assessment measures in psychological studies will improve the quality of these studies.

Table 3:Correlations between the me asured variables(Pearson’s correlations)

In the present study,we examined the psychometric properties of two measures designed to assess worry in a nonclinical population.The results showed that the PSWQ-A had better psychometric properties than the PSWQ.More precisely,the results of the exploratory factor analysis showed that each item loading on PSWQ-A was higher than item loading on PSWQ.Also,the PSWQ-A single-factor structure explains 53.1% of the total variance of the questionnaire,which is slightly lower(49.1%)for the two-factor structure of the PSWQ.Similar results were obtained in confirmatory factor analysis.The goodness of fit indices for both questionnaires is within an acceptable range(＞0.90),but the closer this value is to one,the better fit of the model(Hu and Bentler,1999).The findings of this study support the better fit of the PSWQ-A single-factor model,and the fit indices for this questionnaire are significantly higher than those calculated for PSWQ.However,it should be noted that the fit indices for PSWQ-A improved significantly after releasing a covariance error.Confirmatory factor analysis graphs also show that item loadings in PSWQ are particularly weak in factor 2,but in PSWQ-A,all items have acceptable factor loadings(＞0.50).The convergent validity of these two questionnaires was the same,and there was no significant difference between PSWQ and PSWQ-A,and the correlation of both questionnaires was the same with the GAD-7 questionnaire.The internal consistency method was used to assess the reliability of these two measures.The results showed that the two measures were not significantly different in reliability,and their consistency coefficients were in acceptable range(=0.87).Overall,the present study showed that the psychometric properties of the abbreviated version of the PSWQ-A were better than the psychometric properties of the PSWQ and PSWQ- A could be used for research and therapeutic purposes instead of PSWQ.

The results of the present study were consistent with the findings of the study by Hopko et al.(2003).In their study on the elderly(55-88 years)individuals,they showed that the PSWQ-A had a robust fit,high internal consistency,and acceptable convergent validity.In this study for PSWQ-A,acceptable test-retest reliability and good divergent validity were reported.Kertz et al.(2014)compared the psychometric properties of the original(PSWQ),abbreviated(PSWQ-A),and ultra-brief 3-item version of PSWQ.Again,the findings of the present study were consistent with the mentioned study,such that they found that both abbreviated measures had better psychometric properties than the original version,and the PSWQ-A fits better than the PSWQ.DeLapp,Chapman,and Williams(2016)examined the psychometric properties of PSWQ-A in students from two different African-American and European-American cultures.The results indicate the favorable internal consistency of PSWQ-A(Cronbach’s alpha coefficient for both samples＜0.90)and its good convergent validity with other anxiety measures.However,the divergent validity of PSWQ-A has been reported as poor in the studied population.A study on a clinical sample aimed to compare PSWQ and PSWQ-A in a clinical and nonclinical sample showed that both measures had good internal consistency and good test-retest reliability with interval of 12 weeks.The study also found that the single-factor structure of PSWQ-A fits better than PSWQ and can be used as a valid measure(Wuthrich et al.,2014).

The limitations of the present study should be considered.Lack of evaluation of test-retest reliability is one of the important limitations of this study.Moreover,in this study,the measures used for the examination of the convergent validity were limited to the GAD-7 scale.In addition,the divergent validity of the PSWQ-A was not evaluated.

Conclusion

Despite the limitations,the present study showed that the PSWQ-A has desirable psychometric properties and can be used as a substitute measure for the original version of the PSWQ.However,future studies can examine the psychometric properties of PSWQ-A in a clinical sample and determine its cutoff score.Researchers can also examine the psychometric properties of the ultra-brief 3-item version of the PSWQ.

Additional file

Additional file 1:STROBE checklist.

The researchers are grateful to the Social Determinant of Health Research Center,Zanjan University of Medical Sciences and all the participants who have contributed to this research.

Study concept and design,analysis and interpretation of data:AR and OS;drafting of the manuscript:AR and HM;critical revision of the manuscript for important intellectual content:OS;study supervision:HM.All authors approved the final version of the manuscript.

The authors declare that there are no conflicts of interest.

None.

The present study was carried out after approval of Social Determinants of Health Research Center of Zanjan University of Medical Science with the project code of A-12-924-5 on October 7,2017.The project also approved by the Ethics Committee of Zanjan University of Medical Sciences on October 17,2017 and the approval ID was IR.ZUMS.REC.1396.187.

The authors certify that they have obtained all appropriate participant consent forms.In the form the participants have given their consent for their images and other clinical information to be reported in the journal.The participants understand that their names and initials will not be published and due efforts will be made to conceal their identity.

This study followed the STrengthening the Reporting of OBservational studies in Epidemiology(STROBE)statement.

The statistical methods of this study were reviewed by the biostatistician of Zanjan University of Medical Sciences,Iran.

The Copyright License Agreement has been signed by all authors before publication.

Each participant was informed of their score.Participants could also access their results up to one year after the end of the study.Results will be disseminated through presentations at scientific meetings and/or by publication in a peer-reviewed journal.

Checked twice by iThenticate.

Externally peer reviewed.

This is an open access journal,and articles are distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License,which allows others to remix,tweak,and build upon the work non-commercially,as long as appropriate credit is given and the new creations are licensed under the identical terms.