Positive Approaches Journal, Volume 11, Issue 1
Song, Salzer, Nonnemacher
& Shea | 61-68
Volume 11 ► Issue 1 ► May 2022
Reliability of the Temple University
Community Participation Measure with Adults with Autism
Wei Song, Mark S. Salzer, Stacy L. Nonnemacher, & Lindsay L. Shea
Abstract
This study examined
test-retest reliability of the Temple University Community Participation (TUCP)
measures among a group of autistic adults. The sample consisted of 17 autistic
adults from the Adult Autism Waiver (AAW) and Adult Community Autism Program
(ACAP). At 2-3 days follow-up, TUCP was completed by the same participants
again. Results revealed high test-retest reliability coefficients for the
measure. No significant differences in the participation outcomes were found
over the 2-3 days. These results suggest that TUCP is a reliable tool in the
context of community services and supports.
Background
The Pennsylvania Bureau of Supports for Autism and Special Populations (BSASP) and the Pennsylvania Autism Services, Education, Resources, and Training Collaborative (ASERT) aim to support community participation and inclusion of people on the autism spectrum across the lifespan. Due to few, if any, outcome measures available to autistic adults, BSASP and ASERT strive to identify valid measures that could advance our understanding of autistic individuals’ participation experience. This is essential considering existing evidence showing that participation in community-based activities, including social and recreational activities of autistic people, is associated with lower levels of depression and stress 1 and improved quality of life. 2,3
The Temple University Community Participation (TUCP) measure 4 captures individuals’ participation experiences and preferences across a wide range of community activities. It was first designed for adults with serious mental illnesses. Among adults with psychiatric disabilities, the TUCP has shown robust test-retest reliability 4 and achieved good inter-method reliability by showing consistent results with those obtained using the diary checklist approach.5
Research
has consistently demonstrated that autistic adults participate less and in
fewer areas that are meaningful to them compared to adults in the general
population and adults with other disabilities 1,6,7. A previous study also reported a significant decrease
of social and community participation in transition from adolescence to
adulthood 8. However, the
measures of community participation and the categories of activities
encompassed in the different types of participation are poorly defined in the
ASD literature. Many studies either used national surveys7 or author-constructed measures.1 The TUCP captures a full range of
participation in multiple areas, including employment, education, leisure,
faith, and interactions with friends and family. It also goes beyond measuring
the quantitative scales of participation (i.e., frequency and diversity) and
provides information about individual preferences and sufficiency of
participation.
The existing reliability (or how consistently something is measured) of the TUCP established for people with mental illness gives practitioners and service providers some confidence that the TUCP scores represent useful information about community participation. Nonetheless, it is vital to assess the reliability of the TUCP among autistic people. The main aim of this project is to support the BSASP clinical and quality improvement efforts as they consider possible assessment tools for use with autistic adults they serve. Specifically, this study examined the utility of the TUCP by assessing its test-retest reliability (i.e., the consistency of individual responses) among autistic adults in the Pennsylvania service system.
What
Is TUCP
The TUCP measure assesses independent participation (without the support of staff) in a wide range of areas. For each item, the respondent is asked the number of days they participated in an activity in the last 30 days, the importance of an activity, and whether they did the activity enough. Based on the responses, the following participation outcomes were generated, which were used to test test-retest reliability:
1. Participation days = the total number of days of participation across the 24 activities.
2. Breadth ratio= the ratio of the
number of important activities with at least one day of participation by the
number of important activities.
3. Sufficiency ratio= the ratio of the number of important activities in which participants indicated doing enough by the number of important activities
Data Collection Method
In
collaboration with the Bureau of Supports for Autism and Special Populations
(BSASP), participants from the AAW and ACAP were asked to complete the TUCP.
All autistic individuals had a formal autism spectrum disorder (ASD) diagnosis.
Support Coordinators (SCs) identified individuals who were able to self-report
or complete the TUCP for themselves. A total of 21 autistic adults completed
the TUCP and completed TUCP information for 17 individuals was included.
Completed TUCPs were collected between March to May 2021 using Qualtrics, a web-based survey software that allows the user to create surveys and generate reports. Links to Qualtrics surveys were sent to SCs. SCs explained how to complete the TUCP to the individuals and determined whether assistance was needed to complete the measure. An interview script was created for the SCs to facilitate their conversations with participants. Based on the levels of support individuals needed, the TUCP was completed independently, with SC assistance (e.g., read aloud, recording answers), or were interviewed by SCs. After the first administration, individuals were asked to complete the TUCP and again within 2-3 days of the first TUCP being completed. The methods of completion (independently or via interview) had to be consistent across the two administrations. Among the 17 individuals with completed TUCPs, seven (41%) completed the survey by themselves, six (35%) completed with help, and 4 (24%) were interviewed by SCs.
Main Findings
The average age was 33.5 years (SD=7.9). All individuals had at least a high school degree. About 77% (n=13) had an annual income of less than $10,000.
For the results of test-retest reliability:
· Participation days: Pearson correlations between Time 1 and 2 was 0.90 (p<0.01). The total number of participation days were similar across the two administrations (p>0.05). The high correlation between the two timepoints indicated great consistency in reporting participation days.
· Breadth ratio: Pearson correlations between
Time 1 and 2 was 0.88 (p<0.01). The percentages of participating in
important activities were not significantly different across the two
administrations (p>0.05). This indicated that the test–retest reliability
for breadth ration was excellent.
· Sufficiency ratio:
Pearson correlations between Time 1 and 2 was 0.79 (p<0.01). The levels of sufficiency were not significantly different across the
two administrations (p>0.05). As the other two outcomes, the high
correlation indicated good test-retest reliability for sufficiency ratio.
Conclusion and Potential Use
The results from this effort suggest that the TUCP has good-to-excellent test-retest reliability. Thus, the TUCP can be used to reliably measure the community participation of adults on the autism spectrum in BSASP programs. This combined with findings related to the validity of the measure 9,10, supports its potential use in a range of community-based settings and provides a tool for planning and providing supports and services as well as provide insights into service quality and personal outcomes across service programs.
Acknowledgment
This project was fully supported by the Autism Services, Education,
Resources, & Training Collaborative (ASERT), Bureau of Supports for Autism
and Special Populations (BSASP).
References
1. Stacey T-L, Froude EH, Trollor J, Foley K-R. Leisure participation and satisfaction in autistic adults and neurotypical adults. Autism. 2019/05/01 2019;23(4):993-1004. doi:10.1177/1362361318791275.
2. Billstedt E, Gillberg I, Gillberg C. Aspects of quality of life in adults diagnosed with autism in childhood. Autism. 2011;15(1):7-20. doi:10.1177/1362361309346066.
3. Bishop-Fitzpatrick L, Smith DaWalt L, Greenberg JS, Mailick MR. Participation in recreational activities buffers the impact of perceived stress on quality of life in adults with autism spectrum disorder. https://doi.org/10.1002/aur.1753. Autism Research. 2017/05/01 2017;10(5):973-982. doi:https://doi.org/10.1002/aur.1753.
4. Salzer MS, Brusilovskiy E, Prvu-Bettger J, Kottsieper P. Measuring community participation of adults with psychiatric disabilities: Reliability of two modes of data collection. Rehabilitation psychology. 2014;59(2):211-219. doi:10.1037/a0036002.
5. Salzer MS, Kottsieper P, Brusilovskiy E. Intermethod reliability and factors affecting recall with the Temple University Community Participation measure. Journal of Mental Health. 2015/07/04 2015;24(4):189-195. doi:10.3109/09638237.2015.1036976.
6. Song W, Shea L, Nonnemacher S, Brusilovskiy E, Townley G, Salzer M. Community Participation Comparison Between Adults on the Autism Spectrum and Adults in the General Population. Journal of Autism and Developmental Disorders. 2021;52, 1610–1621. doi:10.1007/s10803-021-05059-9.
7. Orsmond GI, Shattuck PT, Cooper BP, Sterzing PR, Anderson KA. Social Participation Among Young Adults with an Autism Spectrum Disorder. Journal of Autism and Developmental Disorders. 2013/11/01 2013;43(11):2710-2719. doi:10.1007/s10803-013-1833-8.
8. Myers E, Davis BE, Stobbe G, Bjornson K. Community and Social Participation Among Individuals with Autism Spectrum Disorder Transitioning to Adulthood. Journal of Autism and Developmental Disorders. 2015;45(8):2373-2381. doi:10.1007/s10803-015-2403-z.
9.
Shea
LL, Verstreate K, Nonnemacher S, Song W, Salzer MS. Self-reported community
participation experiences and preferences of autistic adults. Autism. 2021;25(5):1295-1306. doi:10.1177/1362361320987682.
10. Song W, Salzer MS, Nonnemacher SL, Shea LL. Community participation patterns among autistic adults and associated characteristics: A latent class analysis. Research in Autism Spectrum Disorders. 2021;89:101884. doi:https://doi.org/10.1016/j.rasd.2021.101884.
Biographies
Dr. Wei Song is a postdoctoral fellow in the College of Public Health at Temple University and works with the Temple University Collaborative on Community Inclusion to examine community participation among adults on the autism spectrum. She received funding from the Eagles Autism Challenge Fellowship to conduct this research.
Mark Salzer, Ph.D. is a psychologist and Professor of Social and Behavioral Sciences in the College of Public Health at Temple University. He is also the Principal Investigator and Director of the Temple University Collaborative on Community Inclusion of Individuals with Psychiatric Disabilities, a rehabilitation research and training center funded by the National Institute on Disability, Independent Living, and Rehabilitation Research since 2003. Dr. Salzer’s research focuses on community inclusion and participation.
Stacy L. Nonnemacher received her Ph.D. from Lehigh University. She is currently the Clinical Director for the Bureau of Supports for Autism and Special Populations in the Office of Developmental Programs in Pennsylvania's Department of Human Services. Stacy is the Editor in Chief of the Positive Approaches Journal. She has been involved in the field of disabilities for over twenty-five years supporting children, adolescents, and adults in home, community, and school settings. Dr. Nonnemacher has conducted applied research and training on Positive Behavioral Support, fostering self-determination, Functional Behavioral Assessment, and the needs of and support for individuals with autism.
Lindsay Shea is a Senior Healthcare Analyst supporting data and policy efforts in ODP’s BSASP. She is also an Associate Professor at Drexel University, and holds a master’s degree in social policy from the University of Pennsylvania and a doctoral degree in health policy from Drexel University.
Contact Information
Dr. Wei Song
College of Public Health, Temple University
Post-doc Fellow
wei.song0001@temple.edu
Mark Salzer:
570-850-2948
Stacy Nonnemacher
Office of Developmental Programs, Bureau of Supports for Autism and Special Populations
Clinical Director
Lindsay Shea
Office of Developmental Programs, Bureau of Supports for Autism and Special Populations
Senior Healthcare Analyst