Positive Approaches Journal, Volume 11, Issue 1

Volume 11 ► Issue 1 ► May 2022

Reliability of the Temple University Community Participation Measure with Adults with Autism

Wei Song, Mark S. Salzer, Stacy L. Nonnemacher, & Lindsay L. Shea

Abstract

This study examined test-retest reliability of the Temple University Community Participation (TUCP) measures among a group of autistic adults. The sample consisted of 17 autistic adults from the Adult Autism Waiver (AAW) and Adult Community Autism Program (ACAP). At 2-3 days follow-up, TUCP was completed by the same participants again. Results revealed high test-retest reliability coefficients for the measure. No significant differences in the participation outcomes were found over the 2-3 days. These results suggest that TUCP is a reliable tool in the context of community services and supports.

Background

The Pennsylvania Bureau of Supports for Autism and Special Populations (BSASP) and the Pennsylvania Autism Services, Education, Resources, and Training Collaborative (ASERT) aim to support community participation and inclusion of people on the autism spectrum across the lifespan. Due to few, if any, outcome measures available to autistic adults, BSASP and ASERT strive to identify valid measures that could advance our understanding of autistic individuals’ participation experience. This is essential considering existing evidence showing that participation in community-based activities, including social and recreational activities of autistic people, is associated with lower levels of depression and stress ¹ and improved quality of life. ^2,3

The Temple University Community Participation (TUCP) measure ⁴ captures individuals’ participation experiences and preferences across a wide range of community activities. It was first designed for adults with serious mental illnesses. Among adults with psychiatric disabilities, the TUCP has shown robust test-retest reliability ⁴ and achieved good inter-method reliability by showing consistent results with those obtained using the diary checklist approach.⁵

Research has consistently demonstrated that autistic adults participate less and in fewer areas that are meaningful to them compared to adults in the general population and adults with other disabilities ^1,6,7. A previous study also reported a significant decrease of social and community participation in transition from adolescence to adulthood ⁸. However, the measures of community participation and the categories of activities encompassed in the different types of participation are poorly defined in the ASD literature. Many studies either used national surveys⁷ or author-constructed measures.¹ The TUCP captures a full range of participation in multiple areas, including employment, education, leisure, faith, and interactions with friends and family. It also goes beyond measuring the quantitative scales of participation (i.e., frequency and diversity) and provides information about individual preferences and sufficiency of participation.

The existing reliability (or how consistently something is measured) of the TUCP established for people with mental illness gives practitioners and service providers some confidence that the TUCP scores represent useful information about community participation. Nonetheless, it is vital to assess the reliability of the TUCP among autistic people. The main aim of this project is to support the BSASP clinical and quality improvement efforts as they consider possible assessment tools for use with autistic adults they serve. Specifically, this study examined the utility of the TUCP by assessing its test-retest reliability (i.e., the consistency of individual responses) among autistic adults in the Pennsylvania service system.

What Is TUCP

The TUCP measure assesses independent participation (without the support of staff) in a wide range of areas. For each item, the respondent is asked the number of days they participated in an activity in the last 30 days, the importance of an activity, and whether they did the activity enough. Based on the responses, the following participation outcomes were generated, which were used to test test-retest reliability:

Data Collection Method

In collaboration with the Bureau of Supports for Autism and Special Populations (BSASP), participants from the AAW and ACAP were asked to complete the TUCP. All autistic individuals had a formal autism spectrum disorder (ASD) diagnosis. Support Coordinators (SCs) identified individuals who were able to self-report or complete the TUCP for themselves. A total of 21 autistic adults completed the TUCP and completed TUCP information for 17 individuals was included.

Completed TUCPs were collected between March to May 2021 using Qualtrics, a web-based survey software that allows the user to create surveys and generate reports. Links to Qualtrics surveys were sent to SCs. SCs explained how to complete the TUCP to the individuals and determined whether assistance was needed to complete the measure. An interview script was created for the SCs to facilitate their conversations with participants. Based on the levels of support individuals needed, the TUCP was completed independently, with SC assistance (e.g., read aloud, recording answers), or were interviewed by SCs. After the first administration, individuals were asked to complete the TUCP and again within 2-3 days of the first TUCP being completed. The methods of completion (independently or via interview) had to be consistent across the two administrations. Among the 17 individuals with completed TUCPs, seven (41%) completed the survey by themselves, six (35%) completed with help, and 4 (24%) were interviewed by SCs.

Main Findings

The average age was 33.5 years (SD=7.9). All individuals had at least a high school degree. About 77% (n=13) had an annual income of less than $10,000.

For the results of test-retest reliability:

Conclusion and Potential Use

The results from this effort suggest that the TUCP has good-to-excellent test-retest reliability. Thus, the TUCP can be used to reliably measure the community participation of adults on the autism spectrum in BSASP programs. This combined with findings related to the validity of the measure ^9,10, supports its potential use in a range of community-based settings and provides a tool for planning and providing supports and services as well as provide insights into service quality and personal outcomes across service programs.

Acknowledgment

This project was fully supported by the Autism Services, Education, Resources, & Training Collaborative (ASERT), Bureau of Supports for Autism and Special Populations (BSASP).

References

1.     Stacey T-L, Froude EH, Trollor J, Foley K-R. Leisure participation and satisfaction in autistic adults and neurotypical adults. Autism. 2019/05/01 2019;23(4):993-1004. doi:10.1177/1362361318791275.

2.     Billstedt E, Gillberg I, Gillberg C. Aspects of quality of life in adults diagnosed with autism in childhood. Autism. 2011;15(1):7-20. doi:10.1177/1362361309346066.

3.     Bishop-Fitzpatrick L, Smith DaWalt L, Greenberg JS, Mailick MR. Participation in recreational activities buffers the impact of perceived stress on quality of life in adults with autism spectrum disorder. https://doi.org/10.1002/aur.1753. Autism Research. 2017/05/01 2017;10(5):973-982. doi:https://doi.org/10.1002/aur.1753.

4.     Salzer MS, Brusilovskiy E, Prvu-Bettger J, Kottsieper P. Measuring community participation of adults with psychiatric disabilities: Reliability of two modes of data collection. Rehabilitation psychology. 2014;59(2):211-219. doi:10.1037/a0036002.

5.     Salzer MS, Kottsieper P, Brusilovskiy E. Intermethod reliability and factors affecting recall with the Temple University Community Participation measure. Journal of Mental Health. 2015/07/04 2015;24(4):189-195. doi:10.3109/09638237.2015.1036976.

6.     Song W, Shea L, Nonnemacher S, Brusilovskiy E, Townley G, Salzer M. Community Participation Comparison Between Adults on the Autism Spectrum and Adults in the General Population. Journal of Autism and Developmental Disorders. 2021;52, 1610–1621. doi:10.1007/s10803-021-05059-9.

7.     Orsmond GI, Shattuck PT, Cooper BP, Sterzing PR, Anderson KA. Social Participation Among Young Adults with an Autism Spectrum Disorder. Journal of Autism and Developmental Disorders. 2013/11/01 2013;43(11):2710-2719. doi:10.1007/s10803-013-1833-8.

8.     Myers E, Davis BE, Stobbe G, Bjornson K. Community and Social Participation Among Individuals with Autism Spectrum Disorder Transitioning to Adulthood. Journal of Autism and Developmental Disorders. 2015;45(8):2373-2381. doi:10.1007/s10803-015-2403-z.

9.     Shea LL, Verstreate K, Nonnemacher S, Song W, Salzer MS. Self-reported community participation experiences and preferences of autistic adults. Autism. 2021;25(5):1295-1306. doi:10.1177/1362361320987682.

10.   Song W, Salzer MS, Nonnemacher SL, Shea LL. Community participation patterns among autistic adults and associated characteristics: A latent class analysis. Research in Autism Spectrum Disorders. 2021;89:101884. doi:https://doi.org/10.1016/j.rasd.2021.101884.

Biographies

Dr. Wei Song is a postdoctoral fellow in the College of Public Health at Temple University and works with the Temple University Collaborative on Community Inclusion to examine community participation among adults on the autism spectrum. She received funding from the Eagles Autism Challenge Fellowship to conduct this research.

Mark Salzer, Ph.D. is a psychologist and Professor of Social and Behavioral Sciences in the College of Public Health at Temple University. He is also the Principal Investigator and Director of the Temple University Collaborative on Community Inclusion of Individuals with Psychiatric Disabilities, a rehabilitation research and training center funded by the National Institute on Disability, Independent Living, and Rehabilitation Research since 2003. Dr. Salzer’s research focuses on community inclusion and participation.

Stacy L. Nonnemacher received her Ph.D. from Lehigh University. She is currently the Clinical Director for the Bureau of Supports for Autism and Special Populations in the Office of Developmental Programs in Pennsylvania's Department of Human Services.  Stacy is the Editor in Chief of the Positive Approaches Journal. She has been involved in the field of disabilities for over twenty-five years supporting children, adolescents, and adults in home, community, and school settings. Dr. Nonnemacher has conducted applied research and training on Positive Behavioral Support, fostering self-determination, Functional Behavioral Assessment, and the needs of and support for individuals with autism.

Lindsay Shea is a Senior Healthcare Analyst supporting data and policy efforts in ODP’s BSASP. She is also an Associate Professor at Drexel University, and holds a master’s degree in social policy from the University of Pennsylvania and a doctoral degree in health policy from Drexel University.

Contact Information

Dr. Wei Song

College of Public Health, Temple University

Post-doc Fellow

wei.song0001@temple.edu

Mark Salzer:

mark.salzer@temple.edu

570-850-2948

Stacy Nonnemacher

Office of Developmental Programs, Bureau of Supports for Autism and Special Populations

Clinical Director

c-snonnema@pa.gov

Lindsay Shea

Office of Developmental Programs, Bureau of Supports for Autism and Special Populations

Senior Healthcare Analyst

c-lishea@pa.gov