Behavior Intervention Competency Scale: Validity and Reliability Study
Özge Çevik – Necmettin Erbakan University, Türkiye - ORCID: 0000-0002-3633-370X
Hakan Sarı – Necmettin Erbakan University, Türkiye - ORCID: 0000-0003-4528-8936
Abstract
This study developed the Behavior Intervention Competency Scale to assess the behavioral intervention competencies of middle school teachers working with students with autism spectrum disorder. The study included 350 teachers from 10 subject areas. An initial 40-item pool was reviewed by three special education experts and one measurement and evaluation expert; 10 items were removed, resulting in a 30-item candidate form. Exploratory factor analysis was conducted with an independent sample of 185 teachers, and confirmatory factor analysis was performed with a separate sample of 165 teachers. Nine items were removed following exploratory analysis, and the remaining 21 items formed three subdimensions: Before Behavioral Intervention, During Behavioral Intervention, and After Behavioral Intervention. Confirmatory factor analysis supported the correlated three-factor model. Cronbach’s alpha was .96 for the total scale and .86, .93, and .93 for the subscales. Composite reliability values ranged from .86 to .93, and average variance extracted values ranged from .53 to .67, supporting convergent validity. Discriminant validity was largely supported, although overlap was observed between the before- and during-intervention dimensions. A four-week test–retest analysis yielded a significant coefficient of r=.75. Overall, the scale demonstrated strong reliability and acceptable validity evidence.
Keyword: Behavioral Intervention
References
American Psychiatric Association (2022). Diagnostic and statistical manual of mental disorders (5th ed, text rev). American Psychiatric Association Publishing.
Bagozzi, RP, & Yi, Y (1988). On the evaluation of structural equation models. Journal of the Academy of Marketing Science, 16, 74–94. https://doi.org/10.1007/BF02723327
Boateng, GO, Neilands, TB, ..., & Young, SL (2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health, 6, Article 149. https://doi.org/10.3389/fpubh.2018.00149
Bradshaw, CP, Koth, CW, Thornton, LA, & Leaf, PJ (2010). Altering school climate through school-wide positive behavioral interventions and supports: Findings from a group-randomized effectiveness trial. Prevention Science, 11(2), 133–144. https://doi.org/10.1007/s11121-009-0119-1
Bryant, FB, & Yarnold, PR (1995). Principal components analysis and exploratory and confirmatory factor analysis. In LG Grimm, & PR Yarnold (Eds), Reading and understanding multivariate statistics (pp 99–136). American Psychological Association.
Büyüköztürk, Ş (2016). Handbook of data analysis for the social sciences (22nd ed.). Pegem Academy.
Byrne, BM (1994). Structural equation modeling with EQS and EQS/Windows: Basic concepts, applications, and programming. SAGE Publications.
Cattell, RB (1978). The scientific use of factor analysis. Plenum Press.
Comrey, AL, & Lee, HB (1992). A first course in factor analysis (2nd ed.). Lawrence Erlbaum Associates.
Çokluk, Ö, Şekercioğlu, G, & Büyüköztürk, Ş (2010). Multivariate statistics for the social sciences: SPSS and LISREL applications. Pegem Academy.
DeVellis, RF (2017). Scale development: Theory and applications (4th ed.). SAGE Publications.
Everitt, BS (1975). Multivariate analysis: The need for data, and other problems. Heinemann Educational Books.
Fabrigar, LR, Wegener, DT, MacCallum, RC, & Strahan, EJ (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4(3), 272–299. https://doi.org/10.1037/1082-989X.4.3.272
Field, A (2018). Discovering statistics using IBM SPSS statistics (5th ed.). SAGE Publications.
Finney, SJ, & DiStefano, C (2006). Non-normal and categorical data in structural equation modeling. In GR Hancock, & RO Mueller (Eds), Structural equation modeling: A second course (pp 269–314). Information Age Publishing.
Fiske, KE, & Delmolino, KA (2008). Treatment integrity of school-based behavior analytic interventions: A review of the research. Behavior Analysis in Practice, 1(2), 19–25. https://doi.org/10.1007/BF03391724
Fornell, C, & Larcker, DF (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.1177/002224378101800104
George, D, & Mallery, P (2019). IBM SPSS statistics 26 step by step: A simple guide and reference (16th ed). Routledge. https://doi.org/10.4324/9780429056765
Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Lawrence Erlbaum Associates.
Hair, JF, Black, WC, Babin, BJ, & Anderson, RE (2019). Multivariate data analysis (8th ed.). Cengage.
Henseler, J, Ringle, CM, & Sarstedt, M (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43, 115–135. https://doi.org/10.1007/s11747-014-0403-8
Hooper, D, Coughlan, J, & Mullen, MR (2008). Structural equation modeling: Guidelines for determining model fit. Electronic Journal of Business Research Methods, 6(1), 53–60.
Hopkins, WG (2014). A new view of statistics. http://www.sportsci.org/resource/stats/effectmag.html
HumeK, Steinbrenner, JR, ..., & Savage, MN (2021). Evidence-based practices for children, youth, and young adults with autism: Third generation review. Journal of Autism and Developmental Disorders, 51(11), 4013–4032. https://doi.org/10.1007/s10803-020-04844-2
Hunt, RJ (1986). Percent agreement, Pearson’s correlation, and kappa as measures of interexaminer reliability. Journal of Dental Research, 65(2), 128–130.
Kline, P (1979). Psychometrics and psychology. Academic Press.
Kline, P (1994). An easy guide to factor analysis. Routledge.
Kline, RB (2016). Principles and practice of structural equation modeling (4th ed.). Guilford Press.
Lorenzo-Seva, U (2022). SOLOMON: A method for splitting a sample into equivalent subsamples in factor analysis. Behavior Research Methods, 54(6), 2665–2677. https://doi.org/10.3758/s13428-021-01750-y
MacCallum, RC, Widaman, KF, Zhang, S, & Hong, S (1999). Sample size in factor analysis. Psychological Methods, 4(1), 84–99. https://doi.org/10.1037/1082-989X.4.1.84
March, RE, & Horner, RH (2002). Feasibility and contributions of functional behavioral assessment in schools. Journal of Emotional and Behavioral Disorders, 10(3), 158–170. https://doi.org/10.1177/10634266020100030401
Marsh, HW, Hau, KT, ..., & Peschar, J (2006). OECD’s brief self-report measure of educational psychology’s most useful affective constructs: Cross-cultural psychometric comparisons across 25 countries. International Journal of Testing, 6(4), 311–360. https://doi.org/10.1207/s15327574ijt0604_1
Ministry of National Education (2018). Regulation on special education services (Official Gazette No. 30471). https://mevzuat.meb.gov.tr/dosyalar/1963.pdf
Ministry of National Education, General Directorate of Special Education and Guidance Services (2017). Circular on inclusive/integrated education practices updated. https://orgm.meb.gov.tr/www/kaynastirmabutunlestirme-yoluyla-egitim-uygulamalari-genelgesi-guncellendi/icerik/934
Mundfrom, DJ, Shaw, DG, & Ke, TL (2005). Minimum sample size recommendations for conducting factor analyses. International Journal of Testing, 5(2), 159–168. https://doi.org/10.1207/s15327574ijt0502_4
Nunnally, JC, & Bernstein, IH (1994). Psychometric theory (3rd ed.). McGraw-Hill.
Özdamar, K (2004). Statistical data analysis with software packages. Kaan Publishing.
Pallant, J (2005). SPSS survival manual: A step by step guide to data analysis using SPSS for Windows (Version 12) (2nd ed.). Open University Press.
Pinkelman, SE, & Horner, RH (2017). Improving implementation of function-based interventions: Self-monitoring, data collection, and data review. Journal of Positive Behavior Interventions, 19(4), 232–243. https://doi.org/10.1177/1098300716683634
Sarmento, RP, & Costa, V (2019). Confirmatory factor analysis—A case study. arXiv. https://arxiv.org/abs/1905.05598
Seçer, İ (2017). Psychological test development and adaptation process: SPSS and LISREL applications. Anı Publishing.
Sugai, G, & Horner, RH (2002). The evolution of discipline practices: School-wide positive behavior supports. Child & Family Behavior Therapy, 24(1–2), 23–50. https://doi.org/10.1300/J019v24n01_03
Tabachnick, BG, & Fidell, LS (2013). Using multivariate statistics (6th ed.). Pearson Education.
Tavşancıl, E (2019). Measurement of attitudes and data analysis with SPSS (6th ed.). Nobel Academic Publishing.
Veneziano, L, & Hooper, JA (1997). A method for quantifying content validity of health-related questionnaires. American Journal of Health Behavior, 21(1), 67–70.
Wong, C, Odom, SL, ..., & Schultz, TR (2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45(7), 1951–1966. https://doi.org/10.1007/s10803-014-2351-z
Worthington, RL, & Whittaker, TA (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34(6), 806–838. https://doi.org/10.1177/0011000006288127