Year 2019, Volume 6, Issue 2, Pages 279 - 299 2019-06-30

Thematic Content Analysis of Studies Using Generalizability Theory

Gülşen Taşdelen Teker [1] , Neşe Güler [2]

0 111

One of the important theories in education and psychology is Generalizability (G) Theory and various properties distinguish it from the other measurement theories. To better understand methodological trends of G theory, a thematic content analysis was conducted. This study analyzes the studies using generalizability theory in the field of education in Turkey by using the method of thematic content analysis. It reviews 60 studies, including 31 articles and 29 theses published from 2004 to 2017. The selected studies underwent thematic content analysis using parameters including tagged information, aim, G Theory type, number of facets used in the study, Turkish word for “facet,” object of measurement, sample size, design type, mixed-design availability, shared results of G and D studies, computer programs, method of calculating negative variance, availability of fixed facets, and design balance. The data were interpreted on the basis of frequencies; both table and figures are included in the study. According to the results, there is an increase in the number of studies conducted by using G theory by years. Of these, many compare theories; most of them applying univariate G Theory and consider two-faceted measurement situations. While a small subset of studies features mixed design, a large group features crossed design, with individuals as the object of measurement. The computer program most commonly used in analyses is EduG. The majority of studies use balanced design. Recommendations are provided accordingly with the results.
Educational sciences, Generalizability theory, Thematic content analysis
  • Arık, R. S. & Türkmen, M. (2009). Examination of the articles in the scientific journals published in the field of educational sciences. Paper presented at I. International Congress of Educational Research, Çanakkale, Turkey.
  • Atılgan, H. (2004). Genellenebilirlik kuramı ve çok değişkenlik kaynaklı Rasch modelinin karşılaştırılmasına ilişkin bir araştırma [A research on the comparison of the generalizability theory and many facet Rasch model] (Doctoral Dissertation). Hacettepe University, Ankara.
  • Atılgan, H. (2013). Sample size estimation of G and Phi coefficients in generalizability theory. Eurasian Journal of Educational Research, 51, 215–228.
  • Au, W. (2007). High-stakes testing and curricular control: A qualitative metasynthesis. Educational Researcher, 36, 258–267.
  • Bektaş, M., Dündar, H. & Ceylan, A. (2013). Investigation of several variables papers national classroom teacher education symposium. Uşak University Journal of Social Sciences, 6(2), 201–226. DOI:
  • Bloch, R. & Norman, G. (2011). G String 4 user manual (Version 6.1.1). Hamilton, Ontario, Canada. Retrieved from 611.pdf
  • Brennan, R. L. (2011). Generalizability Theory and Classical Test Theory. Applied Measurement in Education, 24, 1–21. doi:10.1080/08957347.532417.
  • Brennan, R. L. (2001). Generalizability Theory. New York: Springer-Verlag.
  • Brennan, R. L. (1997). A perspective on the history of Generalizability Theory. Educational Measurement: Issues and Practice, 16(4), 14–20.
  • Brennan, R. L. (1992). Elements of Generalizability Theory. NY: Springer-Verlag.
  • Çalık, M. & Sözbilir, M. (2014). Parameters of content analysis. Education and Science, 39(174), 33–38. doi:10.15390/EB.2014.3412
  • Çiltaş, A. (2012). Content analysis of the graduate thesis and dissertations in mathematics education in Turkey between 2005-2010. The Journal of Academic Social Science Studies, 5(7), 211–228.
  • Çiltaş, A., Güler, G. & Sözbilir, M. (2012). Mathematics education research in Turkey: A content analysis study. Educational Sciences: Theory & Practice, 12(1), 565–580.
  • Deliceoğlu, G. (2009). The Comparison of the reliabilities of the soccer abilities’rating scale based on the Classical Test Theory and Generalizability. (Doctoral Dissertation). Ankara University, Ankara.
  • Doğru, M., Gençosman, T., Ataalkın, A. N. & Şeker, F. (2012). Fen bilimleri eğitiminde çalışılan yüksek lisans ve doktora tezlerinin analizi [Analysis of master's and doctoral theses in science education]. Journal of Turkish Science Education, 9(1), 49–64.
  • Finfgeld, D. L. (2003). Metasynthesis: The state of the art-so far. Qualitative Health Research, 13(7), 893–904. DOI: 10.1177/1049732303253462
  • Gadbury-Amyot, C. C., Kim, J., Palm, R. L., Mills, G. E., Noble, E. & Overman, P. R. (2003). Validity and reliability of portfolio assessment of competency in a baccalaureate dental hygiene program. Journal of Dental Education, 67(9), 991-1002.
  • Göktaş, Y., Küçük, S., Aydemir, M., Telli, E., Arpacık, Ö., Yıldırım, G. & Reisoğlu, İ. (2012). Educational technology research trends in Turkey: A content analysis of the 2000–2009 decade. Educational Sciences: Theory & Practice, 12(1), 191–196.
  • Gülbahar, Y. & Alper, A. (2009). Trends and issues in educational technologies: A review of recent research in TOJET. The Turkish Online Journal of Educational Technology – TOJET, 8(2), 124-135.
  • Güler, N., Kaya Uyanık, G. & Taşdelen Teker, G. (2012). Genellenebilirlik Kuramı [Generalizability Theory]. Ankara: PegemA Yayıncılık.
  • Güler, N. (2008). A research on classical test theory, generalizability theory and Rasch model (Doctoral Dissertation). Hacettepe University, Ankara.
  • Günay, R. & Aydın, H. (2015). Inclinations in studies into multicultural education in Turkey: A content analysis study. Education and Science, 40(178), 1–22. DOI:
  • Hambleton, R. K. & Jones, R. W. (1993). Comparison of classical test theory and item response theory and their applications to test development. Educational Measurement: Issues and Practice, 12(3), 3847.
  • Kaleli Yılmaz, G. (2015). Analysis of technological pedagogical content knowledge studies in Turkey: A meta-synthesis study. Education and Science, 40(178), 103–122. DOI:
  • Karadağ, E. (2009). Eğitim bilimleri alanında yapılmış doktora tezlerinin incelenmesi [A Thematic Analysis on Doctoral Dissertations Made In the Area of Education Sciences],. Ahi Evran Üniversitesi Eğitim Fakültesi Dergisi 10(3), 75–87.
  • Kersting, N. (2008). Using Video Clips of Mathematics Classroom Instruction as Item Prompts to Measure Teachers’ Knowledge of Teaching Mathematics. Educational and Psychological Measurement, 68(5), 845-861. DOI:10.1177/0013164407313369
  • Kılıç Çakmak, E., Çebi, A., Mihçi, P., Günbatar, M. S. & Akçayir, M. (2013). A content analysis of educational technology research in 2011. 4th International Conference on New Horizons in Education. INTE 2013 Proceedings Book, 397–409.
  • Lafave, M. R. & Butterwick, D. J. (2014). A generalizability theory study of athletic taping using the Technical Skill Assessment Instrument. Journal of Athletic Training, 49(3), 368-372. doi: 10.4085/1062-6050-49.2.22
  • Lane, S., Liu, M., Ankenmann, R. D. & Stone, C. A. (1996). Generalizability and Validity of a Mathematics Performance Assessment. Journal of Educational Measurement, 33(1), 71-92.
  • Miles, M, B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded Sourcebook. (2nd ed). Thousand Oaks, CA: Sage.
  • Patton, M.Q. (2002). Qualitative research and evaluation methods (3rd Ed.). London: Sage Publications, Inc.
  • Pekin Z., Çetin S. & Güler N. (2018). Comparison of Interrater Reliability Based on Different Theories for Autism Social Skills Profile. Journal of Measurement and Evaluation in Education and Psychology, 9(2), 202-215.
  • Rios, J.A., Li, X., & Faulkner-Bond, M. (2012, October). A review of methodological trends in generalizability theory. Paper presented at the annual conference of the Northeastern Educational Research Association, Rocky Hill, CT.
  • Saban, A. (2009). Content analysis of Turkish studies about the multiple intelligences theory. Educational Sciences: Theory & Practice, 9(2), 833–876.
  • Shavelson, R. J. & Webb, N. M. (1991). Generalizability Theory: A primer. Newbury Park, CA: Sage.
  • Shavelson, R. J., Baxter, G. P. & Pine, J. (1991). Performance Assessment in Science. Applied Measurement in Education, 4(4), 347-362. DOI: 10.1207/s15324818ame0404_7
  • Taşdelen Teker, G. & Odabaşı, O. (2018). Reliability of scores obtained from standardized patient and instructor assessments. European Journal of Dental Education, 23, 88-94. DOI: 10.1111/eje.12406
  • Turner, A. A., Lozano-Nieto, A. & Bouffard, M. (2006). Generalizability of extracellular-to-intracellular fluid ratio using bio-impedance spectroscopy. Physiological Measurement, 27(4), 385-397. DOI: 10.1088/0967-3334/27/4/005
  • Yalçın, S., Yavuz, H. Ç. & İlgün Dibek, M. (2015). Content analysis of papers published in educational journals with high ımpact factors. Education and Science, 40 (182), 1–28. DOI:10.15390/EB.2015.4868
  • Yin, Y. & Shavelson, R. J. (2008). Application of Generalizability Theory to Concept Map Assessment Research. Applied Measurement in Education, 21(3), 273-291. DOI: 10.1080/08957340802161840
  • Walsh, D. & Downe, S. (2005). Meta-synthesis method for qualitative research: A literature review. Journal of Advanced Nursing, 50(2), 204–211.
  • Webb, N. M., Rowley, G. L., & Shavelson, R. J. (1988). Using Generalizability Theory in counseling and development. Measurement and Evaluation in Counseling and Development, 21, 81–90.
Primary Language en
Subjects Education, Scientific Disciplines
Published Date June
Journal Section Articles

Orcid: 0000-0003-3434-4373
Author: Gülşen Taşdelen Teker (Primary Author)
Institution: Hacettepe University
Country: Turkey

Orcid: 0000-0002-2836-3132
Author: Neşe Güler
Institution: İzmir Demokrasi University
Country: Turkey

Bibtex @research article { ijate569996, journal = {International Journal of Assessment Tools in Education}, issn = {}, eissn = {2148-7456}, address = {İzzet KARA}, year = {2019}, volume = {6}, pages = {279 - 299}, doi = {10.21449/ijate.569996}, title = {Thematic Content Analysis of Studies Using Generalizability Theory}, key = {cite}, author = {Taşdelen Teker, Gülşen and Güler, Neşe} }
APA Taşdelen Teker, G , Güler, N . (2019). Thematic Content Analysis of Studies Using Generalizability Theory. International Journal of Assessment Tools in Education, 6 (2), 279-299. DOI: 10.21449/ijate.569996
MLA Taşdelen Teker, G , Güler, N . "Thematic Content Analysis of Studies Using Generalizability Theory". International Journal of Assessment Tools in Education 6 (2019): 279-299 <>
Chicago Taşdelen Teker, G , Güler, N . "Thematic Content Analysis of Studies Using Generalizability Theory". International Journal of Assessment Tools in Education 6 (2019): 279-299
RIS TY - JOUR T1 - Thematic Content Analysis of Studies Using Generalizability Theory AU - Gülşen Taşdelen Teker , Neşe Güler Y1 - 2019 PY - 2019 N1 - doi: 10.21449/ijate.569996 DO - 10.21449/ijate.569996 T2 - International Journal of Assessment Tools in Education JF - Journal JO - JOR SP - 279 EP - 299 VL - 6 IS - 2 SN - -2148-7456 M3 - doi: 10.21449/ijate.569996 UR - Y2 - 2019 ER -
EndNote %0 International Journal of Assessment Tools in Education Thematic Content Analysis of Studies Using Generalizability Theory %A Gülşen Taşdelen Teker , Neşe Güler %T Thematic Content Analysis of Studies Using Generalizability Theory %D 2019 %J International Journal of Assessment Tools in Education %P -2148-7456 %V 6 %N 2 %R doi: 10.21449/ijate.569996 %U 10.21449/ijate.569996
ISNAD Taşdelen Teker, Gülşen , Güler, Neşe . "Thematic Content Analysis of Studies Using Generalizability Theory". International Journal of Assessment Tools in Education 6 / 2 (June 2019): 279-299.