Taiwan’s university program evaluation has become an important trend. To ensure their quality, meta-evaluation is one of the important methods. The university program evaluation is a central issue in what concerns in Taiwan’s university programs. In terms of meta-evaluation, the Joint Committee on Standards for Educational Evaluation (JCSEE) Program Evaluation Standards have wider applications but more general standard descriptions and more trivial related checklists. This study aims at the meta-evaluation indicators of Taiwan’s university departmental self-evaluation based on the JCSEE Program Evaluation Standards. According to the weights of meta-evaluation’s five major category standards of university program evaluation analyzed in this study, the most important is Propriety Standards, and the secondary important in order are Utility Standards, Feasibility Standards, Accuracy Standards and Evaluation Accountability Standards, different from the JCSEE’s original order, which is worthy of follow-up studies. In this regard, it mainly amended and identified indicators via expert questionnaires and then used Fuzzy Delphi method questionnaires to integrate experts and scholars’ opinions on indicator importance and their weights. The data analysis of 60 meta-evaluation indicators of Taiwan’s university programs was constructed by referring to the JCSEE Program Evaluation Standards’ 30 standards in five major categories. It was concluded that the importance of meta-evaluation category standards of university program evaluation in order is U6 Meaningful Processes and Products (Utility Standards), F3 Contextual Viability (Feasibility Standards), P1 Responsive and Inclusive Orientation (Propriety Standards), A3 Reliable Information (Accuracy Standards) and E1 Evaluation Documentation (Evaluation Accountability Standards). The JCSEE’s standards were constructed, and conclusions and recommendations were provided to university departments for reference while carrying out self-evaluations.

1. Alford, H., & Aharonian, A. (2011). What is institutional research? Retrieved from https://bit.ly/2Vo6U5T
2. Attuluri, S. (2019). Institutional changes in Mexico higher education: A reform to U.S accreditations. International Journal of Humanities, Arts and Social Sciences, 5(1), 9-19. doi:https://dx.doi.org/10.20469/ijhss.5.10002-1
3. Ayob, A. H., & Morell, J. A. (2016). The historical path of evaluation as reflected in the content of evaluation and program planning. Evaluation and Program Planning, 58, 20–27. doi:https://doi.org/10.1016/j.evalprogplan.2016.05.006
4. Blair, E., & Noel, K. V. (2014). Improving higher education practice through student evaluation systems: Is the student voice being heard? Assessment & Evaluation in Higher Education, 39(7), 879–894. doi:https://doi.org/10.1080/02602938.2013.875984
5. Chan, S.-J., & Chan, Y. (2015). Higher education research community in Taiwan: An emerging field. Higher Education Policy, 28(4), 459–475. doi:https://doi.org/10.1057/hep.2015.16
6. Chen, S. J., & Hwang, C. L. (1992). Fuzzy multiple attribute decision making methods. In Fuzzy multiple attribute decision making. Heidelberg, Germany: Springer.
7. Cheng-Ta, W. (2008). Educational policy analysis: Concepts methods, and applications. Taipei, Taiwan: Higher Education.
8. Cousins, J. B., Goh, S. C., Elliott, C. J., & Bourgeois, I. (2014). Framing the capacity to do and use evaluation. New Directions for Evaluation, 2014(141), 7–23. doi:https://doi.org/10.1002/ev.20076
9. Crawford, E. R., Aguayo, D., & Valle, F. (2017). Counselors as leaders who advocate for undocumented stu-dents’ education. Journal of Research on Leadership Education, 14(2), 119–150. doi:https://doi.org/10.1177/1942775117739301
10. Froh, R. C. (1991). Using student persistence research to strengthen the quality of teaching and institutional process. In M. Nugent & J. L. Ratcliff (Eds.), Measurement of quality in higher education. Washington, DC,WA: Office of Educational Research and Improvement.
11. Gall, M. D., Gall, J. P., & Borg, W. R. (2003). Educational research: An introduction. Boston, MA: Allyn & Bacon.
12. Gatpandan, M. P., & Ambat, S. C. (2017). Implementing knowledge discovery in enhancing university student services portfolio management in higher education institutions. Journal of Advanced Research in Social Sciences and Humanities, 2(4), 211-220. doi:https://doi.org/10.26500/jarssh-02-2017-0306
13. Harvey, L., & Williams, J. (2010). Fifteen years of quality in higher education. Quality in Higher Education, 16(1), 3–36. doi:https://doi.org/10.1080/13538321003679457
14. HEEACT. (2015). Ministry of educations pilot implementation project for certifying universities program evaluation outcome examination. Retrieved from https://bit.ly/2ZabTrM
15. Horta, H. (2017). Higher-education researchers in Asia: The risks of insufficient contribution to international higher-education research. In Researching higher education in Asia (pp. 15–36). Gateway East, Singapore: Springer.
16. Hou, A. Y.-C., Morse, R., & Wang, W. (2015). Recognition of academic qualifications in transnational higher education and challenges for recognizing a joint degree in Europe and Asia. Studies in Higher Education, 42(7), 1211–1228. doi:https://doi.org/10.1080/03075079.2015.1085010
17. JCSEE. (2015). Program evaluation standards statements. Retrieved from https://bit.ly/2Bfbe0g
18. Kyriakides, L., & Campbell, R. (2004). School self-evaluation and school improvement: A critique of values and procedures. Studies in Educational Evaluation, 30(1), 23–36. doi:https://doi.org/10.1016/s0191-491x(04)90002-8
19. Mertens, D. M., & Wilson, A. T. (2018). Program evaluation theory and practice. New York, NY: Guilford Publications.
20. Misnistry of Education. (2015). Administrative goals and key tasks. Retrieved from https://bit.ly/2ZbTzys
21. Murray, T. J., Pipino, L. L., & van Gigch, J. P. (1985). A pilot study of fuzzy set modification of delphi. Human Systems Management, 5(1), 76–80. doi:https://doi.org/10.3233/hsm-1985-5111
22. Nazari-Shirkouhi, S., Mousakhani, S., Tavakoli, M., Dalvand, M. R., Šaparauskas, J., & Antuchevicien ˇ e, J. (2020). ̇ Importance-performance analysis based balanced scorecard for performance evaluation in higher education institutions: An integrated fuzzy approach. Journal of Business Economics and Management, 21(3), 647–678. doi:https://doi.org/10.3846/jbem.2020.11940
23. Noorderhaben, N. (1995). Strategic decision making. Boston, MA: Addison Wesley.
24. Perterson, K. D. (2000). Teacher evaluation. New York, NY: Corwin Press Inc.
25. Schumacher, S., & McMillan, J. (2006). Research in education evidence based inquiry (6th ed.). Boston, MA: Allyn & Bacon.
26. Stufflebeam, D. L. (2012). Program evaluations summary meta evaluation checklist (based on the program evaluation standards). Retrieved from https://bit.ly/3i6VJIe
27. Sturges, K. M., & Howley, C. (2016). Responsive meta-evaluation. American Journal of Evaluation, 38(1), 126–137. doi:https://doi.org/10.1177/1098214016630405
28. Tang, X., Wei, G., & Gao, H. (2019). Models for multiple attribute decision making with interval-valued pythagorean fuzzy muirhead mean operators and their application to green suppliers selection. Informatica, 30(1), 153–186. doi:https://doi.org/10.15388/informatica.2018.202
29. Volden, G. H. (2018). Public project success as seen in a broad perspective. Evaluation and Program Planning, 69, 109–117. doi:https://doi.org/10.1016/j.evalprogplan.2018.04.008
30. Wei, G. (2018). TODIM method for picture fuzzy multiple attribute decision making. Informatica, 29(3), 555–566. doi:https://doi.org/10.15388/informatica.2018.181
31. Wolff, R. A. (2010). Key trends for quality assurance in the us today. Evaluation in Higher Education, 4(2), 55–87.
32. Wudhikarn, R. (2016). An efficient resource allocation in strategic management using a novel hybrid method. Management Decision, 54(7), 1702–1731. doi:https://doi.org/10.1108/md-08-2015-0380
33. Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2011). The program evaluation standards: A guide for evaluators and evaluation users. Thousand Oaks, CA: Sage.

Chen, Y.-N., Kang, J.-L., & Guo, C.-U. (2020). The construction of meta-evaluation indicators of Taiwan’s university program evaluation: JCSEE program evaluation standards as a framework. International Journal of Humanities, Arts and Social Sciences, 6(2), 10-24. doi:10.20469/ijhss.6.20002-1