Citra Putriarum Tanudjaya, Michiel Doorman


Indonesian students’ poor performance in the mathematics test of PISA 2015 prompted the decision by the Ministry of Education of Indonesia to pay more attention to the integration of higher-order thinking (HOT) in the curricula starting in 2018. This new regulation emphasizes the need to have a shared understanding of HOT in mathematics on many levels, such as curriculum, pedagogy, and assessment, and among students, teachers and policy makers. This study aims to examine HOT in Indonesian lower secondary mathematics classrooms by assessing students’ ability to demonstrate HOT skills through an open-ended mathematics problem, and by exploring teachers’ views of HOT skills through semi-structured interviews. It involved 372 ninth-grade students and six mathematics teachers from six lower secondary schools in Jakarta and Palembang. The findings show that most students could construct the mathematical model but experienced difficulty in transferring knowledge into new contexts, in applying creative thinking, and with information literacy skills. Besides, some of the teachers were familiar with the concept of HOT, but some viewed HOT as skills for talented students, or HOT problems having a high level of difficulty and long storylines. The knowledge of existing teaching strategies, familiarity with HOT problems, and colleague-support are needed to improve the development of HOT skills in the mathematics classroom.


Higher-order thinking; Curriculum; Pedagogy; Assessment; Teachers’ views; Students’ ability

Full Text:



Abdullah, A. H., Mokhtar, M., Halim, N. D. A., Ali, D. F., Tahir, L. M., & Kohar, U. H. A. (2017). Mathematics teachers’ level of knowledge and practice on the implementation of higher-order thinking skills (HOTS). EURASIA Journal of Mathematics Science and Technology Education, 13(1), 3-17. https://doi.org/10.12973/eurasia.2017.00601a.

Ahmad, S., Prahmana, R. C. I., Kenedi, A. K., Helsa, Y., Arianil, Y., & Zainil, M. (2018). The instruments of higher order thinking skills. Journal of Physics: Conference Series, 943(1), 012053. https://doi.org/10.1088/1742-6596/943/1/012053.

Alhassora, N. S. A., Abu, M. S., & Abdullah, A. H. (2017). Inculcating higher-order thinking skills in mathematics: Why is it so hard? Man in India, 97(13), 51-62. https://doi.org/10.2478/v10274-012-0006-7.

Aini, N. (2018, April 25). KPAI criticizes the use of HOTS problems in national exams [in Bahasa]. Republika. Retrieved from https://republika.co.id/berita/pendidikan/eduaction/18/04/25/p7qrd0382-kpai-kritik-penggunaan-soal-hots-dalam-ujian-nasional.

Alfrey, K., & Cooney, E. (2009, June). Developing a rubric to assess critical thinking in assignments with an open-ended component. Paper presented at the American Society for Engineering Education Annual Conference and Exposition, Austin, TX.

Anwar, L., Budayasa, I. K., Amin, S., & De Haan, D. (2012). Eliciting mathematical thinking of students through realistic mathematics education. Journal on Mathematics Education, 3(1), 55-70. https://doi.org/10.22342/jme.3.1.620.55-70.

Apino, E., & Retnawati, H. (2017). Developing instructional design to improve mathematical higher-order thinking skills of students. Journal of Physics: Conference Series, 812(1), 012100. https://doi.org/10.1088/1742-6596/812/1/012100.

Ariyana, Y., Pudjiastuti, A., Bestary, R., & Zamroni. (2018). The Learning Handbook Oriented Higher-Order Thinking Skills [in Bahasa]. Jakarta: Direktorat Jenderal Guru dan Tenaga Kependidikan Kementrian Pendidikan dan Kebudayaan.

Association of American Colleges and Universities (AACU). (2009). Critical thinking VALUE rubric. Retrieved from https://www.aacu.org/value/rubrics/critical-thinking.

Barrouillet, P., & Gauffroy, C. (2013). The Development of Thinking and Reasoning. London: Psychology Press.

Bieri, J., & Blacker, E. (1956). The generality of cognitive complexity in the perception of people and inkblots. Journal of Abnormal and Social Psychology, 53, 112-117. https://doi.org/10.1037/h0049167.

Bloom, B. (1956). Taxonomy of Educational Objectives Handbook-I: The Cognitive Domain. New York: David McKay.

Brookhart, S. M. (2010). How to Assess Higher-Order Thinking Skills in Your Classroom. Alexandria, VA: ASCD.

Emilya, D., Darmawijoyo, & Putri, R. I. I. (2010). Developing open-ended problems for circle material to improve mathematical reasoning for eight grade students of Sekolah Menengah Pertama Negeri 10 Palembang [in Bahasa]. Jurnal Pendidikan Matematika, 4(2), 8-18. https://doi.org/10.22342/jpm.4.2.316.

Exemplars. (2014). Standards-Based Math Rubric. Retrieved from https://exemplars.com/sites/default/files/2019-06/2019_exemplars_math_rubric_k5pdf.pdf.

Gibson, T. (2012). On the interface: Between media cultures and worlds. Religion & Literature, 44(2), 179-188. Retrieved April 19, 2020, from www.jstor.org/stable/24397677.

Gwambombo, I. (2013). The Effect of Teachers’ Workload on Students’ Academic Performance in Community Secondary Schools: A Study of Mbeya City. The Open University of Tanzania.

Hadi, S., Retnawati, H., Munadi, S., Apino, E., & Wulandari, N. F. (2018). The difficulties of high school students in solving higher-order thinking skills problems. Problems of Education in the 21st Century, 76(4), 520-532. Retrieved from http://oaji.net/articles/2017/457-1533495738.pdf.

Hantoro, J. (2018, April 14). The HOTS Problem that made the High School UN participants noisy [in Bahasa]. Tempo. Retrieved from https://nasional.tempo.co/read/1079399/soal-hots-yang-bikin-gaduh-peserta-un-sma.

Husain, H., Bais, B., Hussain, A., & Samad, S. A. (2012). How to construct open ended questions. Procedia Social and Behavioral Sciences, 60, 456-462. https://doi.org/10.1016/j.sbspro.2012.09.406.

International Centre for STEM Education (ICSE). (2018). Quarterly problem: Round up, please! Freiburg: University of Education. Retrieved from https://icse.eu/quarterly-problem-mathematics/.

Kitcher, P. (1984). The Nature of Mathematical Knowledge. Oxford: Oxford University Press.

Kilpatrick, J., Swafford, J., & Findell, B. (2001). Adding It Up: Helping Children Learn Mathematics. Washington, DC: National Academy Press.

Kirkpatrick, R., & Zang, Y. (2011). The negative influences of exam-oriented education on Chinese high school students: Backwash from classroom to child. Language Testing in Asia, 1(3), 36-45. https://doi.org/10.1186/2229-0443-1-3-36.

Lakatos, I. (1976). Proofs and Refutations: The Logic of Mathematical Discovery. Cambridge: Cambridge University Press.

Leu, D. J., McVerry, J. G., O’Byrne, W. I., Kiili, C., Zawilinski, L., Everett-Cacopardo, H., Kennedy, C., & Forzani, E. (2011). The new literacies of online reading comprehension: Expanding the literacy and learning curriculum. Journal of Adolescent & Adult Literacy, 55(1), 5-14. https://doi.org/10.1598/JAAL.55.1.1.

Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2010). Practical Resources for Assessing and Reporting Intercoder Reliability in Content Analysis Research Projects. Retrieved from http://matthewlombard.com/reliability/index_print.html.

Mathematical Association of America (MAA). (1991). A Call for Change: Recommendations for the Mathematical Preparation of Teachers of Mathematics. Washington, DC: Author.

National Council of Teachers of Mathematics. (1991). Professional Standards for Teaching School Mathematics (Working draft). Reston, VA: Author.

National Research Council. (1989). Everybody Counts. Washington, DC: National Academy of Sciences.

Nugroho, R. A. (2018). Higher-Order Thinking Skills (HOTS). Jakarta: Grasindo.

OECD. (2016). “PISA 2015 Results in Focus”. PISA in Focus, No. 67. Paris: OECD Publishing. https://doi.org/10.1787/aa9237e6-en.

OECD. (2018). “PISA for Development Mathematics Framework”. PISA for Development Assessment and Analytical Framework: Reading, Mathematics, and Science. Paris: OECD Publishing. https://doi.org/10.1787/9789264305274-5-en.

OECD. (2019a). Indonesia - Country Note - PISA 2018 Results. Retrieved from https://www-oecd-org.proxy.library.uu.nl/pisa/publications/PISA2018_CN_IDN.pdf.

OECD. (2019b). “What is PISA?”. PISA 2018 Assessment and Analytical Framework. Paris: OECD Publishing. https://doi.org/10.1787/2c7c311d-en.

Raudenbush, S. W., Rowan, B., & Cheong, Y. F. (1993). Higher order instructional goals in secondary schools: Class, teacher, and school influences. American Educational Research Journal, 30, 523-553. https://doi.org/10.3102/00028312030003523.

Resnick, L. B. (1987). Education and Learning to Think. Washington DC: National Academy Press.

Retnawati, H., Djudi, H., Kartianom, Apino, E., & Anazifa, R. D. (2018). Teachers’ knowledge about higher-order thinking skills and its learning strategy. Problems of Education in the 21st century, 76(2), 215-230. Retrieved from http://oaji.net/articles/2017/457-1524597598.pdf.

Robson, C. (2002). Real world research: A resource for social scientists and practitioner-researchers. Oxford: Blackwell.

Sbaragli, S., & Franchini, E. (2017). Didactic evaluation of standardized fifth grade math tests [in Italian]. Locarno: SUPSI, Dipartimento formazione e apprendimento.

Schoenfeld, A. H. (1992). Learning to think mathematically: Problem solving, metacognition, and sense-making in mathematics. In D. Grouws (Ed.), Handbook for Research on Mathematics Teaching and Learning, 334-370. New York: MacMillan.

Smith, V. D., & Darvas, J. W. (2017). Encouraging student autonomy through higher order thinking skills. Journal of Instructional Research, 6, 29-34.

Stemler, S. E. (2004). A comparison of consensus, consistency, and measurement approaches to estimating inter-rater reliability. Practical Assessment, Research, and Evaluation, 9(4), 1-11. https://doi.org/10.7275/96jp-xz07.

Suryadi, B. (2018). The Implementation of HOTS problems model in national examinations need to be balanced with improvement of teacher and student ability [in Bahasa]. Badan Standar Nasional Pendidikan. Retrieved from http://bsnp-indonesia.org/2018/04/21/penerapan-soal-model-hots-dalam-ujian-nasional-perlu-diimbangi-dengan-peningkatan-kemampuan-guru-dan-siswa/.

Sydoruk, P. D. (2018). An Analysis of the Higher Order Thinking Requirements of a Grade 8 Online-Based English Language Arts Skills Program. Seton Hall University Dissertations and Theses (ETDs). Retrieved from https://scholarship.shu.edu/dissertations/2495.

Tanujaya, B., Mumu, J., & Margono, G. (2017). The relationship between higher-order thinking skills and academic performance of student in mathematics instruction. International Education Studies, 10(11), 78-85. https://doi.org/10.5539/ies.v10n11p78.

Van den Heuvel-Panhuizen, M., & Drijvers, P. (2013). Realistic Mathematics Education. In S. Lerman (Ed.), Encyclopedia of Mathematics Education. Dordrecht, Heidelberg, New York, London: Springer.

Washington State University (WSU). (2001). WSU Critical thinking Rubric. Retrieved from https://assessment.trinity.duke.edu/sites/assessment.trinity.duke.edu/files/page-attachments/WashingtonStateUniversityCriticalThinkingProjectResourceGuide_000.pdf.

Werang, B. R. (2018). The effect of workload, individual characteristics, and school climate on teachers' emotional exhaustion in elementary schools of Papua. Cakrawala Pendidikan, 37(3), 457-469. https://doi.org/10.21831/cp.v38i3.20635.

Wijaya, A. (2016). Students’ information literacy: A perspective from mathematical literacy. Journal on Mathematics Education, 7(2), 73-82. https://doi.org/10.22342/jme.7.2.3532.73-82.

Wijaya, A. (2018). How do open-ended problems promote mathematical creativity? A reflection of bare mathematics problem and contextual problem. Journal of Physics: Conference Series, 983(1), 012114. https://doi.org/10.1088/1742-6596/983/1/012114.

Wolgast, A., & Fischer, N. (2017). You are not alone: colleague support and goal-oriented cooperation as resources to reduce teachers’ stress. Social Psychology of Education, 20(1), 97-114. https://doi.org/10.1007/s11218-017-9366-1.

DOI: https://doi.org/10.22342/jme.11.2.11000.277-300


  • There are currently no refbacks.

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Journal on Mathematics Education
Doctoral Program on Mathematics Education
Faculty of Teacher Training and Education, Universitas Sriwijaya
Kampus FKIP Bukit Besar
Jl. Srijaya Negara, Bukit Besar
Palembang - 30139
email: jme@unsri.ac.id

p-ISSN: 2087-8885 | e-ISSN: 2407-0610
Journal on Mathematics Education is licensed under a Creative Commons Attribution 4.0 International License

View My Stats