Investigating an Assessment Design that Prevents Students from Using ChatGPT as the Sole Basis to Pass Assessment at the Tertiary Level
Michael Stack 
Issue: Vol.4 No.12 Special Issue Article 6 pp.64 -77
DOI: https://doi.org/10.38159/ehass.20234127 | Published online 30th November, 2023
© 2023 The Author(s). This is an open access article under the CCBY license (http://creativecommons.org/licenses/by/4.0/).
This paper investigated how to design undergraduate assessments that could not be adequately answered by ChatGPT. The paper was embedded in the revised Bloom’s Taxonomy as a theoretical model. ChatGPT has presented significant challenges to lecturers when setting assessments at the tertiary level. There is enormous potential for students to attempt to use ChatGPT to write and pass assessments. Design Based Research formed the basis of this paper’s research design. Qualitative research methods were used to interpret recordings of interaction with ChatGPT during Hermeneutic research cycles. The paper found that it was possible to design an assessment that could not be satisfactorily answered solely by ChatGPT. Interactions with ChatGPT were found to be an essential aspect of the research process. In light of the findings, efforts should be made to revise undergraduate assessments to encourage students to engage with them rather than copy and paste from ChatGPT. The paper recommended ChatGPT as a useful tool or form of interactive Google that can support writing assessment but is unable to replace the student effectively. The paper suggests that students should receive training on the limitations of ChatGPT and how they can use it more effectively in their studies.
Keywords: Assessment Design, ChatGPT, Higher Education, Bloom’s Taxonomy
Allam, Hesham, Juan Dempere, Vishwesh Akre, Divya Parakash, Noman Mazher, and Jinesh Ahamed. “Artificial Intelligence in Education: An Argument of Chat-GPT Use in Education.” In 2023 9th International Conference on Information Technology Trends (ITT), 151–56. IEEE, 2023.
Baidoo-Anu, David, and Leticia Owusu Ansah. “Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning.” Journal of AI 7, no. 1 (2023): 52–62.
Bishop, Lea. “A Computer Wrote This Paper: What Chatgpt Means for Education, Research, and Writing.” Research, and Writing (January 26, 2023), 2023.
Cascella, Marco, Jonathan Montomoli, Valentina Bellini, and Elena Bignami. “Evaluating the Feasibility of ChatGPT in Healthcare: An Analysis of Multiple Clinical and Research Scenarios.” Journal of Medical Systems 47, no. 1 (2023): 33.
Castillo, Alejandro Guadalupe Rincón, Giovanna Jackeline Serna Silva, Javier Pedro Flores Arocutipa, Haydeé Quispe Berrios, Marco Antonio Marcos Rodriguez, Guillermo Yanowsky Reyes, Hugo Ricardo Prado Lopez, Rosa Marina Vera Teves, Herbert Victor Huaranga Rivera, and José Luis Arias-Gonzáles. “Effect of Chat GPT on the Digitized Learning Process of University Students.” Journal of Namibian Studies: History Politics Culture 33 (2023): 1–15.
Chaudhry, Iffat Sabir, Sayed Ahmad M Sarwary, Ghaleb A El Refae, and Habib Chabchoub. “Time to Revisit Existing Student’s Performance Evaluation Approach in Higher Education Sector in a New Era of ChatGPT—A Case Study.” Cogent Education 10, no. 1 (2023): 2210461.
Crawford, Joseph, Michael Cowling, and Kelly-Ann Allen. “Leadership Is Needed for Ethical ChatGPT: Character, Assessment, and Learning Using Artificial Intelligence (AI).” Journal of University Teaching & Learning Practice 20, no. 3 (2023): 02.
Creswell, John W, and J David Creswell. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage publications, 2017.
Dale, Robert. “GPT-3: What’s It Good for?” Natural Language Engineering 27, no. 1 (2021): 113–18.
Day, Terence. “A Preliminary Investigation of Fake Peer-Reviewed Citations and References Generated by ChatGPT.” The Professional Geographer, 2023, 1–4.
Department of Basic Education. CAPS History Grade 10 – 12. South Africa: Department of Education, 2011.
———. CAPS Social Sciences Grade 4-6. South Africa: Department of Education, 2011.
———. CAPS Social Sciences Grade 7-9. South Africa: Department of Education, 2011.
Fuchs, Kevin. “Exploring the Opportunities and Challenges of NLP Models in Higher Education: Is Chat GPT a Blessing or a Curse?” In Frontiers in Education, 8:1–4. Frontiers, 2023.
Gul, Rani, Shazia Kanwal, and Sadia Suleman Khan. “Preferences of the Teachers in Employing Revised Blooms Taxonomy in Their Instructions.” Sjesr 3, no. 2 (2020): 258–66.
Hariri, Walid. “Unlocking the Potential of ChatGPT: A Comprehensive Exploration of Its Applications, Advantages, Limitations, and Future Directions in Natural Language Processing.” ArXiv Preprint ArXiv:2304.02017, 2023. https://doi.org/http://dx.doi.org/10.13140/RG.2.2.23433.11360/3.
Kivunja, Charles, and Ahmed Bawa Kuyini. “Understanding and Applying Research Paradigms in Educational Contexts.” International Journal of Higher Education 6, no. 5 (2017): 26–41.
Konstantis, Konstantinos, Antonios Georgas, Antonis Faras, Konstantinos Georgas, and Aristotle Tympas. “Ethical Considerations in Working with ChatGPT on a Questionnaire about the Future of Work with ChatGPT.” AI and Ethics, June 20, 2023. https://doi.org/10.1007/s43681-023-00312-6.
Obilor, Esezi Isaac. “Convenience and Purposive Sampling Techniques: Are They the Same.” International Journal of Innovative Social & Science Education Research 11, no. 1 (2023): 1–7.
OpenAI, ChatGPT to Michael Stack, Output, August 2023. https://chat.openai.com/c/bd67e062-b307-4e6f-ba2b-3939ad1d5c24
OpenAI, ChatGPT to Michael Stack, Output, August 2023. https://chat.openai.com/c/7502475c-f759-4483-99ce-65bf07c13ccf
OpenAI, ChatGPT to Michael Stack, Output, August 2023. https://chat.openai.com/c/d78e778d-c9cf-458d-934f-0cb2dad5d51f
OpenAI, ChatGPT to Michael Stack, Output, August 2023. https://chat.openai.com/c/a2ae4a94-7710-4630-995a-3f5485347a03
OpenAI, ChatGPT to Michael Stack, Output, August 2023. https://chat.openai.com/c/168d7d11-81a9-47d6-acdf-74273752ffa4
OpenAI, ChatGPT to Michael Stack, Output, August 2023. https://chat.openai.com/c/20a3a02f-62d4-456c-a158-a21b16558fab
OpenAI, ChatGPT to Michael Stack, Output, August 2023. https://chat.openai.com/c/750ece33-7c36-488a-8bdd-43db76f9aa04
OpenAI, ChatGPT to Michael Stack, Output, August 2023. https://chat.openai.com/c/a551faa2-1795-44e0-91de-617069b00f47
OpenAI, ChatGPT to Michael Stack, Output, July 2023. https://chat.openai.com/c/54596ce8-13fe-43b0-9e95-6374a4cfa67e
Rudolph, Jürgen, Samson Tan, and Shannon Tan. “ChatGPT: Bullshit Spewer or the End of Traditional Assessments in Higher Education?” Journal of Applied Learning and Teaching 6, no. 1 (2023).
Scott, Emily E., Mary Pat Wenderoth, and Jennifer H. Doherty. “Design-Based Research: A Methodology to Extend and Enrich Biology Education Research.” CBE—Life Sciences Education 19, no. 3 (September 2020): es11. https://doi.org/10.1187/cbe.19-11-0245.
Sobral, Sónia Rolland. “Bloom’s Taxonomy to Improve Teaching-Learning in Introduction to Programming.” International Journal of Information and Education Technology 11, no. 3 (2021): 148–53.
Strzelecki, Artur. “To Use or Not to Use ChatGPT in Higher Education? A Study of Students’ Acceptance and Use of Technology.” Interactive Learning Environments, May 8, 2023, 1–14. https://doi.org/10.1080/10494820.2023.2209881.
Susnjak, Teo. “ChatGPT: The End of Online Exam Integrity?” ArXiv Preprint ArXiv:2212.09292, 2022. https://doi.org/http://dx.doi.org/10.48550/arXiv.2212.09292.
Thakkar, K, and N Jagdishbhai. “Exploring the Capabilities and Limitations of GPT and Chat GPT in Natural Language Processing.” J. Manag. Res. Anal., 2023.
Zhu, Gaoxia, Xiuyi Fan, Chenyu Hou, Tianlong Zhong, Peter Seow, Annabel Chen Shen-Hsing, Preman Rajalingam, Low Kin Yew, and Tan Lay Poh. “Embrace Opportunities and Face Challenges: Using ChatGPT in Undergraduate Students’ Collaborative Interdisciplinary Learning.” ArXiv Preprint ArXiv:2305.18616, 2023.
Michael Stack is a Lecturer in the Department of Social Sciences and Commerce Education at the University of the Free State in South Africa. He received a Bachelor of Arts Honours in Historical Studies from the University of Cape and two master’s degrees from Stellenbosch University, in Curriculum Studies and Historical Studies. He is currently pursuing his PhD studies at the University of the Free State. His research interests include Game Based Learning, Gamification and ICT integration in Education.
Stack, Michael. “Investigating an Assessment Design that Prevents Students from Using ChatGPT as the Sole Basis to Pass Assessment at the Tertiary Level.” E-Journal of Humanities, Arts and Social Sciences 4, no.12 Special Issue (2023): 64 -77. https://doi.org/10.38159/ehass.20234127
© 2023 The Author(s). Published and Maintained by Noyam Journals. This is an open access article under the CCBY license (http://creativecommons.org/licenses/by/4.0/).
