Exploring the pedagogical uses of AI chatbots

Accordingly, chatbots popularized by social media and MIM applications have been widely accepted (Rahman et al., 2018; Smutny & Schreiberova, 2020) and referred to as mobile-based chatbots. Nevertheless, given the possibilities of MIM in conceptualizing an ideal learning environment, we often overlook if instructors are capable of engaging in high-demand learning activities, especially around the clock (Kumar & Silva, 2020). Chatbots can potentially be a solution to such a barrier (Schmulian & Coetzee, 2019), especially by automatically supporting learning communication and interactions (Eeuwen, 2017; Garcia Brustenga et al., 2018) for even a large number of students.

Only one study pointed to high usefulness and subjective satisfaction (Lee et al., 2020), while the others reported low to moderate subjective satisfaction (Table 13). For instance, the chatbot presented in (Lee et al., 2020) aims to increase learning effectiveness by allowing students to ask questions related to the course materials. It turned out that most of the participants agreed that the chatbot is a valuable educational tool that facilitates real-time problem solving and provides a quick recap on course material. The study mentioned in (Mendez et al., 2020) conducted two focus groups to evaluate the efficacy of chatbot used for academic advising. While students were largely satisfied with the answers given by the chatbot, they thought it lacked personalization and the human touch of real academic advisors. Finally, the chatbot discussed by (Verleger & Pembridge, 2018) was built upon a Q&A database related to a programming course.

Meanwhile, North Korea, China, and Russia, in particular, contended that the U.S. might employ ChatGPT for disseminating misinformation. Conversely, OpenAI restricts access to ChatGPT in certain countries, such as Afghanistan and Iran, citing geopolitical constraints, legal considerations, data protection regulations, and internet accessibility as the basis for this decision. Italy became the first Western country to ban ChatGPT (Browne, 2023) after the country’s data protection authority called on OpenAI to stop processing Italian residents’ data. They claimed that ChatGPT did not comply with the European General Data Protection Regulation. However, after OpenAI clarified the data privacy issues with Italian data protection authority, ChatGPT returned to Italy.

Enhanced student engagement through chatbot interactions

The value was determined by looking at the search results in detail using several queries to exclude as few relevant works as possible. This approach showed promising results and, at the same time, did not burden the literature list with irrelevant items. Assignment of coded research topics identified in related literature reviews to research categories.

Chatbots can provide virtual tutoring and mentoring services, guiding students through coursework, assignments, and career advice. They can supplement the support offered by faculty members and academic advisors. Two recent articles in the journal Nature described its application to weather chatbot in education forecasting. Currently, it is difficult and time-consuming because to make predictions, meteorologists must analyze weather variables such as temperature, precipitation, pressure, wind, humidity, and cloudiness individually, but new AI systems can significantly speed up the process.

Lack of Emotional Intelligence

Making up a quarter of all publications, Efficiency of Education is the second most popular objective (25%), while addressing Students’ Motivation and Availability of Education are third (13%) and fourth (11%), respectively. Other objectives also make up a substantial amount of these publications (19%), although they were too diverse to categorize in a uniform way. Examples of these are inclusivity (Heo and Lee, 2019) or the promotion of student teacher interactions (Mendoza et al., 2020). Applying the curve presented in Figure 6 to Gartner’s Hype Cycle (Linden and Fenn, 2003) suggests that technology around chatbots in education may currently be in the “Innovation Trigger” phase. This phase is where many expectations are placed on the technology, but the practical in-depth experience is still largely lacking. At last, we could have missed articles that report an educational chatbot that could not be found in the selected search databases.

One of the takeaways is that the emerging field around educational chatbots has seen much activity in the last two years. Based on the experience of this preliminary search, search terms, queries, and filters were constructed for the actual structured literature review. This structured literature review follows the PRISMA framework (Liberati et al., 2009), a guideline for reporting systematic reviews and meta-analyses. The framework consists of an elaborated structure for systematic literature reviews and sets requirements for reporting information about the review process (see section 3.2 to 3.4).

Chatbots are digital systems that can be interacted with entirely through natural language via text or voice interfaces. They are intended to automate conversations by simulating a human conversation partner and can be integrated into software, such as online platforms, digital assistants, or be interfaced through messaging services. There is also a bias towards empirically evaluated articles as we only selected articles that have an empirical evaluation, such as experiments, evaluation studies, etc. Further, we only analyzed the most recent articles when many articles discussed the same concept by the same researchers. Since different researchers with diverse research experience participated in this study, article classification may have been somewhat inaccurate.

Such a strategy was used to ensure that the instructor could guide the students the next day if there were any issues. Qualitative data were collected through class discussions and assessment reports of the AICS following a template provided through the Moodle platform. During the 1-month intervention period in each educational setting, participants independently completed the assessment reports. They were instructed to provide personal feedback on their interaction with each AIC, using the template to note both positive and negative aspects. Additionally, they were asked to attach 12 screenshots illustrating their interaction, three with each AIC, to support their assessment. QDA Miner Software was used for textual analysis of students’ written evaluations on each AIC, adhering to a provided template.

These AI-driven programs, tailored for educational settings, aim to provide enriched learning experiences. It’s incredible, but chatbots have been used in education since the early 1970s. A chatbot in the education industry is an AI-powered virtual assistant designed to interact with students, teachers, and other stakeholders in the educational ecosystem. Using advanced Conversational AI and Generative AI technologies, chatbots can engage in natural language conversations, providing personalized support and delivering relevant information on various educational topics. Chatbots in education offer unparalleled accessibility, functioning as reliable virtual assistants that remain accessible around the clock.

Participants

Therefore, looking at our results and the challenges presented, we conclude, “No, we are not there yet! ” – There is still much to be done in terms of research on chatbots in education. Still, development in this area seems to have just begun to gain momentum and we expect to see new insights in the coming years. The teaching agents presented in the different studies used various approaches. For instance, some teaching agents recommended tutorials to students based upon learning styles (Redondo-Hernández & Pérez-Marín, 2011), students’ historical learning (Coronado et al., 2018), and pattern matching (Ondáš et al., 2019).

Begin by telling the chatbot that you would like to develop a fictional short story and that you’d like its assistance in developing your ideas. Try different ways of interacting and responding to the chatbot to get a sense of its capabilities. ChatGPT, developed by OpenAI, uses the Generative Pre-training Transformer (GPT) large language model.

After coding a larger set of publications, it became clear that the code for service-oriented chatbots needed to be further distinguished. This was because it summarized e.g. automation activities with activities related to self-regulated learning and thus could not be distinguished sharply enough from the learning role. After refining the code set in the next iteration into a learning role, an assistance role, and a mentoring role, it was then possible to ensure the separation of the individual codes. Research in this area has recently focused on chatbot technology, a subtype of dialog systems, as several technological platforms have matured and led to applications in various domains. Chatbots incorporate generic language models extracted from large parts of the Internet and enable feedback by limiting themselves to text or voice interfaces. For this reason, they have also been proposed and researched for a variety of applications in education (Winkler and Soellner, 2018).

Each has some unique characteristics and nuanced differences in how developers built and trained them, though these differences are not significant for our purposes as educators. We encourage you to try accessing these chatbots as you explore their capabilities. The authors declare that this research paper did not receive any funding from external organizations.

When it comes to education-related applications of AI, the media have paid the most attention to applications like students getting chatbots to compose their essays and term papers. You can foun additiona information about ai customer service and artificial intelligence and NLP. By looking at other relations in more detail, there is surprisingly no relation between Skill Improvement as the most common implementation objective and Assisting, as the 2nd most common pedagogical role. Furthermore, it can be observed that the Mentoring role has nearly equal relations to all of the objectives for implementing chatbots. Given these results, we can summarize four major implementing objectives for chatbots. Of these, Skill Improvement is the most popular objective, constituting around one-third of publications (32%).

With SAT/ACT test score usage waning in many admissions sectors, the narrative portions of college applications may receive additional emphasis in evaluation of merit and deservingness. This was our worry when we found the content of admission essays to be more strongly correlated with income than are SAT scores. Almost all institutions aim to streamline their processes of updating and collecting data. By leveraging AI technology, colleges can efficiently gather and store information. Such optimization will eliminate student involvement in updating their details. As a rule, this advanced data collection system enhances administrative efficiency and enables institutions to use pupils’ information as necessary.

These AI-driven tools create an inclusive studying environment by catering to diverse educational styles and abilities. They offer adaptable content formats, such as audio, visual, and Chat PG text-based materials, ensuring accessibility for all users, regardless of their needs. In 2023, AI chatbots are transforming the education industry with their versatile applications.

The results of the evaluation studies (Table 12) point to various findings such as increased motivation, learning, task completeness, and high subjective satisfaction and engagement. In terms of the medium of interaction, chatbots can be text-based, voice-based, and embodied. Text-based agents allow users to interact by simply typing via a keyboard, whereas voice-based agents allow talking via a mic. Voice-based chatbots are more accessible to older adults and some special-need people (Brewer et al., 2018). An embodied chatbot has a physical body, usually in the form of a human, or a cartoon animal (Serenko et al., 2007), allowing them to exhibit facial expressions and emotions. The chatbot used pattern matching to emulate a psychotherapist conversing with a human patient.

In 2016, Bill Gates has announced that the Bill and Melissa Gates Foundation will invest more than $240 million dollars in a tech project. Facebook has also followed the Bill Gates’s example and joined the world-famous Summit Learning project. One of the biggest breakthroughs in the development of artificial intelligence and natural language procession happened when Georgetown University and IBM joined their forces and presented the first demonstration of machine translation.

This allows the teacher to tweak the chatbot’s design to improve the experience. Equally if not more importantly, it can reveal gaps in knowledge or flawed assumptions the learners hold, which can inform the design of new learning experiences — chatbot-mediated or not. Tutoring, which focuses on skill-building in small groups or one-on-one settings, can benefit learning (Kraft, Schueler, Loeb, & Robinson, 2021). Effective tutors may use questioning techniques, collaborative problem-solving, and personalized instruction to support their students. While Stanford provides a range of tutoring services, not all students use them regularly; students might use AI chatbots as a supplement to tutoring services.

• An AI chatbot might help you by giving students frequent, immediate, and adaptive feedback.
• If students end up being confused and unclear about the topic, all the efforts made by the teachers go in vain.
• It was a great opportunity to be creative and figure out how to activate in-context learning, taking advantage of the unique spaces where the students were, and the wide world out there.

By harnessing the power of generative AI, chatbots can efficiently handle a multitude of conversations with students simultaneously. The technology’s ability to generate human-like responses in real-time allows these AI chatbots to engage with numerous students without compromising the quality of their interactions. This scalability ensures that every learner receives prompt and personalized support, no matter how many students are using the chatbot at the same time.

When you think of advancements in technology, edtech might not be the first thing that pops into your head. But during the COVID-19 pandemic, edtech became a true lifeline for education by making it accessible and easy to use despite there being numerous physical restrictions. Today, technologies like conversational AI and natural language processing (NLP) continue to help educators and students world over teach and learn better.

The Peril and Promise of Chatbots in Education

Looking ahead, allowing students to select specific design aspects of AICs, similar to choosing linguistic features such as target level or accent, could be a crucial step in creating a more adaptive and personalized learning experience. The CHISM results, particularly in the Language Experience (LEX) dimension, revealed significant insights about the teacher candidates’ perceptions of the four evaluated chatbots. When examining why none of the AICs achieved moderate satisfaction in the LEX dimension, it is crucial to consider each AIC’s design and target audience limitations, as pointed out in previous research (Gokturk, 2017; Hajizadeh, 2023). For instance, Mondly’s reliance on pre-programmed responses and Buddy.ai’s focus on repetitive drills for children limit dynamic conversation, resulting in lower satisfaction in maintaining contextually relevant dialogues.

The study was conducted independently and without financial support from any source. The authors have no financial interests or affiliations that could have influenced the design, execution, analysis, or reporting of the research. It’s important to note that some papers raise concerns about excessive reliance on AI-generated information, potentially leading to a negative impact on student’s critical thinking and problem-solving skills (Kasneci et al., 2023). For instance, if students consistently receive solutions or information effortlessly through AI assistance, they might not engage deeply in understanding the topic.

This can be achieved by making information more easily available (Sugondo and Bahana, 2019) or by simplifying processes through the chatbot’s automation (Suwannatee and Suwanyangyuen, 2019). An example of this is the chatbot in (Sandoval, 2018) that answers general questions about a course, such as an exam date or office hours. https://chat.openai.com/ Most articles (13; 36.11%) used an experiment to establish the validity of the used approach, while 10 articles (27.77%) used an evaluation study to validate the usefulness and usability of their approach. The remaining articles used a questionnaire (10; 27.7%) and a focus group (3; 8.22%) as their evaluation methods.

Such a contribution also offers networking opportunities and support for current students. Additionally, this will positively impact the brand image, attracting potential applicants and stakeholders. Overloaded due to tight scheduling and plenty of daily duties, educators often face challenges. Invaluable teaching assistants can give a hand with automation tasks like tests, assessments, and assignment tracking. EdWeek reports that, according to Impact Research, nearly 50% of teachers utilized ChatGPT for lesson planning and generated creative ideas for their classes. Education reaches far beyond the classroom, requiring guidance and support across the entire campus life.

The comprehensive list of included studies, along with relevant data extracted from these studies, is available from the corresponding author upon request. The datasets generated and/or analysed during the current study are not publicly available due privacy reasons but are available from the corresponding author on reasonable request. The American Council on Science and Health is a research and education organization operating under Section 501(c)(3) of the Internal Revenue Code.

Roughly 92% of students worldwide demonstrate a desire for personalized assistance and updates concerning their academic advancement. By analyzing pupils’ learning patterns, these tools customize content and training paths. Such a unique approach ensures that everyone receives tailored support, promoting better comprehension and knowledge retention. Although chatbots can provide information, they should not act as a substitute for, instead of spurring the development of students’ critical thinking and analytical skills. Universities need to emphasize the importance of independent research, critical evaluation, and synthesis of knowledge.

Subsequently, the assessment of specific topics is presented where the user is expected to fill out values, and the chatbot responds with feedback. The level of the assessment becomes more challenging as the student makes progress. A slightly different interaction is explained in (Winkler et al., 2020), where the chatbot challenges the students with a question. If they answer incorrectly, they are explained why the answer is incorrect and then get asked a scaffolding question. The proliferation of smartphones in the late 2000s led to the integration of educational chatbots into mobile applications.

Considering this, the University of Murcia in Spain used an AI chat assistant that successfully addressed more than 38,708 inquiries with an accuracy rate of 91%. Educational chatbots serve as personal assistants, offering individual guidance to everyone. Through intelligent tutoring systems, these models analyze responses, learning patterns, and overall performance, fostering tailored teaching. Bots are particularly beneficial for neurodivergent people, as they address individual comprehension disabilities and adapt study plans accordingly. AI systems may lack the emotional understanding and sensitivity required for dealing with complex sentimental concerns. In educational establishments where mental support is essential, the absence of sensitive intelligence in chatbots can limit their effectiveness in addressing users’ personal needs.

Furthermore, as there is a triangulated relationship between these outcomes, the author speculates that these outcomes were justified, especially with the small sample size used, as Rosenstein (2019) explained. Conversely, Garcia Brustenga et al. (2018) categorized ECs based on eight tasks in the educational context as described in Table 1. Correspondingly, these tasks reflect that ECs may be potentially beneficial in fulfilling the three learning domains by providing a platform for information retrieval, emotional and motivational support, and skills development. The selection of the four AICs, namely Mondly, Andy, John Bot, and Buddy.ai, was guided by specific criteria, including multiplatform compatibility, wide availability, and diverse functionalities such as the integration of different technologies. These AICs offered a wide range of options, such as catering to different English language proficiency levels, providing personalized feedback, adapting to individual learning progress, and incorporating other technologies (AR, VR) in some cases.

Chatbots deployed through MIM applications are simplistic bots known as messenger bots (Schmulian & Coetzee, 2019). These platforms, such as Facebook, WhatsApp, and Telegram, have largely introduced chatbots to facilitate automatic around-the-clock interaction and communication, primarily focusing on the service industries. Even though MIM applications were not intended for pedagogical use, but due to affordance and their undemanding role in facilitating communication, they have established themselves as a learning platform (Kumar et al., 2020; Pereira et al., 2019).

With artificial intelligence, the complete process of enrollment and admissions can be smoother and more streamlined. Administrators can take up other complex, time-consuming tasks that need human attention. From teachers to syllabus, admissions to hygiene, schools can collect information on all the aspects and become champions in their sector. Users should provide feedback to OpenAI, Google, and other relevant creators and stakeholders regarding any concerns or issues they encounter while using chatbots. Reporting any instances of misuse or ethical violations will help to improve the system and its guidelines.

Furthermore, there are also limited studies in strategies that can be used to improvise ECs role as an engaging pedagogical communication agent (Chaves & Gerosa, 2021). Besides, it was stipulated that students’ expectations and the current reality of simplistic bots may not be aligned as Miller (2016) claims that ANI’s limitation has delimited chatbots towards a simplistic menu prompt interaction. According to Kumar and Silva (2020), acceptance, facilities, and skills are still are a significant challenge to students and instructors. Similarly, designing and adapting chatbots into existing learning systems is often taxing (Luo & Gonda, 2019) as instructors sometimes have limited competencies and strategic options in fulfilling EC pedagogical needs (Sandoval, 2018). Moreover, the complexity of designing and capturing all scenarios of how a user might engage with a chatbot also creates frustrations in interaction as expectations may not always be met for both parties (Brandtzaeg & Følstad, 2018). Hence, while ECs as conversational agents may have been projected to substitute learning platforms in the future (Følstad & Brandtzaeg, 2017), much is still to be explored from stakeholders’ viewpoint in facilitating such intervention.

In this study, we carefully look at the interaction style in terms of who is in control of the conversation, i.e., the chatbot or the user. Hobert and Meyer von Wolff (2019), Pérez et al. (2020), and Hwang and Chang (2021) examined the evaluation methods used to assess the effectiveness of educational chatbots. The authors identified that several evaluation methods such as surveys, experiments, and evaluation studies measure acceptance, motivation, and usability.

Secondly, we study selected articles and synthesize results and lastly, we report and discuss the findings. Subsequently, we delve into the methodology, encompassing aspects such as research questions, the search process, inclusion and exclusion criteria, as well as the data extraction strategy. Moving on, we present a comprehensive analysis of the results in the subsequent section. Finally, we conclude by addressing the limitations encountered during the study and offering insights into potential future research directions.

The data that support the findings of this study are available from the corresponding author upon reasonable request. Georgia State University has effectively implemented a personalized communication system. They introduced Pounce, a bespoke smart assistant created to actively engage admitted students.

Exceptionally, a chatbot found in (D’mello & Graesser, 2013) is both a teaching and motivational agent. In general, most desktop-based chatbots were built in or before 2013, probably because desktop-based systems are cumbersome to modern users as they must be downloaded and installed, need frequent updates, and are dependent on operating systems. Unsurprisingly, most chatbots were web-based, probably because the web-based applications are operating system independent, do not require downloading, installing, or updating.

They can offer learners the possibility to engage in simulated conversational interactions in a non-judgmental environment (El Shazly, 2021; Skjuve et al., 2021). For these reasons, chatbots are being increasingly used as virtual tutors to facilitate the development of language skills and communicative competence in the target language (Huang et al., 2022; Hwang & Chang, 2021; Zhang et al., 2023). Navigating the expansive world of educational chatbots reveals a realm where technology meets academia, fostering student engagement, and offering support.

Chatbots can facilitate online discussions, group projects, and collaborative learning experiences, allowing students to engage with peers and share ideas, fostering community and active participation. Chatbots’ ease of use and ability to rapidly create human-like text, including everything from reports, essays, and recipes to computer code, ensure that the AI revolution will be a powerful tool for students at every level to improve their capabilities and expertise. But, like most powerful technologies, the use of chatbots offers challenges as well as opportunities. By grouping the resulting relevant publications according to their date of publication, it is apparent that chatbots in education are currently in a phase of increased attention. The release distribution shows slightly lower publication numbers in the current than in the previous year (Figure 6), which could be attributed to a time lag between the actual publication of manuscripts and their dissemination in databases.

Users should stay informed about the latest developments and best practices in AI ethics. They should strive to understand the limitations and capabilities of chatbots and contribute to the responsible and ethical use of AI technologies. Users are responsible for how they use the content generated by chatbots when interacting with it. They should ensure that the information they provide and how they use the model aligns with ethical standards and legal obligations. Chatbots’ expertise is based on the training data it has received (although they do have the ability to “learn” with exposure to new information), and they may not possess the depth of knowledge in specialized or niche areas. In such cases, subject matter experts should be consulted for accurate and comprehensive information.

Essays offer much better insight into a student’s level of knowledge, methodology, and problem-solving skill, but they are much harder to grade and assess. Ashok Goel, a computer science professor at Georgia Tech, is one of the first teachers to simplify his work in this way, with the help of artificial intelligence. The bot answers students’ questions on an online forum and provides technical information about courses and lectures. Moreover, individual personality traits such as motivation have also been found to influence creativity (van Knippenberg & Hirst, 2020) which indirectly influenced the need for cognition (Pan et al., 2020). Nevertheless, these nonsignificant findings may have some interesting contribution as it implies that project-based learning tends to improve these personality-based learning outcomes. At the same time, the introduction of ECs did not create cognitive barriers that would have affected the cognition, motivational and creative processes involved in project-based learning.

Drawing from extensive systematic literature reviews, as summarized in Table 1, AI chatbots possess the potential to profoundly influence diverse aspects of education. However, it is essential to address concerns regarding the irrational use of technology and the challenges that education systems encounter while striving to harness its capacity and make the best use of it. Three categories of research gaps were identified from empirical findings (i) learning outcomes, (ii) design issues, and (iii) assessment and testing issues. EC studies have primarily focused on language learning, programming, and health courses, implying that EC application and the investigation of learning outcomes have not been investigated in various educational domains and levels of education.

The aim was not to compare the four AICs, but rather to present teacher candidates with a broad overview of these virtual tutors, providing a variety of options and examples. Some studies have emphasized that interactions with AICs can seem detached and lack the human element (Rapp et al., 2021). Additionally, while AICs can handle a wide range of queries, they may struggle with complex language nuances, which could potentially lead to misunderstandings or incorrect language usage. It has also been observed that some students’ interest dwindled after the initial period of engagement due to repetitive conversation patterns and redundancies, making the interaction less natural compared to student–teacher exchanges (Fryer et al., 2019). AI chatbots for education offer backup throughout university life, from the admission process to post-course assistance. They act beyond classroom activities as campus guides, providing valuable information on facilities and helping students.

Examples of these are chatbots simulating a virtual pen pal abroad (Na-Young, 2019). Conversations with this kind of chatbot aim to motivate the students to look up vocabulary, check their grammar, and gain confidence in the foreign language. To understand and underline the current need for research in the use of chatbots in education, we first examined the existing literature, focusing on comprehensive literature reviews. By looking at research questions in these literature reviews, we identified 21 different research topics and extracted findings accordingly.

Most importantly, chatbots played a critical role in the education field, in which most researchers (12 articles; 33.33%) developed chatbots used to teach computer science topics (Fig. 4). Chatbots, also known as conversational agents, enable the interaction of humans with computers through natural language, by applying the technology of natural language processing (NLP) (Bradeško & Mladenić, 2012). In fact, the size of the chatbot market worldwide is expected to be 1.23 billion dollars in 2025 (Kaczorowska-Spychalska, 2019). In the US alone, the chatbot industry was valued at 113 million US dollars and is expected to reach 994.5 million US dollars in 2024 Footnote 1. I do not see chatbots as a replacement for the teacher, but as one more tool in their toolbox, or a new medium that can be used to design learning experiences in a way that extends the capacity and unique abilities of the teacher. In addition, the responses of the learner not only determine the chatbot’s responses, but provide data for the teacher to get to know the learner better.

Nevertheless, because the tool did not produce answers to some questions, some students decided to abandon it and instead use standard search engines to find answers. This paper will help to better understand how educational chatbots can be effectively utilized to enhance education and address the specific needs and challenges of students and educators. As a digital assistant, the EC was designed to aid in managing the team-based project where it was intended to communicate with students to inquire about challenges and provide support and guidance in completing their tasks. According to Cunningham-Nelson et al. (2019), such a role improves academic performance as students prioritize such needs. Therefore, supporting the outcome of this study that observed that the EC groups learning performance and teamwork outcome had a more significant effect size than the CT group.