The Use of an AI-Powered Speaking Tool (SKY) in Higher Education: A Mixed-Methods Study of Learner Engagement and Self-Efficacy [Abstract]
InSITE 2025
• 2025
• pp. 30
Aim/Purpose
This mixed-methods study explores the relationships between the use of an AI-based application for second language learning in higher education and students’ Self-Efficacy, Fear of AI Challenges, Self-Image, and engagement-related indicators.
Background
Despite the growing presence of AI in education, its integration into second language instruction in academic settings is still limited. This study is unique in developing a dedicated AI tool and examining its connection to key learner measures using both quantitative and qualitative methods.
Methodology
580 students completed the pre-course questionnaire and 647 completed the post-course questionnaire. All were enrolled in CEFR-aligned English courses (Pre-Basic 1 to Advanced). As part of the coursework, students used SKY, an AI-powered conversation app via WhatsApp, designed to support speaking—the most difficult skill to teach in second language instruction. Students interacted with SKY three times a week for 15 minutes per session. Quantitative analysis used paired T-tests and regression models; qualitative findings were drawn from open-ended responses.
Contribution
The study introduces SKY as a new AI-based tool and provides data on how its use relates to learners’ confidence, emotions, and engagement.
Findings
Significant increases were found in all Self-Efficacy measures (p < .05). Fear of AI Challenges decreased, particularly in the “Intimidating” and “Unpleasant Feeling” categories, while “Uplifting Challenge” scores rose slightly. Some Self-Image indicators improved. Higher Self-Efficacy was linked to lower fear and greater engagement (Participation, Interaction, Wordy Talk). Qualitative responses confirmed these trends, with students describing SKY as easy, motivating, and flexible.
Recommendations for Practitioners
Integrate AI tools like SKY regularly to encourage speaking practice in low-pressure, accessible formats.
Recommendations for Researchers
Study how specific AI design elements—such as feedback tone and personalization—affect learning experiences.
Impact on Society
AI tools like SKY can promote accessible, inclusive speaking practice and reduce anxiety in academic language learning.
Future Research
Future studies could examine how different types of AI feedback (supportive vs. corrective) influence Self-Efficacy and willingness to speak.
This mixed-methods study explores the relationships between the use of an AI-based application for second language learning in higher education and students’ Self-Efficacy, Fear of AI Challenges, Self-Image, and engagement-related indicators.
Background
Despite the growing presence of AI in education, its integration into second language instruction in academic settings is still limited. This study is unique in developing a dedicated AI tool and examining its connection to key learner measures using both quantitative and qualitative methods.
Methodology
580 students completed the pre-course questionnaire and 647 completed the post-course questionnaire. All were enrolled in CEFR-aligned English courses (Pre-Basic 1 to Advanced). As part of the coursework, students used SKY, an AI-powered conversation app via WhatsApp, designed to support speaking—the most difficult skill to teach in second language instruction. Students interacted with SKY three times a week for 15 minutes per session. Quantitative analysis used paired T-tests and regression models; qualitative findings were drawn from open-ended responses.
Contribution
The study introduces SKY as a new AI-based tool and provides data on how its use relates to learners’ confidence, emotions, and engagement.
Findings
Significant increases were found in all Self-Efficacy measures (p < .05). Fear of AI Challenges decreased, particularly in the “Intimidating” and “Unpleasant Feeling” categories, while “Uplifting Challenge” scores rose slightly. Some Self-Image indicators improved. Higher Self-Efficacy was linked to lower fear and greater engagement (Participation, Interaction, Wordy Talk). Qualitative responses confirmed these trends, with students describing SKY as easy, motivating, and flexible.
Recommendations for Practitioners
Integrate AI tools like SKY regularly to encourage speaking practice in low-pressure, accessible formats.
Recommendations for Researchers
Study how specific AI design elements—such as feedback tone and personalization—affect learning experiences.
Impact on Society
AI tools like SKY can promote accessible, inclusive speaking practice and reduce anxiety in academic language learning.
Future Research
Future studies could examine how different types of AI feedback (supportive vs. corrective) influence Self-Efficacy and willingness to speak.
AI in language learning, second language instruction, higher education, self-efficacy, student engagement
14 total downloads
Back