Response to RQ 1. Does the PT-PBL approach improve student learning motivation compared to the C-PBL approach?
A one-way ANCOVA was used to examine the impact of different learning approaches on students’ learning motivation. The groups using different learning approaches (EG and CG) were considered independent variables, while the learning motivation post-test and pre-test were the dependent and covariate variables, respectively. ANCOVA was chosen because it allows for the control of potential covariates, such as pre-test scores, ensuring a more precise estimation of the intervention effect. This approach reduces error variance and provides a clearer isolation of the effects of the learning approach, offering a reliable basis for comparing post-test outcomes. The assumption of homogeneity of regression coefficients for extrinsic motivation (F = 0.475, p = 0.493) and intrinsic motivation (F = 0.231, p = 0.632) questionnaire scores was confirmed. Table 1 shows the ANCOVA results. Regarding extrinsic motivation, the adjusted means of the EG and CG were 3.57 and 3.28, respectively. Furthermore, there was a significant difference between the EG and CG in extrinsic motivation (F = 5.967, p = 0.018, ƞ2 = 0.092). However, there was no significant difference in intrinsic motivation between the EG and CG.
In addition, students were separated into two subgroups based on the degree of improvement in their motivation (post-test scores minus pre-test scores). Students with an increase in extrinsic motivation equal to or greater than zero were categorized into the Extrinsic Motivation Increase (EM+) subgroup, while the other students were categorized into the Extrinsic Motivation Decrease (EM-) subgroup. Similarly, intrinsic motivation was categorized into the Intrinsic Motivation Increase (IM+) subgroup and Intrinsic Motivation Decrease (IM-) subgroup. In terms of extrinsic motivation, the EM+ subgroup consisted of 25 students (78%) in the EG and 20 students (67%) in the CG. In contrast, the EM- subgroup included 7 students (22%) in the EG and 10 students (33%) in the CG. In terms of intrinsic motivation, the IM+ subgroup consisted of 24 students (75%) in the EG and 16 students (53%) in the CG. In contrast, the IM- subgroup included 8 students (25%) in the EG and 14 students (47%) in the CG.
The results indicated that the PT-PBL approach increased students’ extrinsic motivation more significantly than the C-PBL approach. Although it did not significantly improve students’ intrinsic motivation, it led to an increase in the proportion of students with improved motivation.
Response to RQ 2. To what extent does the PT-PBL approach improve the students’ reading performance in comparison with the C-PBL approach?
A one-way ANCOVA was conducted to examine the impact of different learning approaches on students’ reading performance. The groups employing different learning approaches (EG and CG) were considered independent variables, while the post-test and pre-test reading performance were the dependent and covariate variables, respectively. The assumption of homogeneity of regression coefficients for total (F = 0.205, p = 0.652), explicit (F = 1.841, p = 0.180), and implicit questions (F = 0.628, p = 0.431) scores was confirmed. Table 2 shows the ANCOVA results.
For the total score, the adjusted means for the EG and CG were 80.18 and 75.34, respectively. For the implicit questions, the adjusted means for the EG and CG were 45.73 and 41.92, respectively. Additionally, significant differences were found between the EG and CG for both total scores (F = 11.850, p = 0.001, ƞ2 = 0.167) and implicit questions (F = 14.130, p = 0.000, ƞ2 = 0.193). However, there was no significant difference between the EG and CG in terms of explicit questions.
The results indicated that the PT-PBL approach significantly improved students’ reading performance more than the C-PBL approach, especially in terms of total and implicit question scores.
Response to RQ 3. What are the differences in reading engagement among students using different learning approaches, and how does varying engagement affect their reading performance?
A one-way ANCOVA was conducted to test the impact of different learning approaches on students’ reading engagement. The groups employing different learning approaches (EG and CG) were considered independent variables, while the learning records in week 4 and learning records in week 1 were the dependent and covariate variables, respectively. Furthermore, the pre-test and post-test scores of the engagement questionnaire were the dependent variables and covariates, respectively. The assumption of homogeneity of regression coefficients for the learning records and engagement questionnaire scores was confirmed. Table 3 shows the ANCOVA results.
In terms of total reading time, the adjusted means for the EG and CG were 73.73 and 55.53, respectively. A significant difference was found between the EG and CG in total reading time (F = 13.501, p = 0.000, ƞ2 = 0.186). In terms of total pages read, the adjusted means for the EG and CG were 27.69 and 25.60, respectively. A significant difference was found between the EG and CG in total pages read (F = 9.115, p = 0.004, ƞ2 = 0.134). In terms of task completion rate, the EG showed a significantly higher completion rate compared to the CG (F = 22.725, p = 0.000, ƞ2 = 0.278). The EG also logged in significantly more frequently than the CG (F = 8.420, p = 0.005, ƞ2 = 0.125). Lastly, the EG showed significantly fewer instances of being temporarily away from the learning platform compared to the CG (F = 16.603, p = 0.000, ƞ2 = 0.220). In terms of engagement questionnaire scores, the adjusted means for the EG and CG were 3.46 and 3.15, respectively. A significant difference was found between the EG and CG in engagement questionnaire scores (F = 11.337, p = 0.001, ƞ2 = 0.161).
Additionally, Fig. 6 displays the trends in learning records over the 4-week period for both groups. As shown in the upper half of Fig. 6, the EG demonstrated a steady increase in total reading time and total pages read across the 4 weeks. Total reading time progressed from 28.43 min in Week 1 to 73.60 min in Week 4, while total pages read increased from 24.34 to 27.59 pages. In contrast, the CG showed a slower improvement, with total reading time rising from 32.04 min in Week 1 to 55.66 min in Week 4, and total pages read fluctuating from 24.60 pages to 25.70 pages. The lower-left part of Fig. 6 illustrates the trends in task completion rate and login frequency. The EG showed a consistent increase in task completion rate, from 58% in Week 1 to 91% in Week 4, while the CG increased from 64% in Week 1 to 89% in Week 4. In terms of login frequency, the EG logged in more frequently, increasing from 1.65 logins per week in Week 1 to 4.93 logins per week in Week 4. In comparison, the CG logged in from 1.50 times per week in Week 1 to 3.66 times per week in Week 4. Finally, the lower-right part of Fig. 6 demonstrates the instances of students being temporarily away from the learning platform. The EG showed a significant reduction in instances of being temporarily away, from 3.03 instances per week in Week 1 to 0.87 in Week 4. The CG, on the other hand, showed a more gradual decrease, from 3.23 instances in Week 1 to 1.46 in Week 4.
It displays the trends in learning records of the experimental and control groups over four weeks, including total reading time, total pages read, task completion rate, login frequency, and temporarily away.
Therefore, by analyzing the learning records and the engagement questionnaire, it can be concluded that students using the PT-PBL approach increased reading engagement more than those using the C-PBL approach.
In addition, a two-factor ANOVA was conducted to investigate the differences in reading performance between students using different learning approaches and different levels of engagement. The different learning approaches and different levels of engagement were considered independent variables, while the pre- and post-test results for reading performance were the dependent variables. There were no significant interaction effects of learning approach and engagement on any of the three dimensions of students’ pre- (F = 0.412, p = 0.524; F = 0.026, p = 0.872; F = 1.326, p = 0.254) and post-test (F = 0.653, p = 0.422; F = 3.250, p = 0.077; F = 0.014, p = 0.905) reading performance. Thus, main effects comparisons were conducted.
As shown in Table 4, in the EG, there was no significant difference in the pre-test reading performance of students with different levels of engagement on the dimensions of total, explicit and implicit questions (F = 3.401, p = 0.075; F = 2.584, p = 0.118; F = 3.317, p = 0.079), while there were significant differences in the post-test reading performance of students with different levels of engagement on the dimensions of total, explicit and implicit questions (F = 8.660, p = 0.006, ƞ2 = 0.224; F = 5.397, p = 0.027, ƞ2 = 0.152; F = 11.719, p = 0.002, ƞ2 = 0.281). The results indicate that among students using the PT-PBL approach, those with high levels of engagement tended to perform better than those with low levels of engagement on the dimensions of total score, explicit, and implicit questions.
As shown in Table 5, in the CG, there were no significant differences between students with different levels of engagement on the total score, implicit questions, and explicit questions dimensions of reading performance in both the pre-test (F = 0.437, p = 0.514; F = 3.074, p = 0.090; F = 0.014, p = 0.908) and post-test (F = 1.175, p = 0.288; F = 0.029, p = 0.866; F = 3.729, p = 0.064). The results indicate that there were no significant changes in the performance of students with different levels of engagement on the three dimensions of total scores, implicit, and explicit questions among students using the C-PBL approach.
Figure 7 demonstrates the effects of using different learning approaches and different levels of engagement on students’ total scores in reading performance. It can be found that the gap between the pre-test and post-test reading performance of students with different levels of engagement in the EG was larger than that in the CG. The results suggest that the PT-PBL approach may be more effective for students with high engagement and less effective for students with low engagement.
It shows the impact of different learning methods and levels of engagement on students’ overall reading performance. H refers to high engagement; L refers to low engagement.
Interview results
Semi-structured interviews were analyzed using NVivo 11.4 to understand students’ perceptions regarding different approaches. Table 6 shows the analysis results from the interviews, including the themes, coding items, and frequency of mentions. Overall, in the interview results, students in the EG mentioned these aspects more than the CG across the four aspects of supporting personalized learning, promoting reflection, improving learning performance, and increasing learning motivation. Conversely, the CG provided more suggestions for improvement.
In terms of “supporting personalized learning”, the EG (n = 22) mentioned it much more frequently than the CG (n = 7). Students in the EG felt that the problems they encountered were more in line with their cognitive level, especially in the self-directed learning phase. In addition, the feedback they received helped them to overcome difficulties.
For example, a student from the EG said, “Compared to traditional learning approaches, this approach presented problems that better matched my cognitive level. Additionally, I received more timely and targeted feedback, which truly helped to improve my problem-solving skills.”
Regarding “prompting reflection”, the EG (n = 18) mentioned it slightly more often than the CG (n = 12). In the first phase, both groups of students felt that their reflective thinking was enhanced during discussions and interactions with others (e.g., peers, teachers). In the second phase, students in the EG received more detailed and targeted feedback through interactions with ChatGPT, and thus they were more inclined to reflect on and optimize their solutions more frequently.
For instance, an EG student stated, “In the first phase, I thought my solution was pretty good, but after receiving the system’s targeted guidance in the second phase, I realized that I could further optimize it.”
With regard to “improving learning performance”, the EG (n = 24) mentioned it much more frequently than the CG (n = 12). Students in the EG felt that this learning approach promoted in-depth comprehension of the reading material and critical thinking, whereas students in the CG rarely reported on these two aspects.
One EG student said, “Compared to traditional learning approaches, this approach helps me understand characters in the story from different perspectives and allows me to analyse their traits more deeply.”
In terms of “increasing learning motivation”, it was mentioned more often in the EG (n = 27) than in the CG (n = 7). Students in the EG were inclined to report that this learning approach promoted active reading and eagerness for the next stage of the task, whereas students in the CG seldom reported on any of these aspects.
An EG student stated, “Not only did I not find these problems boring, but I also thought they were quite interesting. I’m looking forward to solving more of them, which gives me a sense of accomplishment.”
As for “suggestions for improvement”, the CG (n = 18) reported more than the EG (n = 7). Students in the CG felt that the difficulty of the problems was not appropriate (too difficult or too easy). In addition, they reported that the feedback they received was not timely or helpful, whereas the EG students rarely mentioned these aspects.
For instance, a student from the CG said, “Initially, the problems were relatively easy, but later on, they became quite challenging for me. Without timely guidance, I found it difficult to come up with good solutions.”
