Formative assessment in the age of AI
A portal for ECON2041
Emilia Tjernström, Strategic Teaching Day
9 September 2026
AI: homework marks \uparrow, exam marks \downarrow
![]()
Homework score (100 = sample mean)
![]()
Homework completion time (minutes)
Red: after adoption
Green: adopters before adoption
Blue: never-adopters
Strömberg, Lei, and Wu (2026), "The generative AI learning penalty: evidence from Chinese secondary education"
AI: homework marks \uparrow, exam marks \downarrow
4 in 5 AI users look like they're "outsourcing"
Strömberg, Lei, and Wu (2026), "The generative AI learning penalty: evidence from Chinese secondary education"
Removing the incentive to outsource
I built a portal with lecture activities & practice questions
Without grades, students have less of an incentive to "outsource"
A wrong answer costs the student nothing (but tells me what they're struggling with!)
Design choices: "login" with a persistent pseudonymous code (no account)
Retrieval practice with feedback
Low-stakes formative assessment with feedback, spread out over time > re-reading notes
My practice bank replicates this: ungraded, feedback on every answer, open all semester
Not "assessment of learning," so "Open" vs. "Observed" is moot
Roediger, Agarwal, McDaniel, and McDermott (2011)
What it looks like & your turn
What students see: lecture & practice
Your turn: guess some stats
Use the sliders to guess
- the total number of questions attempted
- the time spent on the portal
(unconditional & conditional)
About 270 students in the unit
Practice questions open for 2 weeks before quiz
Portal · Projector
Wage data: Graduate Outcomes Survey 2025 medians
Wage data: Graduate Outcomes Survey 2025 medians
Two weeks of practice data
Two weeks of use
Portal dashboard as of 9 September; each set is 3 questions
Use skyrocketed before the quiz
201 students (codes) have submitted practice, answering 16,460 questions between them
194 of those codes were active in the past 7 days
7,142 of the 7,633 sets were drawn in the past 7 days
Practice time, conditional on use
Gaps between a code's submissions, minus minutes with the tab hidden, a new sitting after 30 idle minutes; 200 codes as of 9 September
Engagement beyond attendance
About 45 of 270 students come to the live lecture
201 students have engaged with the practice questions
I see how much, when, and on which topic (whether or not the student attends)
Which topics students choose
Portal dashboard as of 9 September; the last 3 topics opened later than the first 3
Which questions students get wrong
| Research questions and data |
192 |
70% |
| Math tools |
165 |
74% |
| Statistics and probability |
151 |
68% |
| Samples and estimators |
128 |
72% |
| Simple regression |
129 |
69% |
| Regression fit and multiple regression |
99 |
63% |
I explained the questions with the lowest first-try success in a mini-recording
Student feedback
The practice questions have been extremely helpful; I have made my way through some today and will continue to do so whilst reviewing this week's lecture.
The questions have definitely highlighted some spots where I need to do some extra revision, which is helpful to know now 2 weeks before the quiz, rather than the night before!
Unprompted ECON2041 student email
What I don't know yet
Have the practice questions changed how students prepare for the quiz?
Will students return after the quiz, and for how long?
Are the students who practice the ones who needed to?
Homework score (100 = sample mean)
Homework completion time (minutes)
