Each of the Design Assignments is setup to intentionally focus on a different kind of learning mechanism to help you get a sense for what kinds of game mechanics make sense for which kinds of learning you're trying to evoke. This second assignment will focus on an induction / refinement based learning goal and have you expand out the process from what you did previously to include knowledge elicitation and iteration, to prepare for doing a full process when you get to the final game.

Similar to last time your goal will be to generate 10 different digital game ideas as lo-fi prototypes, and playtest at least 2 of them, however you will start the process by doing an empirical cognitive task analysis with 2 "novices", and you will finish out the process by iterating on 1 of your prototypes and doing an additional playtest. At the end of this process we will want to see playable prototypes of both your first iteration and second iteration games as well as documentation of your process along the way. As with the previous assignment we are interested in a fully-playable core game loop that works for this educational goal and audience, and don’t care much (at all) about how the materials look or level of polish.

As with the previous assignment everyone will design a game for the same instructional domain: Geometric Problem Solving with Circle Angle Theorems, for High School Seniors / Early College Students who self-identify as "not math people". This is a domain we have used in this class for several years and is not being provided by a client, however, goal is to focus on using Knowledge Elicitation to understand how novices struggle with the domain instead of consulting literature of experts.

Knowledge Elicitation: Geometric Problem Solving

This assignment will focus primarily on the practice of doing Knowledge Elicitation to discover issues that novices struggle with in a domain. To keep things focused, everyone will use a standard problem set and accompanying answer key that covers the Interior Angle, Exterior Angle, and Major-Minor Arc theorems. I've used this task for teaching Knowledge Elicitation for a few years now and it tends to work well with adult participants who have likely forgotten the topic. The problem set provides a few potential variations in the task that you might explore, such as providing different levels of scaffolding and reference material, however I recommend you use the entire packet as students in the past who have cut it short have missed on on observing interesting behavior.

Observing Problem Solving

In this assignment you will do a focused Knowledge Elicitation to uncover details of student thinking in a given task domain. Specifically, the goal is to identify and describe the knowledge, strategies, and skills that students need to solve problems, the misconceptions or shallow knowledge that may interfere with learning, any “hidden skills” that may make problems easier than expected, and the kinds of errors that novices typically make. You are asked to collect problem solving protocols from two novices, analyze their similarities and differences, and synthesize a model of novice problem solving in the target domain.

"Novice" Learners

For the purposes of this assignment we will use a flexible definition of novice. For example, in the geometry task it may be hard to find true novices easily but you can use people who may not have done geometry in a long time and are rusty. The focus of this assignment is more on the collection of problem solving protocols to inform design rather than emphasize a particular demographic of learners. Because of this the demographic suggestion of "High School Seniors / Early College Students who self-identify as 'not math people'" is intended to be a rough guideline for the kinds of convenience sample people that would likely serve as a good novices for this task. If, unexpectedly, one of your participants executes the task flawlessly, then that protocol will not count as a novice protocol and you will need to run an additional participant.

Note that outside of the in-class activity we did to demonstrate the cognitive task analysis process, you should avoid using your classmates as novices because you all have access to the same answer key materials. That being said, your classmates should still be OK to serve as playtesters.

Specific Prompt

Design an educational game that:

Leverages Induction / Refinement Learning Mechanisms
Teaches the Interior Angle, Exterior Angle, and Major-Minor Arc theorems informed by Knowledge Elicitation of "novices"
1. Your game must touch on these theorems, it may contain other geometry concepts if you observe them in your novices, but it should not center more prominently on other concepts from geometry
Would appeal to High School Seniors / Early College Students who self-identify as "not math people"

Steps

Collect protocols of at least two novice problem solvers. Recruit 2 participants, run sessions with them following the Gomoll guidelines, and record the protocols. It is important that you recruit novices, that is, people who will not ace the tasks that you present to them. It is also important that you assign the same set of problems to everyone. For the purposes of this assignment we will use a flexible definition of novice. For example in the geometry task, it may be hard to find true novices easily but you can use people who may not have done geometry in a long time and are rusty. If, unexpectedly, one of your participants executes the task flawlessly, then that protocol will not count as a novice protocol and you will need to run an additional participant.
Analyze the protocols. Carefully study the protocols you collected. You should try to follow in an almost line-by-line manner what the thought process was behind each protocol. Pay particular attention to any decision making moments and what features of the task the participant was attending to. For each participant create a table showing for each problem attempted whether they completed it correctly, a brief summary of the strategy they employed, and any other important notes about their problem solving. Then create a flow chart and/or concept map that summarizes each participant's problem solving approach across the problems they attempted.
Synthesize novice approaches. Compare the novice approaches to each other and take note of any similarities or differences. Summarize the main difficulties (2 or more) that the novices seemed to be having. For each, point to at least one protocol and the specific place in the protocol in which it occurs. Based on your analysis of the novice protocols, identify a typical novice approach. Create a flow chart or concept map of this approach. Briefly discuss how the novice strategies you observed differ from your theoretical model created in step.
Brainstorm Design Ideas - Ideate on a list of at least 10 game ideas that satisfy the target prompt. Consider the Game Elements Framework (MDA), Learning Principles relevant to Induction and Refinement, and other ideas we've talked about in class. Document your brainstorming process.
Create Prototypes - Select 2 of your ideas and create low-fidelity playable prototypes of them. These should be demonstrative of the core gameplay loop and allow for players to play with the consequence of any relevant branches in the loop (linear Figma-style click through prototypes would not be sufficient). Diagram out the gameplay loop of each prototype as well.
Playtest your Prototypes - Find a group of playtesters and try out your prototypes. Be sure to take notes on who your playtesters were, whether or not you played with them, how long the session took, and any feedback you got during or after the test.
Iterate on your feedback - Once you have some feedback from playtesting your prototypes consider how you might change your ideas and pick one to iterate
Playtest the iteration - Do another playtest of the iterated prototype. Again, be sure to take notes on who your playtesters were and how the session went.

Collaboration

Individual work, as well as collaboration is permitted for this assignment. Everyone needs to turn in their own work, but in the submission comment collaborators must be listed. For all process portions, we expect a volume of work equal to the amount of collaboration, with the exception of prototyping. Collaboration can be done on some deliverables and not others.

For example, if two students collaborated, we would expect: 4 novice protocols, 20 ideas, more than 1 playable prototype, 1 or more iteration and 6 playtests.

Deliverables

Similar to Assignment 1, this project will be split into 2 deliverable check-ins. The first deliverable (Design Assignment 2A) will involve collecting and analyzing 2 "novice" protocols and brainstorming 10 game design ideas. The second deliverable (Design Assignment 2B) will be a report-out of the prototyping, playtesting, and iteration process of you 2 game ideas and 1 iteration.

Assignment 2A

The general expectation for format for each stage of this assignment is flexible. In comparison to Assignment 1, this assignment involves more writing so a typical document may make sense, though there are also a number of diagrams involved that might be better done as slides. Use whatever format makes sense to you but please submit everything as a single pdf, or a single google doc that links or embeds the other components. Please include the following:

Empirical CTA Participant Recruitment - Provide a description of the participant recruitment strategy along with the backgrounds of the individual participants. DO NOT USE PARTICIPANT'S REAL NAMES (~1 paragraph)
Empirical CTA Participant Strategy Notes (x2) - For each participant provide notes from the individual sessions. This does not need to be an annotated transcript and can be in any format that made sense for taking notes. If you don't know what format to use feel free to the following table as a guide: (2 sets of notes, 1 per participant)

Problem What was their answer? What was the strategy they used? What difficulties did they have? Additional Notes

Problem 1

Problem 2

...
Empirical CTA Unified Diagram - Provide a flow chart or concept map thoroughly covers the strategies used by and could be used as a general novice strategy to the task. This should be one chart describing a general strategy across the novices NOT than one for each problem in the problem set. (1 chart).
Empirical CTA Unified Description - Provide a written synthesis of the unified empirical CTA that describes the overall problem solving strategy novices use, digests the contents of the diagram, provides a summary of the common difficulties they faced, and suggests implications for game design. (~3-4 paragraphs).
Your (at least) 10 game ideas: Individual ideas can be structured using whichever format is appropriate for how you approach the process. The could be anything from a short paragraph, or bulleted list, to a hand drawn sketch or continuous Figma or Miro document. The only format requirement is that we can easily access it and identify distinct ideas. This list will be lightly graded for depth. For example, a pair of ideas like "X, then X but on a phone" would be considered shallow. You can (and probably should) have these kinds of ideas in the process but we are looking for at least 10 that have some real substance to them.

Assignment 2B

The goal for Assignment 2B is to present two playable, playtested games, a game loop diagram for each, and then an iteration and additional playtest of at least one of the prototypes. For this assignment we recommend you turn in a single document with your descriptions and notes. If you have prototypes or other materials in different places you can link out to them from the doc itself. Please be sure any relevant sharing settings are set to "anyone with link". Please include the following:

Description and Gameplay Loops of 2 Prototypes: Descriptions of each of your game concepts, and some discussion (~1-2 paragraphs) of your approach in making each of them, and a justification for how it fits the provided prompt (~1-2 paragraphs). Also include a diagram for each game's core gameplay loop.
2 Low-fi Playable Prototypes: Hand in playable versions of each prototype. This can be physical paper prototypes or low-fi digital interactive as long as they are playable (potentially with the help of a human "game master"). If you do not submit the prototype directly, you MUST submit a video walk through explaining how the game works.
Notes from your playtesting sessions: Describe each of your playtesting sessions. Be sure to include:
1. Details on who your playtesters were (but NOT identifiable information like their name) and any relevant background knowledge they might have had about your learning goal.
2. How you conducted the playtest - include things like: What question were you trying to answer? Did you participate? How did you setup your prototype?
3. How the session went - We are not looking for a transcript, more like a narrative account of what happened. Include things like: Did it go smoothly? Were people confused by any aspect in particular? Did you have to ad lib something? Where there any surprises?
4. A set of insights you got about the prototype in question and what you would change.
Description of Iteration: Describe what you chose to change in iteration and how you changed things.
Your iterated prototype: Provide a playable version of your prototype. If your iterated directly on one of your prior prototypes, clearly describe the changes from the original.
Notes from a playtest of the iteration: Describe the playtesting session. Be sure to include similar details as the individual playtests.
A final reflection: Reflect on the overall process (~1 page). What would you have done differently? What was surprisingly hard or easy? What would be the next thing you would do if you were to continue the final prototype you have?

Use of Generative AI on this Assignment

As with the previous Design Assignment, Generative AI is allowed on this assignment for helping with ideation or creating assets for your prototypes. You may consider using Gen AI to synthesize insights from transcripts of the knowledge elicitation portion of the assignment, however you may not use a conversational agent as a participant for data collection. Part of what we are looking for is the practice of observing real human novices.