Skip to content
KinPilot™Learn & Explore
Menu

Grades 9–12

What can a score hide about the people behind it?

A score summarizes a record, so its meaning depends on what was counted. A completion rate divides completed tasks by attempted tasks. When easy and hard tasks are combined, each group contributes according to the number of attempts it contains. Different mixes can produce a surprising overall comparison.

N08 visual explanationAll records are fictional puzzle completions. A completes 81/90 easy puzzles and 1/10 hard puzzles: 90% and 10%, combined 82/100 = 82%. B completes 19/20 easy puzzles and 16/80 hard puzzles: 95% and 20%, combined 35/100 = 35%. B has the higher rate in each difficulty; A has the higher combined rate. A has many more easy puzzles in its mix. These associations alone do not identify a causal effect or recommend either system.Grouped and combined rates can reverseFictional puzzle completions / attemptsDifficultySystem ASystem BEasy81 / 9019 / 2090%95%Hard1 / 1016 / 8010%20%Combined82 / 10035 / 10082%35%Easy-puzzle share: A 90/100; B 20/100.B is higher in both groups; group sizes differ.

In the diagramGrouped and combined rates can reverseFictional puzzle completions / attemptsDifficultySystem ASystem BEasy81 / 9019 / 2090%95%Hard1 / 1016 / 8010%20%Combined82 / 10035 / 10082%35%Easy-puzzle share: A 90/100; B 20/100.B is higher in both groups; group sizes differ.

The same records can give different grouped and combined rankings when the group sizes differ.

In the pictured fictional records, B completes 95% of easy tasks and 20% of hard tasks, above A’s 90% and 10%. Yet A completes 82% overall while B completes 35%. A has many more easy tasks in its mix: 90 of its 100 attempts, compared with 20 of B’s 100. The combined rate gives those easier tasks greater weight in A’s score.

The chosen measure can affect behavior. If a toy reward pays only for submission count, increasing submissions may earn more points without improving the intended goal of verified work. That result follows from the stated reward rules; real people may respond in many different ways.

Wording can emphasize one side of a record: “80 successes” and “20 failures” may describe the same 100 attempts. Timing can also tempt a stronger conclusion than the evidence supports. An increase after a reward begins does not isolate the reward’s effect without a suitable comparison and attention to other changes. A useful score needs its denominator, context and limits.

Where these ideas lead

Illustrations simplify the real processes; any invented quantities or records are identified in the text.

Sources

  • Pennsylvania State University, STAT 504 — Three-Way Tables: Types of Independence — Simpson’s paradox
  • Stanford Encyclopedia of Philosophy — Simpson’s Paradox
  • OpenStax, Rice University — 8.5 Individual and Group Incentive Plans
  • OpenStax, Rice University — 2.4 Agency Issues: Shareholders and Corporate Boards
  • National Council for the Social Studies — C3 Framework for Social Studies State Standards
  • Amos Tversky and Daniel Kahneman, Science — The Framing of Decisions and the Psychology of Choice (1981)
  • National Park Service — Historic Places and the Inquiry Method
  • OpenStax — Scatter Plots, Correlation, and Regression Lines
← More connected ideas