Challenges in Measuring Language: Designing Tasks and Construct Irrelevant Variance


One of the main challenges is that language
depends on cognitive processes that often times we cannot observe. Unlike measuring
something like a desk, for example, where we can take a measuring tape and figure out
how long that desk is — language is not like that. So, if I’m interested in measuring something
like the range of vocabulary that this person has, it’s unclear how we can do that. It’s
not easy to go in and measure the distinct representations that a person has in their
lexicon. So what we have to do is figure out different ways to get that information. So what we ask them to do, is we ask them
to perform specific tasks, and based on those tasks, we obtain scores. And based on those
scores, then, we make inferences about the cognitive skills that we care about. One of the challenges there is that the tasks
we design, it’s not clear if they elicit information just about the thing we care about, or whether
they tap into other cognitive skills that we might not be interested in. What we refer
to as “construct irrelevant variance.” That’s one of the challenges, obtaining a
score that will give us information about the cognitive skills that we care about.

Leave a Reply

Your email address will not be published. Required fields are marked *