On the strengths and weaknesses of “theoretically”

Do scientists use the word “theoretically” the opposite of the way non-scientists do? Let’s see. Below, I walk through a cheap and dirty corpus linguistics method for investigating distributional differences.

This week a colleague of mine brought up an interesting point about the lay person’s use of the word “theoretically” and I thought it merited some investigation and a post. My colleague’s point sprang from a quote that a political pundit made regarding the West Virginia primary and the impact of racism. The pundit was making what is now a rather conventional observation: roughly 20% of white Clinton voters in West Virginia were willing to publicly admit that race was a factor in their voting decision (I don’t know if this is true or not, but it has been repeated in the mainstream media many times, here’s one AP example); if we assume that most people who are at least slightly racist are also at least slightly private in their racism, then we can derive the deduction that more than 20% actually considered the race of the candidates when making their decisions, but some of them refused to admit it.

The pundit chose to express this reasoning by saying something like this (I don’t have a direct quote): ‘In WV, 20% of white Clinton voters admitted that race was a factor; theoretically, this means it’s possible that more than 20% actually considered race important’.

My colleague’s point was that this use of theoretically does not really mean the speaker is referencing some specific scientific theory which predicts a particular outcome. It means something closer to arguably or probably. I agreed and proposed that it might be acting like an inferential evidential, couching the claim in the guise of a fact derived by deductive reasoning.

This got me to wondering about how the word theoretically patterns in common usage. I decided to conduct a brief experiment comparing the words theoretically, arguably, and probably. In order to do this efficiently, I chose to use the freely available and wholly online Collins Cobuild Concordance and Collocations Sampler. This handy dandy online tool allows anyone to extract distributional facts about words quickly from a corpus of 56 million words composed of three subcorpora:

British books, ephemera, radio, newspapers, magazines (36m words)
American books, ephemera and radio (10m words)
British transcribed speech (10m words)

There are two basic tools available:

1) Corpus Concordance Sampler: provides the search word and the sentence it occurred in (well, not quite the sentence, but close enough for Saturday afternoon)

2) Collocation Sampler: provides the words that are statistically significantly associated with the search words (Mutual Information Score plus t-score of significance)

Let’s start with collocates. Below I’ve pasted the ten most significant collocates for the three words under investigation. You could interpret this table as saying something like this: the pronoun it is 3.5 times more likely to occur within 8 words of theoretically than you would assume from random co-occurrence. It is standard within corpus linguistics to interpret this non-random co-occurrence to mean that these two words have some semantically meaningful association (pssst, we should be careful not to over-interpret the meaning of the co-occurrence behavior of pronouns).

theoretically: Collins t-score

Collocate	Corpus Freq	Joint Freq	Significance
it	494702	33	3.555134
could	59556	12	3.027003
least	12333	8	2.717569
possible	10266	6	2.342936
be	234656	15	2.332595
can	113012	9	2.042261
was	340423	16	1.83627
should	35882	5	1.828091
less	14186	4	1.819667
is	407114	18	1.803011

arguably: Collins t-score

Collocate	Corpus Freq	Joint Freq	Significance
the	2313407	179	7.308255
most	43653	53	7.069587
is	407114	56	5.573264
best	20161	32	5.531725
greatest	2506	13	3.581149
important	13468	12	3.327601
was	340423	29	3.165729
finest	1067	7	2.631592
beautiful	4076	7	2.591662
more	94468	11	2.316599

probably: Collins t-score

Collocate	Corpus Freq	Joint Freq	Significance
would	97660	1076	26.18472
it	494702	2333	25.53529
will	111798	1092	25.52538
i	512080	2204	22.70137
think	70465	779	22.29877
you	421797	1797	20.274
ll	34908	545	20.02152
be	234656	1159	18.72308
d	43704	462	16.97461
most	43653	457	16.83863

Note that “d” is likely the contraction for “would” and “ll” is the contraction for “will”.

Analysis: the fact that jumps out at me is the pervasiveness of modal verbs on the lists for theoretically and probably. They both have four modals in their top ten. In all cases, they seem to be expressing epistemic modality in order to hedge the certainty of whatever is being claimed. On the other hand, the word arguably has zero modals in its top ten, but it does have four superlatives, while theoretically has zero superlatives and probably has only one. A picture appears to be emerging.

First Pass Interpretation: All three words appear to be used as hedges. But theoretically and probably appear to pattern closely together as generic hedges, while arguably seems to be a hedge that is strongly associated with superlative claims.

It seems to me that this use of theoretically as a hedge in lay discourse is in contrast with its use in scientific discourse. I would assume scientists are more likely to use theoretically in a sentence to strengthen their claim, not weaken it. However, in non-scientific circles I would guess that scientific theory is regarded with some suspicion. When a claim is based on a theory, non-scientist are more likely than not to consider it not really true. Using the word theoretically, for the lay person, is a way to say “I’m not sure…maybe not.”

Now let’s looks at the concordance. I took the concordance output and performed a little judgment task. For each output set, I created two alternative documents by replacing the target word with one of its alternatives. So the theoretically document had a theoretically_into_probably alternative and a theoretically_into_arguably alternative. I used a five point scale to judge the synonymy of each substitution (i.e., I asked myself ‘does this sentence mean the same thing with the substitution?’):

1. clearly the same meaning
2. kind of the same meaning
3. can’t decide
4. kind of different meaning
5. clearly different meaning

The winner, so to speak, of the most similar prize seems to be arguably into probably. This is to say that I find it generally synonymous to replace the word arguably with probably. Although the reverse, probably into arguably, was pretty good too.

The cases where the theoretically into probably substitution works best seems to be the sentence initial cases where the word is acting as an adverbial hedge over the proposition encoded by the whole clause. But these cases are few. Although these two words seemed to have similar collocates, they do not seem very similar in their actual distributions. This is driven, I think, by the existence of the semantically dissimilar technical use for theoretically. Neither probably nor arguably have this sort of clearly technical usage. Statisticians would likely use the noun probability rather than the adverb probably. I doubt logicians use arguably at all.

theoretically into probably

score	sentence
5	on the way much of the material treated theoretically probably in the lectures.
5	for me the course was very stimulating theoretically probably and practically.
4	Theoretically Probably, this means that at one point during the
4	Theoretically Probably, HMS Champion has been reduced to a lot
5	Let's say it could be done -- theoretically probably."
2	theoretically probably followed in the footsteps of the more
5	Theoretically Probably, underdevelopment could last forever. His
3	Theoretically Probably, it is based on the false notion that
5	`It" exists simply because it is theoretically probably defined,
2	The Social Democrats theoretically probably could now force the Kohl government to

theoretically into arguably

score	sentence
5	on the way much of the material treated theoretically arguably in the lectures.
5	for me the course was very stimulating theoretically arguably and practically.
2	Theoretically Arguably, this means that at one point during the
2	Theoretically Arguably, HMS Champion has been reduced to a lot
4	Let's say it could be done -- theoretically arguably."
4	Theoretically Arguably, underdevelopment could last forever.
5	probably claim that their disciplined, theoretically arguably-informed way of perceiving the social
5	Theoretically Arguably, it is based on the false notion that
5	`It" exists simply because it is theoretically arguably defined, and the ensuing model is then
4	The Social Democrats theoretically probably could now force the Kohl government to

arguably into probably

score	sentence
2	The finishes are arguably probably the finest in Tudor Court. [p] Adjacent to
2	Arguably Probably one of the finest album releases this year.
2	Orfeo is arguably probably the greatest dance work created in Britain
1	A really dark orange, arguably probably the darkest we have ever seen.
1	for instance the first, and arguably probably most important, aim to develop lively
3	Indeed, songs like `Green', arguably probably the standout track of their debut,
2	it belongs to arguably probably the toughest bunch of surfers in the world.
2	Wayne Westner, arguably probably the longest hitter on the European Tour,
2	Arguably Probably, Sheila Henry's behaviour in holding the
2	A week there is arguably probably the most interesting seven days currently

arguably into theoretically

score	sentence
5	The finishes are arguably theoretically the finest in Tudor Court.
5	Arguably Theoretically one of the finest album releases this year.
5	Orfeo is arguably theoretically the greatest dance work created in Britain
5	A really dark orange, arguably theoretically the darkest we have ever seen.
5	for instance the first, and arguably theoretically most important, aim to develop lively
5	Indeed, songs like `Green', arguably theoretically the standout track of their debut,
5	it belongs to arguably theoretically the toughest bunch of surfers in the world.
5	Wayne Westner, arguably theoretically the longest hitter on the European Tour,
5	Arguably Theoretically, Sheila Henry's behaviour in holding the
5	A week there is arguably theoretically the most interesting seven days currently

probably into arguably

score	sentence
2	your property is probably arguably worth far more than you owe on it.
5	You've probably arguably guessed that we couldn't resist using white
2	It probably arguably would, but you'd just have been postponing
4	It probably arguably sounds silly,' she adds,
3	We came to agreement at 1am, probably arguably at the highest pitch of drunkenness
4	a member and an ANC team which it said would probably arguably include Mr Mandela.
5	I'm probably arguably all wrong," he said.
4	it was the last that probably arguably accounted for their rapid answers.
2	Those in the high intelligence group are probably arguably similarly subject to difficulties
1	It was probably arguably the most comprehensive compilation of facts

probably into theoretically

score	sentence
4	your property is probably theoretically worth far more than you owe on it.
5	You've probably theoretically guessed that we couldn't resist using white
3	It probably theoretically would, but you'd just have been postponing
5	It probably theoretically sounds silly,' she adds, `but when it was
5	We came to agreement at 1am, probably theoretically at the highest pitch of drunkenness
4	a member and an ANC team which it said would probably theoretically include Mr Mandela.
5	I'm probably theoretically all wrong," he said.
5	it was the last that probably theoretically accounted for their rapid answers.
5	Those in the high intelligence group are probably theoretically similarly subject to difficulties
5	It was probably theoretically the most comprehensive compilation of facts