mathematical linguistics for high school students

I received the following email this weekend:

I'm a high school junior from southern California.

For our final project in AP Calculus class, I'm doing a presentation on the connection between mathematics and linguistics, and I stumbled on your blogpost "Why Linguists Should Study Math" while researching my topic.

I was wondering if you could point me towards some resources (that are relatively easy to understand) about how math is present in and affects our written and spoken language.

Some things that I am considering are:

- the occurrences of words in our language

- how grammar uses mathematical principles

- algorithms we use to construct sentences

Thanks,

M.

My [edited] response (suggestions from y'all as to better resources are much appreciated; I'll forward; I wanted to get a response out quickly because the final is presumably fast approaching):

M.,

Thanks for reaching out to me. Of course, I think you’ve chosen a good topic. There are two broad ways in which linguistics and math intersects:

• How the human brain uses math in natural language (psycholinguistics)
• How linguists use math to study and model languages (computational linguistics)

From your email, it appears you are mostly interested in #1. However, in contemporary linguistics, the two are fast becoming one. Most contemporary linguists use math as a tool.

Let me address your three areas of interest with respect to how the human brain might use math to process and produce language:

The occurrences of words in our language: For the most part, this means “frequency” which really means counting. Linguists love to count. We use large corpora of texts to count words and phrases. Lancaster University in the UK is a well-known corpus linguistics school. Their web page has a lot of good introductory information (although I find it a bit clunky looking).

UPDATE: I forgot to include the one item that most directly answers the basic question: frequency effects in language. Human's are very aware of how often they hear words. In some way, we count words automatically, even if it's not quite a specific count like 75, somehow we know which words, phonemes, syntactic structures we hear/read more than others. This gives rise to a variety of frequency effects in language processing. This is the clearest example of how the brain uses math for language.

For example, we recognize high frequency words much faster than low frequency words. The website for Paul Warren's book "Introducing Psycholinguistics" has an online demo for a word frequency task you can walk through to see how linguists study this.
 

What do linguists count?

• Words: I’m sure you’ve seen word clouds like Wordle. This is composed of simple word frequency counts. One of the most enduring facts about word counts is Zipf’s Law which says “the most frequent word [in a corpus of texts] will occur approximately twice as often as the second most frequent word, three times as often as the third most frequent word, etc.” Why would this be true? Linguists have been studying this for decades.
• Ngrams: sets of two-word, three-words, four-word strings, etc. This helps provide more context than mere single word frequencies. Have some fun playing around with Google’s Ngram Viewer if you haven’t already. Try plotting the change in frequency of “mathematical linguistics” and “corpus linguistics” (paste those two phrases into the search box with no quotes and only a comma separating them). Scholars are trying to use this to plot changes in culture. For example, take a look at this PDF.
• Other: We also count many other things too, like parts of speech (verbs, nouns, prepositions, etc). We also count the co-occurrence of linguistics items that are not right next to each other. If you want to dig into more frequency fun, check out the more advanced tools at BYU. You can read more about how these tools help us study language here.

How grammar uses mathematical principles: One of the most commonly studied types of mathematical principle in language is statistical learning. A good example of this is transitional probabilities, which are sets of probabilities for what linguistic item might come next given a string of items (e.g., words or phonemes). For example, if you read “The author signed the _______”, you could guess what the blank word is based on the previous four words (most likely, it’s “book”).  This is based on the psycholinguistic tests called “Cloze tests”. Linguists have discovered that the brain tracks transitional probabilities for all kinds of linguistic items. In fact, this is one of the most robust areas of study in language acquisition. Linguists study how babies use transitional probabilities to learn language. For example, one of the most challenging problems is figuring out how babies learn to separate a continuous stream of audio noise coming in to their ears into separate words, without any knowledge of what words are or what they mean. One theory is that babies quickly learn transitional probabilities of sounds that tell them where one word ends and another begins. But transitional probabilities alone are not enough. For a challenge, try reviewing this PDF:

Algorithms we use to construct sentences: This is the most controversial area you’ve asked about. The fact is, we linguists don’t really know how the brain constructs sentences. As I mentioned above, there are models based on transitional probabilities like Markov models, a computer algorithm designed to make those same kinds of guesses we made about “book”. Markov models and Cloze tests are a good example of psycholinguistics and computational linguistics coming together. As a theoretical contrast to statistical models, there are rule-based models like formal grammars. These are not mathematical in a typical sense, but they are based on formal logic, which is the underlying foundation of mathematics. Linguistics is in the middle of a war between the formal grammar camp and the statistical grammar camp. There’s no consensus on which is the *correct* model of language. However, in the last decade or so, the statistical side seems to have gained the advantage. If you really want to dig in to this war, here’s a challenging read.

Additional Reading:

Linguists who count (the comments are especially engaging; your teacher might be particularly interested in the calculus vs. algebra debate that ensues).

I hope this gets you off to a good start. Please don’t hesitate to ask for clarifications or more resources (especially let me know if you need more intro level or more advanced level; I wasn’t sure if I hit the level right or not). I’m happy to be of more assistance if I can. As a smart, dedicated student, I’m sure you’re ready to dig in to ngrams and Markov models. But, as a high school junior in southern California with June fast approaching, I’m also sure you’re ready for the beach. Both are required for a healthy life of the mind.

college is a ponzi scheme

This NYTs article is getting a lot of buzz on Twitter and Facebook: For Poor, Leap to College Often Ends in a Hard Fall.

Sad but true. College is no longer a mechanism for people to pull themselves up by their own bootstraps. It is a Ponzi scheme meant to prey on poor and working class families. I didn't pay more than $300 for a semester of undergrad between 1989-1993. I would not be where I am today if I had to pay $3000+ per semester for undergrad. If Chico State cost in 1990 what it does today, my family could not have afforded it. No way. I would never have gone to college, never gone to grad school, never gone to the East Coast, and would not have the career I have now. No way.

And I did not go to an elite school. I didn't need to. All I needed was a good public college. State subsidized college worked for me 20 years ago in exactly the way it was supposed to. I went from a poor uneducated family and made an upper middle class career for myself. Expensive college is failing to do any good today except trap poor kids in a downward spiral of debt and diminishing returns.

My income probably puts me into the top 7% of Americans. I hereby agree to pay higher taxes to help kids go to school. It works.

Math Rocks

(image from NYT)

The post title is intentionally ambiguous. In this case, rather than it being a full clause where math is the subject and rocks is the intransitive verb, it is a simple NP where math modifies the plural noun rocks. This is the better reading in relation to this post simply because I am referring to the second installment of Steven Strogatz's excellent NYT series wherein he explains the elements of mathematics to a lay audience. His first topic was the value of abstractness. His second, the value of rocks (or rather, the value of concrete teaching methods like using groups of rocks to demonstrate the meaning of squares, primes, odd vs even numbers, etc). This series is fast turning into a must read. In case anyone wonders why a linguist is referencing a math blog, read THIS.

Unreasonable Effectiveness

Let's be honest, many of us find math intimidating. But it need not be. I recently explained why linguists should study math; now, over the next several weeks, Steven Strogatz,  professor of applied mathematics at Cornell, will be blogging an informal introduction to the basic concepts of mathematics from pre-school to grad school. He starts with Sesame Street and counting fish to explain the basic idea that numbers are abstractions:

The creative process here is the same as the one that gave us numbers in the first place. Just as numbers are a shortcut for counting by ones, addition is a shortcut for counting by any amount. This is how mathematics grows. The right abstraction leads to new insight, and new power.

This is a NYT blog, so let's hope they don't put it behind their new paywall..

HT kotkke

A Working Class Hero Is Something To Be…

(screen shot from Hulu's State of the Nation feed)

"And so tonight, I ask every American to commit to at least one year or more of higher education or career training. This can be community college or a four-year school; vocational training or an apprenticeship. But whatever the training may be, every American will need to get more than a high school diploma."
U.S. President Barack Obama, State of the Nation speech (Feb 24, 2009)

My first reaction is to agree. I think this is a great vision of the future of our country. But President Obama was wise to note the problem of increasing costs of tuition. When I started state college in California in 1989, I spent $198 on tuition per semester. It jacked up to almost $1000 per semester by the time I graduated four years later. The same school now charges over $2000 per semester to attend. This same school was free when I started elementary school.

But what does Obama’s vision of increased post-secondary school training really mean? Is this feasible? The realist in me is forced to think of what this means to actual day-to-day classroom teaching. Does this mean overcrowding post-secondary institutions with sub-par students? That's not good. Are we to expect post-secondary institutions to lower their standards to admit all these new applicants, or are we to "hope" that secondary schools manage to train their students for the post-secondary world?

I know of what I speak. I spent 12 years teaching at colleges and universities. I taught at a community college in California, two private colleges in New York state as well as two public universities on the East Coast. None of these were elite colleges. I had a few "rich" students whose parents were paying their full cost of college but the vast majority of students were working-class or poor students who were in-debt just to get to school every day. This experience gave me a very good sense of the skills and needs of the "average" college student in America. Most were poorly prepared and that affected my day-to-day lesson plans.

The sad truth is that there is an emerging class of private colleges whose business model is based on recruiting exactly the kinds of students Obama just reached out to: those unable to obtain "normal" admission to colleges. The average private college in America looks nothing like Harvard or Stanford. Far from it. They look like businesses.

• They charge $10,000 to $20,000 per year for tuition.

• They tend to focus on one or two vocational majors (like occupational therapy or veterinary technician).

• They tend to have 1000-4000 students.

• They tend to employ large numbers of part-time adjunct faculty (cheap labor and my bread and butter for 12 years).

• They tend to skimp on “non-essential” courses.

I believe President Obama really wants to attain his dream of educational opportunities for all. I believe he genuinely wants to reform teacher training and incentives. But in order to make those dreams a reality, we’re going to have to look at what happened to college tuition between 1975 and 1995, because those were the dark days. In 1975, Obama’s vision would have been feasible. After 1995, his vision became a nightmare.

Can we really expect all Americans to attain post-secondary education under current circumstances? I suspect not.

UPDATE: the AP has published a story addressing this very issue and contains a variety of perspectives here.

UPDATE 2: the Chronicle of Higher Education has a related story on the increasing costs of college here (HT Daily Dish).