Google linguist interview

Purpose: This post reviews my experience interviewing for a Linguist position at Google in Santa Monica, CA on February 29, 2008. I've long meant to post this but only now got around to it. There are lots of Google interview stories on the web. It appears to be its own genre. This is my contribution to the genre.

I originally wrote it as an email to a friend who wanted to know how my big day at Google went. It’s rather long, but then again, you don’t have to read it, you clearly have better things to do…

I found a job posting on the Google jobs board for a full time Linguist. I applied and was given a phone interview with a recruiter around late January, 2008:

Thank you for your interest in Google. I'd like to set up a time for us to discuss Google Linguist opportunities and your qualifications. Please let me know a day/time when you would be available to speak with me as well as the best phone number for me to contact you. I'll email you back to confirm.

I hope to hear from you soon!

Cheers,
JF
Google Staffing

During that phone interview the recruiter shared a Google doc which I was instructed to complete in about 45 minutes…



I do not recall the exact questions, but they mostly dealt with asking me to rank ads with web pages results (e.g., if I do a Google search for “lock” and an ad for a locksmith appears, how relevant to the search term was that ad?). It also gave me topics and asked me to provide example web pages that matched the topics.

Once done with this task it was clear that the position I was applying for was not a linguist in any traditional sense.

After they reviewed my answers I was granted a phone interview with a “Linguist” at Google that occurred about a week after the online doc interview.

During this week I snooped around and discovered some interesting facts: The back story is that Google acquired a company called Applied Semantics in 2003. This company was based in Santa Monica, so the office stayed there. Applied Semantics created AdSense, a Latent Semantic Analysis-style algorithm to compare the linguistic similarity of web pages and ads. That company hired people they called "linguists" to 1) evaluate the quality of the comparisons and 2) build and test taxonomies of web pages and ads. These people now work for Google. They also routinely hire people on 1 year contracts to act as test evaluators (see here), but I was interviewing for a full-time, permanent position.

Before the trip, I had a phone interview with LS in the taxonomy group. He had a PhD in formal semantics (I’m pretty sure he was a UCLA linguistics alum). I would say his academic background more closely matched his Google work than anyone else. I don’t recall the specifics of that interview, but I must have done well enough to be invited to fly to Santa Monica for a full day of interviews in late February 2008.

Hi Christopher,

We are very excited that you are coming to talk with us about opportunities here at Google! I've scheduled your interviews per the availability that you gave to JF. We look forward to seeing you on Friday, February 29, 2008 at 10:45am. Your interviews will last approximately 2 to 5 hours. Please ask for MC upon arrival.

Google takes an academic approach to its interviewing process. This means that we are interested in your thought process, your approach to problem solving, and in your programming skills as well. You may also be asked questions that relate to algorithms, data structures, and distributed systems. The dress is business casual/something you are comfortable in (we are more interested in what you have to say than what you are wearing).

The trip began oddly. I was booked into a more expensive hotel than I was originally told (I had to pay for it myself and wait for re-imbursement). Plus, my whole coast-to-coast trip was 36 hours long. Little time to adjust.

I want to give my general impressions of those Google interviews. Nothing specific (because I don’t recall the specifics, hehe) but rather the "vibe" I got. Keep in mind I interviewed with what they call "linguists", not with any engineers, so these are people outside their core employee constituency. I’m pretty sure most, if not all, of the people I interviewed with were hired by the previous company Applied Semantics prior to the Google acquisition.

Generally speaking, I walked away thinking I was qualified to perform any and all of the tasks they described (at least as qualified as anyone I met with given their own backgrounds). I felt like all of these people were smart, but performing tasks for which they had no special training.

I met with five people on-site in addition to my phone interview with LS earlier. One had a PhD in linguistics but on a theoretical topic and one was ABD in art history (both non-empirical methods training) and the phone interview guy had a PhD in linguistics and formal semantics (again, non-empirical training) but he had done empirical, lab-based linguistics work as a post-doc at a well respected east coast university. One person had a B.A in linguistics and another had a B.A. in literature and classics. One guy had a B.A. in French and literature and was a professional translator before being hired.

Almost all of them had been hired prior to the Google acquisition, so they mostly had 6 or more years of experience doing the job, but none seemed to have prior training that matched. I was encouraged by this as my own background did not match what they did. Apparently, that was fine with them.

There were two "groups" of linguists that I met with (literally in different buildings a couple blocks from each other in downtown Santa Monica (near the 3rd street promenade) and very close to the beach. Group one dealt mostly with the taxonomy, or categorization, of web pages and ads (e.g., is web page X a kind of sports page or auction page, etc). The second group dealt with human evaluation of relevance of ad + search correspondence (e.g., ‘how relevant is ad X to web page Y?’).

The first interview I had during the trip was with a taxonomy guy M. who had a BA in French and was a translator who was hired pre-acquisition. He had been with this group for almost 10 years, and he was now a "project manager" and he had the authority to originate his own projects. We discussed a series of things that I felt were close to what I am good at and I liked that interview the best. But, I realized that this guy I was talking to had had little background for the stuff he was doing. He must have learned entirely on-the-job. He asked me two semi-technical questions: He asked me to define “precision” and “recall” (easy) and then he asked me which would be better to use on a new project and why, ASCII or UTF-8 (UTF-8). He described two projects he was working on (one involved trying to auto-detect compound nouns and their relationships, like the difference between “hotdog” and “wedding cake”) and I thought to myself, "why is a guy like you working on that? These are computational linguistics problems. There are people who have solid training in this. Why is a French translator doing this?"

Part of the answer to this is the corporate culture of Google. They let anyone at his level initiate any project they want. He just has to show some kind of results at some point. That probably works great when you have a small group of talented engineers working on tasks within their field of specialization, but when you get so big that you let French translators try to solve problems that PhDs in computational linguistics try to solve, it's bound to go bad.

The second guy I met with, J., only had lunch with me so it wasn't really an "interview". He had a B.A. in lit and classics. I got the worst vibe from him. He wasn't very talkative and I got the impression he wasn't very ambitious. He had been there as long as the first guy, but seemed to be doing lower level tasks, and I think he liked it that way. I felt I had to work a bit too hard to get him to talk, like he was barely tolerating my presence.

After lunch I moved to the second group who deal with human evaluators where I met with three people. I liked everyone I met. They were all responsible for designing online studies to test relevance. These tests were web based human evaluations. I had spent a few years tangentially associated with a psycholinguistics lab in grad school. That lab ran experiments on human subjects testing the natural processing of sentences. I also taught a psycholinguistic course. I have a modest background in testing methodology. I'm not a pro by any means, but at least I've had some exposure to experimental methods.

I felt like most of these people had nothing even close to training in experimental design. The first person, TW, had a BA in linguistics from Berkeley. That’s a nice degree, but it’s unlikely she had much experimental design experience at that level. Unfortunately, I was also suffering a bit of a post-lunch lull in energy and jet lag during her interview, so my concentration and energy was off and I fumbled a bit with some of her hypothetical situations. I felt I did the worst with her.

The second evaluation guy, C., was ABD in art history. I tended to over-talk with him and he had to cut me off a few times in order to ask more questions. I may have been over-compensating given my weak performance with TW just before. The final guy, A.M., had a PhD in linguistics (again, from UCLA). His diss was an OT analysis of something. His advisors were Steriade and Hayes as I recall. We seemed to have the most fun together and laughed quite a bit (I was also loopy from jet lag and a full day of interviews). He accidentally stabbed me with his pencil at one point. It didn’t hurt, but I joked about it. “The guy’s almost outta here and you killed him!”

All of the second group’s interviews went roughly the same. Each one kept asking me how I would design an evaluation for specific kinds of relevancy tasks. Essentially, it was a bunch of what-if questions. They all seemed most interested in my ability to understand experimental methodology.

After the day was done, I walked to the beach (it was over 70 mdegrees and I was about to fly back to the east coast with temps in the teens). I literary took my shoes off and just stood in the warm Pacific ocean for an hour.

So, they’ve got a French translator doing computational linguistics and an art historian doing experimental human evaluation. Hmmmm? What to make of this?

I don’t want to give the impression that I think a person’s college degree pigeon-holes their career. Not at all; my own career proves the opposite, but Google is famous for hiring highly trained subject matter experts. The Google Linguists are different. They are not doing linguistics. Therefore, my own education and experience should have been no more of an issue than the people with whom I interviewed. I feel my resume was at least as good as anyone there for the tasks they described.

Another suspicious part was the fact that at least four of these interviewers asked me if I would be comfortable doing all the little, day-to-day tasks associated with the job. M. even went so far as to say “we support the engineers”. The message I received was that they are low men on the totem pole. I gave the same answer I always give to this question: This is what being a professional means. You do whatever tasks it takes to get the job done.

I left feeling very good. I felt my resume matched well with the people I met and the interviews mostly went well. Three years later, I still haven’t heard from them. Any chance they’re still mulling it over?