How would this questioner successfully ask about Google as a research tool?

Question

• https://superuser.com/questions/875548/

Very often when this is asked, it's actually a linguistics question. It may be in disguise, to a greater or lesser extent, but it's usually a variant of

I'm using Google to obtain word/phrase frequency data, and those data are blatantly not consistent/reasonable. What the Heck?

As you can see from my answers, this is a linguistics question. Linguists such as M. Kilgariff have asked it, long since. And they have answered it too, generally to the effect that a Google Web Search "hit count" isn't the right tool for the job for a lot of reasons.

But it's also a fair bet that had that question been asked here it would have been objected to by people who entirely missed the "try and get a quick sense of which term is more widely used" in the question. This is someone who is doing a little bit of linguistic research on a couple of phrases, and wondering why the tool that xe is using is providing nonsense answers, and whether it's even the right tool to be using. But it's possible to miss that and mis-read this as a question about how Google works rather than as a question about whether Google can be used as a tool for linguistic analysis.

So how would you have phrased this question for this readership of "professional linguists"? Would you have seen it as a linguistics question as it stands?

Answer 1

The most important thing would be to re-phrase it as a question about language structure, and not about the workings of software. General use of Google reveals that it is forgiving about word order and grammatical form, so you might as e.g. "Does Google ignore tense or person inflection on verbs; does it matter what order words come in?". These are essentially linguistic questions, though they tend to engender a "Seriously, dude? Do the experiment". My toy experiment reveals the following search / count results involving collocations of rat and trap:

rat trap = 3,220,000, rat traps = 1,140,000, rats trap = 9,610,000, rats traps = 664,000, "rat trap" = 494,000, "rat traps" = 360,000, "rats traps" = 7,690, "rats trap" = 8,780, trap rat = 21,100,000, trap rats = 9,610,000, "trap rat" = 66,000

I think this shows that if they have a linguistic module in their box, it is screwed up.

This focuses on the potential that there is something linguistic in nature about the behavior of the Google black box. The other side of the coin is the use of Google as a cheap and easy way of generating numeric data about language, a notion that I think is dangerous. As an exercise, try to determine if you can say in Swahili kuku anapika which would mean (a bit of paraphrasing) "the chicken is being cooked". You might think so, though the answer is definitively "no". We contributed to the problem -- Google sucked up ungrammatical examples from StackExchange. Google can't distinguish between speakers of a language and non-speakers. (And other such problems). In that kind of case, all I can suggest is pointing to some concrete absurdity.