Quantifying Google’s Bias

Leo Goldstein wrote a guest article on What’s Up With That, and it is arguably important, to those of us that blog, but also to those of you who are looking for unbiased information. The short form is: Ya ain’t gonna get it from Google.

Abstract

The percentage of domain traffic, referred by Google Search, net of brand searches (PGSTN), tends to be in or around the range 25%-30% for a broad class of web domains.  This hypothesis is tested by calculating the correlation between the popularity of news/opinions websites and their PGSTN, and finding it to be near zero.  Thus, PGSTN can be used rigorously to detect and even quantify Google Search intentional bias.  Intentional bias is the bias that has been introduced by internal Google decisions, and unrelated to external factors, such as the dominance of particular viewpoints on the web.  Here, the PGSTN method is applied for intentional bias detection about climate debate and in general political discourse.

Google Search is found to be extremely biased in favor of climate alarmism and against climate realism.  The PGSTN ranges for climate realism and climate alarmism do not even overlap!  Some of the most important climate realist domains, including low-controversial judithcurry.com, have such a low PGSTN that they can be considered blacklisted by Google.

Google Search is found to be biased in favor of left/liberal domains and against conservative domains with a confidence of 95%.  Further, certain hard-Left domains have such a high PGSTN that their standing raises suspicions that they have been hand-picked for prominent placement.  Certain respected conservative domains are blacklisted.


[…]  Google servers crawl the whole web, extracting text, links, and other data from trillions of pages.  Google constantly and successfully fights attempts to artificially promote websites through collusive linking, and other search engine optimization techniques.  In its undertaking, Google also uses an enormous amount of off-web information, which it collects through Chrome browser, other Google applications and services, analytics beacons, domains registrar status, and so on.  This information includes domains popularity and ownership.  Google also processes immediate feedback from the users in the form of frequency of clicks on the results, bounce rate, the frequency of repeated searches with modified terms, etc.

Google is very good at its job.  Sites and domains that are less popular with the visitors tend to be less likely to receive traffic from Google, and vice versa.  The effect is that percentage of net traffic that domains receive from Google Search tends to be similar across web domains!  […]

Given the robustness of PGSTN, I conclude that statistically significant difference in PGSTN between a priori defined sets of comparable domains is due to intentional bias by Google, unless there is another good explanation.

I’d say this is by no means a manual operation, like nearly everything Google does, it is an algorithm. But my anecdotal evidence confirms what Mr. Goldstein is saying here. Historically, our search referrals were in that range, until July 2016, when they dropped drastically, as they did at AATW where I also write. I  was very noticeable here since we are a small blog and our view stats dropped almost instantly about 50%, nor have we yet reached the level we were at in June of 2016.

Google Bias in General Political Discourse

To quantify Google general political bias, I selected top U.S. news and opinions sites by their ranking in Alexa, then added some lower ranking conservative sites based on my personal knowledge and/or Alexa suggestions.  There was an element of subjectivity in selection and classification, and I omitted some domains that I could not classify.  Nevertheless, the most popular domains in both left/liberal (including Left, Mainstream Liberal, and Mainstream Center) and conservative (including Conservative and Mainstream Conservative) categories have been selected and classified rigorously, and use of weighted statistics minimized the element of subjectivity in the results.

The results show that Google Search is heavily biased against conservative domains, and some respectable conservative domains seem to be blacklisted:

thegatewaypundit.com

pjmedia.com

americanthinker.com

redstate.com

powerlineblog.com

drudgereport.com

Those are some pretty serious political sites, and the part of this I didn’t highlight is that these (NEO too) are climate realist sites, I’m inclined to think it’s natural for those of a conservative outlook to be skeptical of such things. But I have yet to see anything that even came close to convincing me. And that is likely why this was published on Watts Up With That. They are much more involved with the climate debate and the Google bias looks even worse there as well.

Now mind Google is a private company entitled to treat its products as it wishes. But it pays to understand if one’s provider of information is providing slanted data, and just how it is slanted.

 

Advertisements

Googling Censorship

So, this story is out, and tell me why I’m not surprised. I noticed it from John Hinderaker at PowerLine, and he linked on to PJ Media, which has a long story by Paula Boyard up. I suspect it going to be a long series by many of us on this matter. It’s both frightening and interesting. Here’s some of it.

Google revealed in a blog post that it is now using machine learning to document “hate crimes and events” in America. They’ve partnered with liberal groups like ProPublica, BuzzFeed News, and the Southern Poverty Law Center (SPLC) to make information about “hate events” easily accessible to journalists. And now, there are troubling signs that this tool could be used to ferret out writers and websites that run afoul of the progressive orthodoxy.

In the announcement, Simon Rogers, data editor of Google News Labs, wrote:

Now, with ProPublica, we are launching a new machine learning tool to help journalists covering hate news leverage this data in their reporting.

The Documenting Hate News Index — built by the Google News Lab, data visualization studio Pitch Interactive and ProPublica — takes a raw feed of Google News articles from the past six months and uses the Google Cloud Natural Language API to create a visual tool to help reporters find news happening across the country. It’s a constantly-updating snapshot of data from this year, one which is valuable as a starting point to reporting on this area of news.

The Documenting Hate project launched in response to the lack of national data on hate crimes. While the FBI is required by law to collect data about hate crimes, the data is incomplete because local jurisdictions aren’t required to report incidents up to the federal government.

All of which underlines the value of the Documenting Hate Project, which is powered by a number of different news organisations and journalists who collect and verify reports of hate crimes and events. Documenting Hate is informed by both reports from members of the public and raw Google News data of stories from across the nation.

On the surface, this looks rather innocuous. It’s presented by Google as an attempt to create a database of hate crimes — information that should be available with a quick Google search, it should be noted. But a quick glance at the list of partners for this project should raise some red flags:

The  ProPublica-led coalition includes  The Google News Lab,  Univision News, the  New York Times,  WNYC,  BuzzFeed News,  First DraftMeedan,  New America Media,  The Root,  Latino USA,  The Advocate100 Days in Appalachia and  Ushahidi. The coalition is also working with civil-rights groups such as the  Southern Poverty Law Center, and schools such as the  University of Miami School of Communications.

ProPublica poses as a middle-of-the-road non-profit journalistic operation, but in reality, it’s funded by a stable of uber-liberal donors, including George Soros’s Open Society Foundations and Herb and Marion Sandler, billionaire former mortgage bankers whose Golden West Financial Corp. allegedly targeted subprime borrowers with “pick-a-pay” mortgages that led to toxic assets that were blamed for the collapse of Wachovia. The Southern Poverty Law Center, of course, is infamous for targeting legitimate conservatives groups, branding them as “hate groups” because they refuse to walk in lockstep with the progressive agenda. And it goes with out saying that The New York Times and BuzzFeed News lean left.

A perusal of the raw data that’s been compiled thus far on hate stories shows articles from a wide array of center-right sites, including The Daily Caller, Breitbart News, The Washington Times, National Review, and the Washington Examiner. It also includes many articles from liberal sites like BuzzFeed News and The New York TimesOne story from PJ Media’s Bridget Johnson is included in the list. It’s a report about a Sikh ad campaign aimed at reducing hate crimes against members of their faith community. Many of the articles are simply reports about alleged hate crimes from sources running the gamut of the political spectrum.

ProPublica vows to diligently track “hate incidents” in the coming months. “Everyday people — not just avowed ‘white nationalists’ — intimidate, harass, humiliate and even harm their fellow Americans because of the color of their skin, how they worship or who they love.” [Emphasis added] Note that they’re not just focusing on hate “crimes.”

It’s easy enough to figure out the direction of this project by taking it for a test drive. A search for “Scalise” returned four results, one of which didn’t even mention Steve Scalise, the congressman who was shot by a crazed leftist in June. A search for “Trump” during the same time period yielded more than 200 results. A search of the raw data resulted in 1178 hits for Trump and not a single mention of Scalise.

Note that Google, which recently fired an employee for expressing his counter-progressive opinions, thinks this information could be used to “help journalists covering hate news leverage this data in their reporting.” What do they mean by “leverage this data”? They don’t say, but an email sent to several conservative writers by a ProPublica reporter may give us some indication. Pamela Geller and Robert Spencer along with some others received this from ProPublica “reporter” Lauren Kirchner:

I am a reporter at ProPublica, a nonprofit investigative newsroom in New York. I am contacting you to let you know that we are including your website in a list of sites that have been designated as hate or extremist by the American Defamation League or the Southern Poverty Law Center. We have identified all the tech platforms that are supporting websites on the ADL and SPLC lists.

We would like to ask you a few questions:

1) Do you disagree with the designation of your website as hate or extremist? Why?

2) We identified several tech companies on your website: PayPal, Amazon, Newsmax, and Revcontent. Can you confirm that you receive funds from your relationship with those tech companies? How would the loss of those funds affect your operations, and how would you be able to replace them?

3) Have you been shut down by other tech companies for being an alleged hate or extremist web site? Which companies?

4) Many people opposed to sites like yours are currently pressuring tech companies to cease their relationships with them – what is your view of this campaign? Why?

In other words, nice website you’ve got there. It would be a shame if anything happened to it.

There is an update to that story dated August 19th.

ProPublica came out today with the expected hit piece on Robert Spencer, Jihad Watch, and others they disagree with, repeating the Southern Poverty Law Center’s smears and legitimizing the dishonest group’s hate list. In the article titled “Despite Disavowals, Leading Tech Companies Help Extremist Sites Monetize Hate,” Lauren Kirchner along with two fellow journalistsactivists documented the recent blacklisting of “hate websites” by tech companies and, although they didn’t come right out and say it, strongly implied that this should be the norm. They accept without question the hate designations bestowed by the SPCL and the Anti-Defamation League (ADL). The article leaves no doubt that ProPublica — which is working with Google, remember — wants to see more blacklisting. They will not rest until every one of the names on SPLC’s dubious 900-member hate list is purged from the Internet. Make no mistake. They are marshaling forces to pressure advertisers and tech providers to take conservative sites down. Just take a look at this list of Christian groups that made the listbecause they haven’t jumped on the LGBTQ bandwagon. […]

Do read it all at Is Google Working with Liberal Groups to Snuff Out Conservative Websites?

In a related matter, one of the reasons, beyond simple convenience, that I’ve stayed all these years with WordPress.com is their often pledged word, “WordPress and its parent company Automatic do not censor, period.” I’ve always found that to be true. But perhaps that just changed as well. From Fast Company.

“Fascist” is often an epithet used to demean an opponent, but for alt-right organization Vanguard America, it’s a badge of honor. As of last night, the group lacks a website where it can proclaim that message. Going to its URL bloodandsoil.org leads to a message from site host WordPress that reads, “This blog has been archived or suspended in accordance with our Terms of Service.”

That’s somewhat surprising. A few months ago, I asked WordPress about its hosting of Vanguard America, United Dixie White Knights of the KKK, and several other far-right organizations for a story about hate sites and their tech providers. The stock answer was that WordPress and its parent company Automatic do not censor, period.

Vanguard America’s website as of last night.

Now mind, I’ve never been to that website, for me they are beyond the pale. But freedom of speech means the freedom to offend. And they have just as much right to speak as I do, or for that matter as <insert violent left-wing organization here> does.Gives me a sort of chilly feeling and reminds me that it is about time to back up the website again, out of reach of all the hypocrites.

 

The Week in Picture: The Bombing Starts in 5 Minutes Edition

Hah, Saturday snuck up on me, but I saw it coming. So a bit has gone on this week, as usual, summed up well in pictures. Here’s some of them.

 

Aws usual, most from PowerLine. Have a better week

 

Education, Students Loans, and John Adams

quote-education-makes-a-greater-difference-between-man-and-man-than-nature-has-made-between-man-and-brute-john-adams-314611John Adams once wrote this to Abigail:

“The science of government it is my duty to study, more than all other sciences; the arts of legislation and administration and negotiation ought to take the place of, indeed exclude, in a manner, all other arts. I must study politics and war, that our sons may have liberty to study mathematics and philosophy. Our sons ought to study mathematics and philosophy, geography, natural history and naval architecture, navigation, commerce and agriculture in order to give their children a right to study painting, poetry, music, architecture, statuary, tapestry and porcelain.”

Personally, I think higher education in this country has lost its way. Easy money has converted it from what Adams thought his grandsons should study to what he had studied. It has become little more than a trade school, a factory for diplomas, and often a very expensive one.

Now mind, there is nothing at all wrong with trade schools, we must, if we are to live even moderately well, know how to govern ourselves, and defend ourselves, not to mention fix the roads and plumbing. That is all very honorable, but it does not require, although it often benefits from, an education in the classic liberal arts, and the practitioners always do. But it does not require it.

To me, Adam’s second tier, that his sons should study, is represented these days mostly by the so-called STEM courses: science, technology, engineering, math. They are the middle way, more abstract thinking, and vision but rooted in the practical, adding to that an ability to communicate clearly and effectively, and you create the world of tomorrow. This is the realm of the inventor/entrepreneur: the Edisons, the Bells, but also the Thomas Crappers, the Commodore Vanderbilts, the Carnegies, and also Steve Jobs and Mark Zuckerberg, not to mention Dr. Jonas Salk,  those who take ideas, and make them practical, and bring them to market.

But that third tier, has little direct connection with the practical. this is where we learn about ourselves, and learn to make men better. It is the highest expression of civilization, if it is not, something has gone wrong. There is an upper limit, and it is quite low, on the number of people who can be supported adequately to study this. In large measure, the prosperity of Britain and America in the last four hundred, or so, years, has allowed us to lead civilization, because we could afford to think, to question, and to discuss, these matters.

And so, if you are a high school senior, you likely want to go to college. Why? To be a better barista? Well, no doubt you will be, but enough better to justify the cost? Or to be an engineer? That will justify much more education than being a barista will, but not an infinite cost. Always, always, as you enter the job market, your value is based on what you know that is relative to the job on offer. If I’m hiring an apprentice, I don’t expect you to know much about electricity (and most of that will be wrong) as I expect you to have a strong back, and a willingness to learn. Frankly a know-it-all with a degree is less attractive than a high school drop-out who desperately wants to earn a living. And that is the trap, my young friend, when you come out of college, with that expensive degree, in whatever irrelevant (to me) subject, bought with borrowed money, you are worth no more in the market that drop-out working for his next meal, and that’s what I’ll pay you. Will you advance further and/or faster? Perhaps, that’s up to you, your application of your knowledge (and ability to learn) and your attitude in a number of ways.

Hard words? Perhaps, but they’re also true ones won in the school of hard knocks provided by experience. Here are some more

And always remember that you do not go to college to learns stuff. You go to college to learn how to think, and learn.

Facts are stubborn things; and whatever may be our wishes, our inclinations, or the dictates of our passion, they cannot alter the state of facts and evidence.”
John Adams, The Portable John Adams

EU Preps for War Against the Internet: Decides to Lose Again

AAEAAQAAAAAAAANYAAAAJGU4MmZmYjg2LTg5NjQtNDFiNS04MWRkLTcwZmMyNmY0M2RkMAWell, this is interesting, although not very surprising, really. Does anybody really think that Europe (especially Germany and France) can compete with the US on a level playing field? No, me neither. The UK, maybe, but nobody else has a chance, and if good sense ever breaks out in the ruling clique in Britain (or they lose the election) they’ll likely get with the program and with their friends and run away from Europe, again.

I say that because I’ve noticed something. If you look at European technical prowess, especially innovation, in anything from civil engineering to the internet, you’ll find the British leading, and everybody else following, while they whine about ‘the Anglo-Saxons’.

They’re right, as well. The American Interest noted today that the EU wants to regulate Google et. al., much more than they do.

THE EU VS SILICON VALLEY

EU Preps for War Against the Internet

EU Preps for War Against the Internet – The American Interest.

As an aside, I’m no huge fan of Google, I think they’re more than a bit intrusive, and I’m not overfond of their data mining and selling my information to all and sundry. But you know what, I use Google products because they work, I don’t have to. There are other providers, just as I no longer use Microsoft products. But it’s remarkable that a company that started in an American garage a few years ago has all Europe scared of them 🙂

Maybe I’m just old-fashioned but I hope they do. Why? because if they do, the US will simply increase our lead over the hidebound, over-regulated Europeans, while the best Europeans will again come to America where they can innovate much more freely than they can at home. (And make us still richer, and more innovative!)

Funny thing, isn’t it? We’ve built this powerhouse of a country (not that we don’t have plenty of problems, ourselves) on the freedom to try new things and see if you can make a living with them. We’ve done this since about 1650,nd we have built the most powerful economy in the world, and protect it with the most dominant military the world has ever seen with our pocket change. We’ve done this by letting people try and fail, and try and fail, and finally try and succeed.

It’s a hard model. It’s follows from that old saying about the Oregon Trail, “The weak never started and the sick died along the way,” But, you know, there was nearly always someone around to feed the hungry and nurse the sick, and the dead got a decent burial. And the ones that made it, built a world that their grandfathers couldn’t have imagined, where one of the consequences of being poor is being too fat, because you eat too much while playing video games.

I don’t condone such a lifestyle but I’m in awe at a system that can take a world that nearly starved for billions of years and in a few generations make that happen.

And that is what America has done, with some British help (and gold) and with the people who were stifled by Europe. It’s a logarithmic curve, if you haven’t noticed, constantly accelerating, if we keep going there is no way to know where we’ll be in twenty-five years, let alone a hundred.

Carroll Bryant once said:

Some people make things happen.

Some people watch things happen.

And then there are those who wonder, ‘What the hell just happened?”

I know where I want to be. How about you?

%d bloggers like this: