Quantifying Google’s Bias

Leo Goldstein wrote a guest article on What’s Up With That, and it is arguably important, to those of us that blog, but also to those of you who are looking for unbiased information. The short form is: Ya ain’t gonna get it from Google.

Abstract

The percentage of domain traffic, referred by Google Search, net of brand searches (PGSTN), tends to be in or around the range 25%-30% for a broad class of web domains.  This hypothesis is tested by calculating the correlation between the popularity of news/opinions websites and their PGSTN, and finding it to be near zero.  Thus, PGSTN can be used rigorously to detect and even quantify Google Search intentional bias.  Intentional bias is the bias that has been introduced by internal Google decisions, and unrelated to external factors, such as the dominance of particular viewpoints on the web.  Here, the PGSTN method is applied for intentional bias detection about climate debate and in general political discourse.

Google Search is found to be extremely biased in favor of climate alarmism and against climate realism.  The PGSTN ranges for climate realism and climate alarmism do not even overlap!  Some of the most important climate realist domains, including low-controversial judithcurry.com, have such a low PGSTN that they can be considered blacklisted by Google.

Google Search is found to be biased in favor of left/liberal domains and against conservative domains with a confidence of 95%.  Further, certain hard-Left domains have such a high PGSTN that their standing raises suspicions that they have been hand-picked for prominent placement.  Certain respected conservative domains are blacklisted.


[…]  Google servers crawl the whole web, extracting text, links, and other data from trillions of pages.  Google constantly and successfully fights attempts to artificially promote websites through collusive linking, and other search engine optimization techniques.  In its undertaking, Google also uses an enormous amount of off-web information, which it collects through Chrome browser, other Google applications and services, analytics beacons, domains registrar status, and so on.  This information includes domains popularity and ownership.  Google also processes immediate feedback from the users in the form of frequency of clicks on the results, bounce rate, the frequency of repeated searches with modified terms, etc.

Google is very good at its job.  Sites and domains that are less popular with the visitors tend to be less likely to receive traffic from Google, and vice versa.  The effect is that percentage of net traffic that domains receive from Google Search tends to be similar across web domains!  […]

Given the robustness of PGSTN, I conclude that statistically significant difference in PGSTN between a priori defined sets of comparable domains is due to intentional bias by Google, unless there is another good explanation.

I’d say this is by no means a manual operation, like nearly everything Google does, it is an algorithm. But my anecdotal evidence confirms what Mr. Goldstein is saying here. Historically, our search referrals were in that range, until July 2016, when they dropped drastically, as they did at AATW where I also write. I  was very noticeable here since we are a small blog and our view stats dropped almost instantly about 50%, nor have we yet reached the level we were at in June of 2016.

Google Bias in General Political Discourse

To quantify Google general political bias, I selected top U.S. news and opinions sites by their ranking in Alexa, then added some lower ranking conservative sites based on my personal knowledge and/or Alexa suggestions.  There was an element of subjectivity in selection and classification, and I omitted some domains that I could not classify.  Nevertheless, the most popular domains in both left/liberal (including Left, Mainstream Liberal, and Mainstream Center) and conservative (including Conservative and Mainstream Conservative) categories have been selected and classified rigorously, and use of weighted statistics minimized the element of subjectivity in the results.

The results show that Google Search is heavily biased against conservative domains, and some respectable conservative domains seem to be blacklisted:

thegatewaypundit.com

pjmedia.com

americanthinker.com

redstate.com

powerlineblog.com

drudgereport.com

Those are some pretty serious political sites, and the part of this I didn’t highlight is that these (NEO too) are climate realist sites, I’m inclined to think it’s natural for those of a conservative outlook to be skeptical of such things. But I have yet to see anything that even came close to convincing me. And that is likely why this was published on Watts Up With That. They are much more involved with the climate debate and the Google bias looks even worse there as well.

Now mind Google is a private company entitled to treat its products as it wishes. But it pays to understand if one’s provider of information is providing slanted data, and just how it is slanted.

 

Advertisements

Leaving on a Jet Plane

Well, I have to get on a jet plane in a few hours. It was unplanned, which is always unpleasant, perhaps we’ll talk about it when I get back, we’ll see. In the meantime, I’ve selected several articles for you from the top twenty all time read articles here (from the several thousand we have written. I’ll only have my phone but will try to check in periodically. Uffda! In the meantime, from my friend, Oyia Brown…

An 85-year-old man was requested by his doctor for a sperm count as part of his physical exam. The doctor gave the man a jar and said, “Take this jar home and bring back a semen sample tomorrow.”The next day the 85-year-old man reappeared at the doctor’s office and gave him the jar, which was as clean and empty as on the previous day. The doctor asked, what happened and the man explained.

“Well, doc, it’s like this–first I tried with my right hand, but nothing. Then I tried with my left hand, but still nothing. Then I asked my wife for help. She tried with her right hand, then with her left, still nothing. She tried with her mouth, first with the teeth in, then with her teeth out, still nothing.

We even called up Arleen, the lady next door, and she tried too. First with both hands, then an armpit, and she even tried squeezin’ it between her knees, but still nothing.”

Continued at: If You Don’t At First Succeed…

See you soon.

Googling Diversity

So should we look a bit at Google’s self-inflicted hangnail? Google is so huge that is about what it amounts to. Still, it is interesting in several ways.

Now mind, I believe that one may run a company one owns as one pleases, and if you choose to promote less competent social justice warriors into posts of responsibility instead of competent software engineers – well on your head be it. The counter argument is that Google is nearly a public utility, but that is not so. The Internet is not necessary (yet, anyway), however desirable it is. And Google does have, in all areas, competitors, some very good.

Ben Domenech lays it out well at The Federalist.

Yesterday, the internet lit up with a flame-war of epic proportions over an internal memo written by James Damore, a Google engineer with a Harvard PhD, who wrote at length about workplace diversity. The memo was the type of statistical analysis one could expect from a dispassionate engineer irritated by a lack of clarity in why fewer women choose to participate in his field, attempting to quantify it beyond the vague assumptions favored by corporate PR. He offered various reasons and explanations for why this could be the case, and offered to discuss the memo further with anyone interested in doing so. For writing this thoughtcrime, he was fired. Google’s CEO claims the memo violated its Code of Conduct. You can read the CEO’s statement here, which stresses that his views were “Not OK”.

These views are consistent with those of Google’s Eric Schmidt, who recently rejected the idea advanced in a Q&A that any right of center views need to be considered part of the bucket of “diversity” favored within Google. As I noted at the time: 

Note the response from Eric Schmidt, who rejects the idea that anyone disagreeing with him politically could be operating from a position of ‘science-based thinking’. The level of diversity and inclusiveness welcomed by Google is precisely as much as is needed to achieve their corporate aims. ‘You’ll also find that all of the other companies in our field agree with us’ – yes, we know.

You see the same tone advanced by Danielle Brown, Google’s vice president of diversity, integrity and governance, who insists the memo’s gender assumptions were “incorrect”.

As always “not consistent” does not mean right or wrong. It means “he doesn’t agree with me, and it’s my company”. As it happens, I believe Damore makes a hell of a lot of sense. But my name isn’t Schmidt, and I’m not CEO of Google, and I do think he can run the company however the shareholders will let him, no matter how stupid. And I note that Google does appear to make money! 🙂

Domenech quotes several of his contributors in his article. As usual David Harsanyi makes a good logical case.

Gizmodo calls a Google engineer’s leaked internal memo about the company’s diversity initiatives an ‘anti-diversity screed.’ Recode calls it ‘sexist.’ Most major news organizations frame it in similar terms. The memo has gone viral. (Update: Google has fired the author for ‘perpetuating gender stereotypes.’) In reality, the problem is that a senior software engineer, perhaps unwittingly, admitted to pondering three of the most scandalous thought-crimes of contemporary American society. The first was to propose that a meritocracy might be heathier for a company than bean-counting race, ethnicity, and sex. The second is pointing that ideological diversity matters. The third, and most grievous of all the wrongthinks, is suggesting that men and women are, in general, physiologically and psychologically different from each other, and thus they tend to excel at different things.

And that my friends is the real ‘thoughtcrime’ here. Wouldn’t it be terrible if we promoted people based on their skills instead of their politics?

History and How It Moves Us

Headquarters parlor VF right

Well, guys, the blog is having the best week it has had in months, so I’m grateful to you all. The last year has been a trial, some say that Google changed the algorithms to hurt conservative blogs. I have no idea if that is so, which is why I haven’t written about it, but I can tell you our readership suddenly halved last July, and just in the last week are we approaching where we were. Hope I’m not jinxing it by telling you. 🙂

There have been quite a few times in the last year when I came very close to hanging it up, just seemed like I wasn’t accomplishing anything, but habit is strong, and I haven’t run out of things to say. So on we go. Things look a bit better here in the Great Republic but much remains to be done, and one hopes it will be. So we’ll see.

As I write this, I’m feeling very down. It is an anniversary in my life, something started some years ago on this date, that I thought would be part of my life as long as I was in this world. Sadly it seems it was not to be, and I’m much the poorer for it. Nothing any of us can do, but last night was a quite hard one for me. Well, such is life, it seems.

Winge over, thanks for listening!

This, however, is very interesting. Did you ever wonder how they would see us say a hundred years from now? Two Nerdy History Girls published on that very topic a few days ago.

Susan reporting,

The perception of the historical past is always changing. Each new generation looks at history with fresh eyes, and fresh ideas, too.

Nowhere is this more evident than in how we Americans have treated our historically important buildings. In the years following the American Revolution, many of the place we now venerate most were simply old buildings, allowed to grow more shabby by the year.

Portions of Independence Hall in Philadelphia – the site of the signing of the Declaration of Independence – had already fallen into such disrepair that they were torn down in 1812. Federal Hall in New York City – home of the first Congress as well as where George Washington was inaugurated as the first president – was also demolished barely a generation later in 1812. Built in 1713, the Old State House in Boston witnessed the Boston Massacre, but was later cut up into shops and businesses, and finally suffered the ultimate indignity of having a subway station built into its basement.

But the Centennial celebrations of 1876 brought a new interest in preserving the past. Older buildings were finally beginning to be recognized and preserved for their historical importance. Sometimes, however, these early preservationists often relied on a romanticized version of the 18thc, with some interesting results.

The present-day Valley Forge National Historical Park in Pennsylvania first became recognized as a state park focused on history in 1893. Then, as now, the centerpiece of the park was the stone farm house used by General George Washington as his headquarters during the Continental Army’s winter encampment of 1777-78. Also known as the Isaac Potts House for the original owner, the Headquarters was occupied not only General Washington, but by his wife Martha Washington, seven aides-de-camp, servants, and occasional visitors. The house is not large, especially not considering how many people were squeezed inside it, and from contemporary reports, quarters were cramped, and tempers often ran short.

Follow the link above and keep reading, I, at least, think it is fascinating, how our perceptions have changed. And like them, I wonder what they will think in 2117.`I just have to tell you, a few years ago I was visiting Valley Forge (again) and I happened to find the Chapel. Well, I can’t remember when I have been so moved. A small Gothic chapel, with the battle flags of the Continental Army as the only decoration other than the altar furniture. It moved me deep in my soul, to the point that now, some thirty years later, I remember it as if it was yesterday.

And that is what a historical site, well presented, will do. Suddenly, we will be able to perceive the struggles that our forebears went through to make our today’s possible. It is simply amazing.

Losses, and Despair

This blogging is a funny thing, often. We end up considering people that we’ve never spoken to as friends and colleagues. One of the people I most enjoyed reading, although only a few times have I referenced his blog was Kevin O’Brien, who went by the screen name of Hognose. He wrote mostly about guns, and their role in our society and his posts were excellent. He was an expert on Czech weapons and made them fascinating. Quite a trick with a guy like me, that tends to think the world begins and ends with Colt. In any case, the last post on his blog was by his brother

I’m sorry to have to tell you all that my brother Kevin O’Brien, host of this blog, passed away peacefully this morning at Brigham and Women’s Hospital in Boston. […]

Now I’d like to tell you more about Kevin and how he lived and died.  He was born in 1958 to Robert and Barbara O’Brien.  We grew up in Westborough, Mass.  Kevin graduated from high school in 1975 and joined the Army in (I believe) 1979.  He learned Czech at DLI and became a Ranger and a member of Special Forces.

Kevin’s happiest times were in the Army.  He loved the service and was deeply committed to it.  We were so proud when he earned the Green Beret.  He was active duty for eight years and then stayed in the Reserves and National Guard for many years, including a deployment to Afghanistan in 2003.  He told me after that that Afghan tour was when he felt he had made his strongest contribution to the world.

He was the best of us, and I’ll miss him, always. Rest in peace, my friend, whom I never interacted with at all.

Also, and even sadder, Bob Owens of Bearing Arms, died yesterday. Preliminary indications are that it was suicide. Bob was an expert on all thing connected with the 2d Amendment. As far as I recall, he only appeared here once, but there are many drafts here for things that for one reason or another didn’t get published. The one that did was from that very dark period following the 2012 election, when it looked like the wheels were really coming off. He said then:

When I wrote What you’ll seen in the rebellion, I had no idea how widely read an article warning of the dangers of second Revolutionary War could become. It has been “shared” thousands of times in social media, and new readers and comments are pouring in constantly.

Since the time it was written, state legislators in Illinois and New York have formally pressed for crushing gun control measures that would outlaw standard-capacity magazines, all semi-automatic handguns, rifles, and pistols, and even pump shotguns. There would be no grandfathering in these proposals, and no inheritance rights. A complete confiscation of these arms, “liberty’s teeth” as other have termed them, are the goal of these totalitarian governments. Legislators in other states are plotting similar measures.

In Washington, DC, federal Democrats—goaded by figures in the poli-media calling for the murders of gun owners and politicians that would oppose their “common sense” statism—have pushed for a similar raft of legislation, supporting the criminalization and confiscation of the most common firearms in the United States and various accessories. The Vice President of the United States has signaled to the mayor of Boston (how historically apt) that the White House itself may attempt to circumvent Congress and attempt to outlaw these common firearms with an executive order.

The people have responded to these threats by not merely buying up the firearms, magazines, and ammunition that might be affected by these proposed bans as they did in 1993/94 before the Clinton-era ban was pushed through, but by purchasing nearly every firearm of military utility of the past 100+ years. Ruger 10/22s and other common .22LR rifles have doubled in price when they can be found at all. Inexpensive Mosin-Nagants, originally designed in 1891 and typically found by the dozen in your average neighborhoodgun shops, are nowhere to be found, and their ammunition is gone as well. U.S. citizens are preparing for war against their government by the millions. Americans aren’t “going Galt” in response to the push by would-be elites to surging statism. We’re on the cusp ofgoing Häyhä. […]

Most of the downtown shops, in fact, weren’t doing much business except the two gun stores. I’d been in one several days ago to pick a .22LR for an article I’d be writing forShooting Illustrated, and decided to stop in at the other to see what the current political environment had left behind.

There were no less than six clerks working feverishly with the dozen or so customers, so I simply stepped to the side and walked the aisles. The cases of ammunition that typically lined the far wall were picked to pieces. There was a 100-round case of .50 BMG, and cases of European shotshells suitable for small game. The .223 Remington, 5.56 NATO, 7.62×39, 7.62 NATO, and 7.62x54R had sold out long ago, along with the bulk 9mm, .40 S&W, and .45 ACP.

A few pump shotguns remained along with a smattering of deer rifles, single-shots, and longer double-barreled shotguns suitable only for trap or skeet. Even the semi-automatic .22LR rifles like Ruger 10/22s were gone, along with all but one BX-25 magazine.

The customers in the shop were picking through what remained; lever-action rifles, oddball shotguns, and the smattering of name-brand centerfire pistols. One man was attempting to trade in an antique double-barrel shotgun for something more current.

I did speak to one harried clerk, briefly.

They didn’t know when they’d be getting anything back in stock, from magazines to rifles to pistols. Manufacturers were running full-bore, but couldn’t come close to keeping up with market demand.It wasn’t just the AR-15s, the AK-pattern rifles, the M1As, and the FALs that were sold out. It really hit me when I realized that the World War-era M1 Garands , M1 carbines, and Enfield .303s were gone, along with every last shell. Ubiquitous Mosin-Nagants—of which every gun store always seems to have 10-20—were gone. So was their ammo. Only a dust free space marked their passing. I’ve never seen anything like it.

Every weapon of military utility designed within the past 100+ years was gone. This isn’t a society stocking up on certain guns because they fear they may be banned. This is a society preparing for war.

He was one of those men who could easily read between the lines, the ones we need to help us all.

From the reports I’ve read, it sounds to me like he suffered from depression, and it’s not uncommon. I’ve dealt with it on a more minor scale, and even there it is an insidious disease, affecting us in ways we simply do not understand, and sometimes cannot even explain to those who want to help. Sometimes nothing at all helps, and even the good things, like the relaxation of the pressure on the Amendment in the last few months, can make things even worse. Yesterday, Nina Bookout at Victory Girls Blog said this

We were shocked and saddened to hear the news that Bob Owens died yesterday of an apparent suicide. Bob Owens deeply loved his family, this country, our Constitution, and was a fierce advocate for our 2nd Amendment rights. One only has to go back through his many blog posts at Bob Owens, his work with PJ Media or as the editor of Bearing Arms, to get a sense of who this man was. I myself first learned of him through his original blog, Confederate Yankee.

I too first read him at Confederate Yankee, and I agree with all she says here. There is a Go-Fund-Me for Bob’s family that you can access here.

God rest his soul, and give his family peace.

Once when she was having a bad spot, Jess wrote to me, ending with this,  “I guess I just have to ride this out and not give in to despair.” I guess she was right for all of us.

Poetry Wednesday

I don’t know why, although I have some ideas, but lately, I have trouble sleeping. Usually, prayer calms me down enough to get to sleep, and then I’m good for the night. The last few months, it hasn’t worked all that well, and so sometimes, I end up having a few stiff scotches to help, even though it’s hardly optimal. I’ve long since asked God to fix it, and still am for it’s more a matter of the soul than anything else, and we all know His power. I’m sure He will, but He hasn’t yet.

So often it ends up that I’m up and wide awake in the middle of the night, doing this or that. Often I’ll read old posts, either here or at AATW, and wonder how to get back to the way I was writing then. Last night was one of those, and I ran across a poem that Fr Robert mentioned in comments a few years ago. It’s one by Henry Vaughn, and in fact, one that is also an old favorite of mine as well. So, I decided to share it once again with you all. Enjoy

THEY ARE ALL GONE INTO THE WORLD OF LIGHT.

THEY are all gone into the world of light !
And I alone sit ling’ring here ;
Their very memory is fair and bright,
And my sad thoughts doth clear.

It glows and glitters in my cloudy breast,
Like stars upon some gloomy grove,
Or those faint beams in which this hill is dress’d,
After the sun’s remove.

I see them walking in an air of glory,
Whose light doth trample on my days :
My days, which are at best but dull and hoary,
Mere glimmering and decays.

O holy Hope ! and high Humility,
High as the heavens above !
These are your walks, and you have show’d them me,
To kindle my cold love.

Dear, beauteous Death !  the jewel of the just,
Shining nowhere, but in the dark ;
What mysteries do lie beyond thy dust,
Could man outlook that mark !

He that hath found some fledg’d bird’s nest, may know
At first sight, if the bird be flown ;
But what fair well or grove he sings in now,
That is to him unknown.

And yet, as angels in some brighter dreams
Call to the soul when man doth sleep,
So some strange thoughts transcend our wonted themes,
And into glory peep.

If a star were confin’d into a tomb,
Her captive flames must needs burn there ;
But when the hand that lock’d her up, gives room,
She’ll shine through all the sphere.

O Father of eternal life, and all
Created glories under Thee !
Resume Thy spirit from this world of thrall
Into true liberty.

Either disperse these mists, which blot and fill
My perspective still as they pass :
Or else remove me hence unto that hill
Where I shall need no glass.

%d bloggers like this: