What are Algorithms—and are They Killing the "Kindle Revolution"?


First, an awesome announcement: this blog will be named to Writer's Digest's Best 101 Sites for Writers in the May-June issue! (special thanks to Lila and Janet for the heads-up.) We are so jazzed! The magazine should be in stores soon. Subscribers have already got their copies. Many, many thanks to whoever nominated us!! 

To celebrate this exciting news, my publisher has made all my mysteries 99 cents until the end of April. That's at Amazon and Barnes and Noble. This includes The Gatsby Game Food of Love, and the Camilla Randall Mysteries, either as the boxed set or singly, and No Place Like Home, usually $4.99 (still only at Amazon, alas.) Click those links or click through the book covers in the sidebar. Here's my author page at Barnes and Noble.

And Ruth Harris has joined in, making her Park Avenue Boxed Set only 99 cents as well. (That's 33c a book!) 

OK, we hear about them all the time, but what the *%@# are algorithms, anyway?

An algorithm is a line of code that gives a search engine a step-by-step process to produce a desired result: like a list of websites, a bra that fits, or a suggestion for the next title to put on your TBR list.

They aren't new. In fact they were invented in the 8th century AD by a Persian guy named Abū ʿAbdallāh Muḥammad ibn Mūsā al-Khwārizmi . The word "algorithm" or "algorism" is a corruption of his name: al-Khwarismi.

But in the digital age this venerable type of formula has taken on a huge importance for the marketplace. An algorithm can make or break sales of any product online. As superstar author Hugh Howey said in an interview on Reddit, “I'm guessing 90% of my sales are from reader recommendations and Amazon algorithms.”

Or as his interviewer called them "the inscrutable Amazon algos."

They're sure not scrutable to me. I'm pretty sure they have to do with math, at which I suck. Seriously. I had to be counseled for “math anxiety” in high school.

Besides, algorithms aren’t that easy to define—even for the people who work with them. The Wikipedia article says, “While there is no generally accepted formal definition of 'algorithm,' an informal definition could be 'a set of rules that precisely defines a sequence of operations.' For some people, a program is only an algorithm if it stops eventually; for others, a program is only an algorithm if it stops before a given number of calculation steps”

Yeah, right. If you’re like me, your eyes have started to glaze over already. Secretly, I think of them as little robot elves who get things done inside my computer.

Like bringing up 64,300,000 links in .52 seconds when you put the word “algorithm” into Google’s search window.

Pretty durn clever, these elves. And kind of Big Brothery. They're what Facebook uses to analyze your “Likes” and figure out if you’re gay, conservative and/or smart.

They’re not infallible. They're responsible for sending you those emails from Amazon suggesting you buy or review your own book, and they're the reason you got spammed with dozens of ads for refrigerators after you accidently Googled “fridge” instead of “fudge” last week.

But they work well enough that businesses increasingly depend on them. A new online lingerie retailer, True&Co, uses a questionnaire and an algorithm to sell mail-order bras—a product that’s difficult to fit even in real-life dressing rooms. According to the NYT, at True&Co “women end up buying more of the bras chosen by the algorithm than the ones they select themselves.”

As you might imagine, online book retailers depend heavily on algorithms to get books in front of the people who might buy them.

David Gaughran talked about the complexity of Amazon’s algorithms in making Amazon’s bestseller lists on his blog, Let’s Get Digital.

“Amazon has a bevy of Bestseller Lists, all split into Free and Paid listings. The big one is theTop 100 in the Kindle Store, and placement on this list can drive staggering amounts of sales. This list is populated with items ranked #1 to #100 in the overall Kindle Store, which includes not just e-books, but also things like games, magazines, and newspapers.

The exact algorithm Amazon uses to assign a Sales Rank to each book is a closely-guarded secret, but the general make-up is easy to deduce. Simply put, your Sales Rank tells you how many books are selling more than you at this moment in time (it’s updated hourly). However, it also takes account of historical sales. More recent sales are weighted much more heavily in the algorithm, though, and velocity plays a big part too (how much your sales are increasing at that moment in time). There’s a lot more to it, but those are the basics.”

David talks more about how Amazon’s algorithm-driven recommendation engines work in a post from February 2013.

But the most important thing to know about the algorithms book retailers use is that they CHANGE. Those robot elves are programmed by real people—who are constantly inputting new data.

Mark Coker changed algorithms at Smashwords last year to encourage authors to stop devaluing their work by giving away so many free books.

Mr. Coker said, “At Smashwords, up until January of this year [2012], our algorithm for best sellers just looked at absolute downloads, but authors gamed that system by offering discounted or free books using the Smashwords coupon tool. So we changed our algorithm to focus on gross sales, so now when you look at the Smashwords best seller list, you’re looking at absolute dollar sales, which we are using as a representation of the interest of customers.”

Amazon changes its algorithms more often than Smashwords. Which means a whole lot of the “rules” you hear about how to sell big on Amazon are based on algorithms that no longer exist.

If you're in the indie author scene, you've no doubt heard a ton of rumors about what the Amazon algorithms will reward or penalize. You’ll hear that more than 50—or 100—“likes” will get more recommendations, or any 3-star or lower reviews will exclude you from the also-boughts or downgrade your author rank. Or that uploading from a non-US country downgrades your status, or dissing Amazon in their forums will be reflected in your sales ranking. Some of these may be true or may have once been true, but they’re hard to prove. And trying to game them generally isn't a good idea.

That's because the one thing we do know for sure about algorithms is that Amazon does not like it when you try to game them.

Which is probably why the Zon is removing the “like” buttons and “tags” on US buy pages. Every author group I belong to asks people to “like and tag” each other’s book pages whether or not the books are in a genre you read or enjoy. Which means the “likes and tags” no longer have meaning for tracking a person's individual preferences, which was the point in the first place.

Even though nobody can know for sure how the “inscrutable algos” are programmed, Amazon watchers can tell when there’s been a change. Some people make a serious study of them.

With the advent of KDP Select—Amazon’s program for books they have exclusive rights to sell—the algorithms were changed to benefit Select members. Select books get more weight in the popularity lists, more suggestions for “also bought” books, e-mail recommendations, and are featured when somebody searches the genre. That makes sense. When you ask for an exclusive, you need to give perks.

Now there’s an even more privileged “White Glove” program for books that have been “self-published” through a literary agency. It has a longer exclusive period and gets the benefit of even more also-boughts and other promotions.

Nothing wrong with that. It’s Amazon’s store and they can promote what they want. But it does make it harder on the truly self-published and small-press competition.

The “Kindle Revolution” that Amazon sparked in 2009 allowed individual authors to self-publish cheaply and gave them an equal opportunity with big publishing’s authors to get discovered. This was a brilliant way of providing a lot of inexpensive books for their brand new Kindle at a time when most people didn’t even think they needed an ereading device.

It was win/win. Amazon sold Kindles—and individual authors got to enter a marketplace that had been reserved for big corporations for half a century.

In doing that, Amazon shook up the book world and did a service to new authors that was pretty unprecedented.

But since then, Amazon has become a publishing company itself. Their Thomas and Mercer, 47North, Montlake Romance, and other imprints now compete with the Big 5. They also compete with the indies. Guess who the algorithms are likely to favor? (I did mention the elves are smart?)

Meanwhile, Amazon's competition as a retailer continues to grow. Kobo and Apple are gobbling up market share. Like any big business, the Zon wants to dominate the competition and keep its place as Alpha Dog of the online retail business.

So they’ve made it very appealing for authors to sell exclusively through them—both with their KDP Select and White Glove program.

And last month they bought Goodreads. We still have to see how that will affect self-and small-publishers. David Gaughran thinks it’s a boon to authors. But some others, like Jarek Steele, and The Author's Guild —not so much. But Mr. Coker saw a silver lining: a way of thwarting people who try to manipulate the elves with paid reviews. For a nice, balanced overview of the argument, read Porter Anderson's piece in Ether for Authors at Publishing Perspectives.

The one thing people agree on is that Amazon wanted Goodreads' data to feed its algorithms. As Gaughran says “Amazon’s recommendation algorithms will be vastly improved with all the data that Goodreads has been collecting.”

So is it true that the Amazon algorithm elves are no longer friendly to indies? Do they only like you if you publish through them, or have an agent—or at least give them an exclusive?

I think it's clear that multi-platform indies are not in the Zon's A-list any more.  I’m not sure we’ll see as many 100% self-pubbed authors reach the heights that Amanda Hocking and John Locke did in the heady early days of the "Kindle" revolution. But indie books by previously traditionally-published hybrid authors are still breaking records. And agents are still trolling the Amazon bestseller lists.

Indies are also hurt by the Big Six books that are being offered as cheaply as indies. (There have been amazing giveaways of big name author’s books, especially in the UK. And this week I saw Barbara Kingslover's iconic novel The Bean Trees for $1.99)

Some people think this all means that the self-publishing revolution is dying.

But the fact is, the trend to self-publishing and boutique digital press publishing is not only alive and well, but growing.

This is because Big Publishing has done nothing to change their attitude that the publisher/author relationship must be one of master/slave. They have made themselves less attractive to new writers with ever more draconian contracts and demands that authors hire publicists and provide expensive traditional marketing with their own funds while the publisher risks nothing, sometimes offering no advance.

The fact that Amazon aspires to become a big publisher itself may make Amazon less attractive for the truly independent author.

But the door to successful self-publishing has been opened and it’s not going to close. The Zon may no longer be the fairy godperson of indie self-publishers, but it's not the Great Satan, either.

And more important:  it's not the only game in town. Amazon now represents less than 50% of the ebook market.

Self-and small press published authors who want to be on top of the next trend need to spread their marketing to all platforms, instead of spending all their time courting the Amazon algorithms with freebies and/or organizing armies of fellow authors to give likes and tags and fake reviews.

Launching a book in Amazon's Select program still seems to be a good move. (It works for me, anyway) but after the first three to six months I think most writers need to spread out to other platforms and promote there.

What we do not want to do is keep repeating what worked when the algorithms were the BFFs of the indie authorpreneur and the word "Kindle" was synonymous with "ereader".

The  Indie "Kindle" Revolution may be fading, but the Indie Digital Revolution is still in a dynamic and robust infancy.

What about you, scriveners? Have your sales been affected by the changes in algorithms? Have you heard rumors that you need to do certain things to keep on the good side of the Zon elves? Do you miss the "likes" and "tags"? Do you fear the indie revolution is petering out? 

UPDATE: There is now an algorithm that writes poetry. See, I said those elves are smart! The New York Times now has a Tumbr site where they post algorithm-written haikus based on news stories, called Times Haiku. Here's my favorite. (From an article on Margaret Thatcher's legacy.)

But we are British
So most of us stood around
In awkward silence. 

OPPORTUNITY ALERTS:

1) Readwave: A showcase for short stories: ReadWave is a community of readers and writers who love to discover and share new stories from contemporary writers. Readers can access thousands of stories and read them for free on mobile or desktop--and writers can use ReadWave to build up a fanbase and market their stories online. ReadWave has created a new reading widget, that allows bloggers and website owners to embed stories online in a compact form. The ReadWave widget is the first reading widget to allow readers to "follow" the writer. When a reader follows a writer they are added to the writer’s fanbase and can receive updates on all of the writer’s future stories. ReadWave puts writers in touch with the readers that are right for them. This looks like a great innovative site. You know how I've been encouraging you to write more short fiction? This is where to put it to start building a fan base.

2) The Saturday Evening Post’s Second Annual Great American Fiction Contest—yes, THAT Saturday Evening Post is holding a short fiction contest. Could you join the likes of F. Scott Fitzgerald; William Faulkner; Kurt Vonnegut, Jr.; Ray Bradbury; Louis L’Amour; Sinclair Lewis; Jack London; and Edgar Allan Poe? $10 entry fee Deadline July 1, 2013

3) Inspirational anthology accepting submissions: A "Chicken Soup for the Soul" author is looking for heartwarming inspirational nonfiction pieces. Do You Have a Story on "Staying Sane in the Chaotic 24/7 World"? If you have a great story and would like to be considered for the anthology, 30 Days to Sanity, Send submissions to: 30 Days to Sanity at Box 31453, Santa Fe, NM 87594-1453. Or e-mail stories to stephanie@30daystosanity.com The maximum word count is 1200 words. For each story selected for the program a permission fee of $100 will be offered for one-time rights. There are no limits on the number of submissions. Deadline is May 1, 2013

4) Stanford Story Slam The first ever Stanford Story Slam has opened, a chance for a team of writers to win $500. Anybody can enter. To enter, you must collaborate to write about this prompt: “There are over 15,000 bikes used by students, staff, and faculty to get around Stanford campus. Over 300 bikes are stolen each year. Where do they go?” The Stanford Arts Review will publish the winning entry. Here’s more from the organizers You don't have to be a Stanford student to enter. Deadline is April 22.

5) The 35th annual Nimrod Literary Contest: The Pablo Neruda Prize for Poetry and The Katherine Anne Porter Prize for Fiction. The Awards offer first prizes of $2,000 and publication and second prizes of $1,000 and publication. One of the oldest “little magazines” in the country, Nimrod has continually published new and extraordinary writers since. For more information about Nimrod, visit their website at www.utulsa.edu/nimrod. Deadline is April 30th.

Labels: , , , , , , , , , , , ,