The State of Book Publishing Part II

This is the second of two posts on the state of publishing. The big disruption has been the rise about ten years ago of self-publishing as a viable financial strategy. In part one I showed that, so far as I can tell with the numbers available, the industry as a whole has seen slowly rising revenues over that time, with about 90% going to traditional publishers and 10% to self-published books, the bulk being fiction ebooks.

Is there a stable equilibrium to be struck between traditional and self publishing, or are we in the early stages of a continuing shift that will eventually result in the death, or at least marginalization, of traditional publishing? I believe that it is the former. There is an equilibrium and we are close to it, if not already there. To understand why, we need to understand this, from The Diary of a Bookseller by Shawn Bythell, owner of the largest used bookstore in Scotland. The book is a delightfully curmudgeonly memoir of a year in the life. One anecdote he relates is when a self-published author requested he stock the book:

I am frequently presented with this sort of thing, and I take it on sale or return for purely diplomatic reasons. Without exception, one year later I end up returning every single copy.

Bythell is not unusual here. Many independent bookstore owners, including of stores that sell primarily new books, report the same thing. Why is this? I am a history guy. When I want to understand the present, I look to how we got here–in this case, the history of prose fiction, and in particular the novel.

I start in the 18th century. This isn’t the beginning of the novel, but it serves my purpose here. It was when the novel became a significant form. The novel was not, however, high prestige. It was what silly girls read, with all the hand-wringing by their elders you would expect about this trash putting ideas in their pretty little heads. Jane Austen, with her keen eye for irony, makes this a central plot point to Northanger Abbey.

The novel, over the course of the 19th century, spread in both directions of the prestige ladder. On the one hand, self-consciously literary writers adopted the form, where earlier they might have written in verse. I am more interested here in the trip down the prestige ladder. The possibilities for earlier novels were limited. The market was limited to persons both literate and the ability to buy expensive books. The 19th century saw broadening literacy, while at the same time the inventions of the rotary press and wood pulp paper drove costs down. The dime novel was born. The end of the century also saw the rise of magazines–only gradually distinguished from weekly newspapers–and the distribution network to get them to the wider public. The early 20th century was the heyday of the magazine. At the top of the prestige ladder were entries such as The Atlantic Monthly. At the bottom were commercial genre fiction on cheap paper, known to us today as the pulps.

The pulp fiction magazine industry collapsed after World War II. The causes include wartime paper shortages and changes in the distribution network. The niche didn’t disappear, but it moved to a new format: the mass market paperback. Like the pulps, these were cheaply made, and their distribution was in some respects more like magazines than like traditional hardback books. Mass market paperbacks ran the gamut of the prestige ladder. You could find an inexpensive paperback edition of a literary work, or the trashiest of trash. There was no clear division between higher and lower prestige books, but there were some general clues. Higher prestige books tended first to be published in hardback, with a paperback edition coming later. The lowest prestige books often were marketed by publisher more than by author (think Harlequin Romances). But apart from cover art, a paperback edition of a literary novel resembled, as a physical object, a commercial genre novel.

This brings us up to the 21st century. The mass market paperback is in retreat. There are various reasons for this, with the traditional explanation of shifts to the distribution network leading the way. What matters here is that just as mass market paperbacks filled the vacated pulp magazine niche, the self-published novel is taking the place of the mass market paperback.

To bring this home, I have three anecdata. The first is from the marvelous essay “Bookshop Memories” by George Orwell, about his time working in retail in the 1930s. He tells how the shop had, as was usual practice at the time, a lending library as a sideline:

Our shop stood exactly on the frontier between Hampstead and Camden Town, and we were frequented by all types from baronets to bus-conductors. Probably our library subscribers were a fair cross-section of London’s reading public. It is therefore worth noting that of all the authors in our library the one who ‘went out’ the best was — Priestley? Hemingway? Walpole? Wodehouse? No, Ethel M. Dell, with Warwick Deeping a good second and Jeffrey Farnol, I should say, third. Dell’s novels, of course, are read solely by women, but by women of all kinds and ages and not, as one might expect, merely by wistful spinsters and the fat wives of tobacconists. It is not true that men don’t read novels, but it is true that there are whole branches of fiction that they avoid. Roughly speaking, what one might call the average novel — the ordinary, good-bad, Galsworthy-and-water stuff which is the norm of the English novel — seems to exist only for women. Men read either the novels it is possible to respect, or detective stories. But their consumption of detective stories is terrific. One of our subscribers to my knowledge read four or five detective stories every week for over a year, besides others which he got from another library.

Note those two lists of authors: The first list is of mostly familiar names. I had to look up J. B. Priestley, but I knew the others, and have no-shit read some of them. The second list are all ciphers. They rate Wikipedia entries, and if you really want to you can find some of their earlier works online. I don’t recommend. This stuff doesn’t age well.

My second anecdatum is that fifteen years ago there was a used bookstore in my town. Its inventory was exclusively mass market paperbacks. Romances filled well over half the store. Mysteries came next. Then there was an entire aisle of Tom Clancy knock-offs, next to a modest section of westerns. The SF section was very small and very bad. (Used bookstores often have trouble stocking a good SF section, as SF readers tend to be hoarders.) Across the board, the sales price was half the cover price, and the store would buy it back for half of that. The store was functionally like the lending library sideline that Orwell described.

My final anecdatum is from Reading the Romance, a classic sociological study of romance readers by Janice Radway, first published in 1984. Her research included a survey of romance readers’ reading habits. The sheer volume of, um…, volumes is impressive. Over half reported reading between one and four romances a week, and more than a third between five and nine a week. Four readers claimed between fifteen and twenty-five. This seems implausible, and Radway is skeptical, but that isn’t the point. Neither are the absolute numbers, lest we get bogged down in discussions of self-reporting, small sample size, and sample selection. What comes through is that there is a body of readers for whom the word “voracious” exists. These are people whose primary leisure activity is reading, and they read a lot. But not, as a rule, prestige fiction. They aren’t reading from Orwell’s first list of authors, who are still read today. They are reading from today’s version of the second list, of forgotten writers.

Once I figured this out, several other pieces to the puzzle clicked into place. When I first got interested in the topic, I started reading blogs from both sides of the industry. They talk past each other a lot, but in different ways. The traditional side can be kind of clueless, like this piece from Publisher’s Weekly wondering why fiction sales are down, meaning traditionally published fiction, seemingly unaware that self-publishing exists.

This is nothing to the sheer vitriol from the self-publishing side. Traditional publishing is regarded with loathing. This extends to bookstores. The response to a news stories about Barnes & Noble circling the drain is dancing on the grave before the body is even in it. A news story about an independent bookstore is met with derision. This had me scratching my head at first. I would have expected even the most enthusiastic user of ebooks to have grown up thinking of bookstores as happy places, having kind thoughts about bookstores in the ideal, even if recognizing their limitations in practice.

It came together when I realized that these are voracious readers, but of commercial genre fiction of the sort where bookstores never really were the point. You might buy these books at a bookstore, but you were at least as likely to buy it at a news stand or supermarket, or off a spinner rack at a drug store. These are book people, but they aren’t bookstore people.

Once I looked at self-publishing in this light, it became clear that there are within the community two groups of authors. One is mid-list or former mid-list writers. “Mid-list” is a vaguely defined term of art, referring to writers who, at least back in the day, could make a decent living publishing a novel a year, but neither they nor their publishers were getting rich off it. The fate of the mid list is an ongoing topic of discussion, with opinions ranging from it imploded in the 1990s to it is still there and doing just fine. If you were a mid-lister and got dropped by your publisher, then the mid list imploded for you. You might go back to your day job, but today you might self-publish. And even if you are still getting traditionally published, you might cross over and dabble in self-publishing, taking home a higher royalty rate. The second group is purely self-published authors. These are writers who have never been traditionally published, and quite possibly never submitted a manuscript.

With mid-listers, past and present, they have name recognition. If they can train their readers to follow them, then they have a natural following. It is the pure self-published that really interest me. Because this brings us to the question of the slush pile. Many junior editors have done a stint where they were assigned to read unsolicited manuscripts, a/k/a the slush pile. They often have horror stories about how awful the experience was. Talk to self-published authors and “gatekeeper” is the word that really makes them spit. In some respects they have a point. Gatekeepers are reflexively conservative, accepting works that resemble books that have sold well in the past. Removing the gatekeepers allows authors the freedom to experiment and fail on their own dime. But removing the gatekeeper means removing the guy paid to wade through the slush pile, meaning that they are asking me, the reader, to do it, and to pay for the privilege. The prospect does not tempt. I, as a reader, want that gatekeeper!

So how does this work? Everyone, after all, has some standard of acceptable prose, if only because the slush pile includes incomprehensible gibberish. Partly there is a thriving online community of reviews. But this can be (and is) gamed, and in any case tastes vary, as do acceptable gibberish levels. The general solution seems to be first to narrow the search by sub-sub-genre. There have always been sub-genres, but they are more narrowly defined. Want regency vampire stories? You’re covered! How explicit do you want it? The range runs from the gently heaving bosom to hardcore pornographic. Next, authors make sampling by the reader a very low stakes decision. That eleven volume (so far) series of soft-core regency vampire novels will have the first volume very cheap, or even free. Finally, the whole thing is very inexpensive compared with traditional publishing. If you read one or two of these a day, you aren’t wiling to put out $20 a pop. Prices for the pure self-published authors run up to perhaps $4 for a new book, and lower for the earlier volumes. (The mid-list types can charge more, having a different audience.)

In this self-publishing environment it is possible to make a living income, but it comes at a cost. You are focusing on a small number of readers who each read a lot, rather than aiming at a large group with lower per capita reading. Once your target audience finds you, they will read everything you write, but there aren’t really all that many of them. With prices so low, this means you have to churn out verbiage at a furious rate. Look up any of these people on Amazon and calculate how many books they are putting out per year. It might actually be considerably more than that, since many use different pen names for different sub-sub-genres. Putting out a book a month is not at all unheard of. Did I mention time put into marketing? No? Best contemplated merely in passing. Burnout is an issue. The first generation of the self-publishing revolution is mostly gone now. The pace is unsustainable for most people, and doesn’t sound at all fun. (It also, by the way, is strikingly reminiscent of accounts by the old pulp fiction writers, though they didn’t have to handle the marketing themselves. The price for the finished book even matches the old dime novels, once you take inflation into account.)

Bringing this discussion around, these two groups of self-publishing authors are both self-limiting. The mid-list group is by definition the product of traditional publishing. I won’t be surprised if ten years from now there is an established career path of starting out in traditional publishing and switching, whether through necessity or desire, to self-publishing. The two forms of publishing in this model overlap and coexist. As for pure self-publishing, it fills the pulp fiction niche, which while significant is not and never has been more than a fraction of the fiction market, much less the book market in general. It is structurally unsuited to go beyond this. The low stakes promise to the prospective reader extends to the prose. It favors the “grab ‘em on the first page” sort of book, and is completely unsuited to the “worth the effort” kind. It will also be interesting to see if there develops a career path from pure self-publishing to traditional publishing. It is very much worth noting that the authors who win the spin of the wheel and make millions are all traditionally published. The cap for self-publishing is “decent middle class income while it lasts.” Is there a path to the big time? E. L. James is the only example, and she wasn’t in the self-publishing world very long. I gather that she has a knack for self-marketing, so if there is a career path to be found here it may be to use self-publishing to hone those skills.

As for non-fiction, it is a different market. There isn’t a market segment reading a book a day. Within my area of sports history there are a few odds and ends that seem to be the product of hobbyists trying to make a few bucks on the side off their interest. There is nothing wrong with this, and I have found a couple of these books useful in my research. As a matter market share, this sort of thing will never by more than marginal.

Final analysis, until the next game-changer: Self-publishing is only a marginal slice of the non-fiction market and is unlikely to change. Self-published fiction is a real thing and not going away, but will never drive traditional publishers out of the market. Much of the commercial genre fiction market is lost to traditional publishers and isn’t coming back. The mid-list is shared by traditional and self-publishing, and will continue to be. The open question is how the market will be divided. My guess is that traditional publishing will continue to lose market share for a few more years, at which point a stable equilibrium will be established.

You heard it here first.