Technology should help publishers refine their literary product offerings

Technology should help publishers make better decisions, not make those decisions for them. [iStockphoto]

One of the most common accusations against legacy publishers, especially where creative works are concerned, is that they prefer working with established writers at the expense of new talent.

The common rebuttal by publishers has, over the years, been that as business enterprises, everything they do must generate returns for the shareholder. So publishers cannot ignore a writer whose works have proven themselves and acquitted themselves well in the market, much as they honour the near sacred duty of discovering and nurturing new talent.

Perhaps the deeper, often unasked question is this: could there be a gem among the untapped talent that could possibly do better in the market than even the so-called established writers? Weren't these established writers themselves, at some point, untapped talent that someone believed in strongly enough to vouch for their works? And given that most established writers are older people, where does prioritising the old guard leave the quest for literary continuity and even business continuity?

These are the questions literary editors have been grappling with for decades. In the legacy book publishing world, the process was rather rigorous. An editor read a manuscript and, if it showed potential, passed it on to a literary adviser for assessment. If the first adviser felt it showed promise, a second and sometimes a third professional opinion was sought.

If there was unanimous agreement that the manuscript had what it took to go places, it was subjected to another test. In some places, where Max Weber's bureaucratic approach to management applied, the editor wrote a proposal for the big boys and girls to approve. In some publishing firms, the big boys and girls gathered in a boardroom to hear the editor explain why the company should invest shareholders' money in the proposed book.

This rigorous process was important for quality and cost management. Even so, as a young man sweating through editorial board meetings, I could still see biases. Now and then you saw someone nodding enthusiastically at a proposal and you could tell the author was a friend. At other times, someone glowered throughout the meeting because they had already decided against the proposed book before the editor started sweating through his or her thesis.

Shortly before publication, everyone gathered in the production studio or boardroom to choose between different fonts, cover concepts and text designs. In a word, the whole process was predicated on the assumption that the team could decide for the market what was good for everyone to read. To be fair to publishers, there was an equally elaborate feedback loop through which firms gathered market intelligence and gave editorial teams insights into what was performing well.

For generations, publishers relied largely on editorial judgement, market surveys and the instincts of experienced booksellers to understand what readers wanted. Today, however, artificial intelligence, machine learning, predictive analytics and other newfangled technologies are opening up entirely new possibilities.

It is interesting to see how publishers will integrate these tools, especially to gain sharper insights into what readers really need. With these technologies' ability to process millions of data points in milliseconds, publishers could work with technology firms to develop models that show them what is making waves in the market, why readers liked particular literary products and what they need to do to improve future offerings. For instance, they could determine the average age, interests, location and reading habits of people who like a certain novel and then market similar products more aggressively to that segment.

I know someone will protest that literary work and its sampling should never be left to robots and bots because poetry and great literature are expressions of the human soul itself. I know there are great works that leave you feeling warm in a way that defies words. I know it sounds offensive to imprison culture and literature in technology models that are often developed with little data from our corner of the world. But come with me, please.

Henry Chakava (GBHS), the undisputed father of East African publishing, often reminded us that publishing occupies the middle ground between the cathedral and the stock exchange. On the one hand, you were doing the soul's, perhaps even God's, work by giving people a platform through which to express themselves and grow culturally.

On the other hand, there was a business to run profitably. If you lost that balance, one critical side of the publishing enterprise faltered. It is therefore on the stock-exchange side of the business that technology should be used to determine objectively what is trending among readers, beyond the consensus reached in a boardroom.

This is already happening in many of our newsrooms. Slightly over a decade ago, a marketer would be shown the front page of tomorrow's newspaper, or a few options, and asked which was likely to sell the following day. Today, however, many newsrooms have large digital dashboards on their walls displaying real-time audience engagement metrics such as page views, unique visitors, average reading time, scroll depth, click-through rates, bounce rates, shares, comments, subscriptions and conversion rates.

Editors use these figures to determine which stories performed best over the previous 24 hours. Using the right algorithms, it is easy to detect patterns and know with reasonable certainty what kinds of stories people want to read on X, TikTok, Facebook, Instagram, YouTube and other digital platforms.

Such technology would not only help reduce editorial selection bias in the choice of manuscripts, it could also guide publishing houses to publish more profitably and therefore have more resources to nurture new talent. It would enable management to know which book series to create, discontinue or strengthen based on objective evidence rather than intuition alone.

It would guide editors on what to commission next. It would also reduce the time it takes to assess manuscripts because technology could determine which submissions pass preliminary screening before being referred to a literary adviser.

Publishers could gather continuous feedback through surveys, reviews, clicks, likes and other engagement metrics. Where people once crammed into a van to visit several counties in search of readers' opinions, much of the preliminary work could now be done digitally. The final literary judgement, however, should remain firmly in human hands.

Technology should help publishers make better decisions, not make those decisions for them.