A machine learning model at the New York Times boosts subscriptions by allowing non-subscribers a monthly read limit.

The model – called the Dynamic Meter – sets personalised metre limits to make its paywall smarter, senior data scientist Rohit Supekar says.

In an INMA Ideas blog, he explains that while the publisher introduced its paywall in 2011, the new ‘metred’ access strategy has generated subscriptions while allowing initial exploratory access to new readers.

“In fact, in February 2022, when the Times acquired Athletic Media, the Times achieved its goal of ten million subscriptions and set a new target of 15 million subscribers by the end of 2027,” he says.

With the Times’ transformation into a data-driven digital company, the causal machine learning model is being used to set personalised metre limits.

“The company’s paywall strategy revolves around the concept of the subscription funnel,” he says. “At the top of the funnel are unregistered users without an account. Once they hit the metre limit for their unregistered status, they are shown a registration wall that blocks access and asks them to log in or make an account with us.

“Doing this gives them access to more free content and allows us to better understand their current appetite for Times content through their ID-linked reading history. Once registered users hit their metre limit, they are served a paywall with a subscription offer.”

Supekar says this is the moment the Dynamic Meter model controls. “The model learns from the first-party engagement data of registered users and determines the appropriate metre limit to optimise for one or more business KPIs.”

The model has to play a dual role – helping people understand the world, and the business goal of acquiring subscriptions. “This is done by optimising for two metrics simultaneously – the engagement that registered users have with Times content and the number of subscriptions the paywall generates,” he says.

“These metrics have an inherent trade-off since serving more paywalls naturally leads to more subscriptions but at the cost of article readership.”

A randomised control trial randomly assigns users different metre limits; with larger metre limits, average engagement gets larger and is accompanied by a reduction in the conversion rate for subscriptions.

“Conversely, a larger amount of friction due to tighter metre limits also impacts readers’ habituation and they are less interested in our content. In essence, the Dynamic Meter must optimise for conversion and engagement while balancing a trade-off between them.”

The Meter is a prescriptive ML model that learns the causal effect of assigning different meter limits on each user’s engagement and subscription likelihood. A weight factor combines these two objectives and can be adjusted based on business requirements, allowing the model to change the desired conversion rate flexibly.

“The model is trained using historical RCT data and its performance is assessed by comparing its KPIs with the RCT.”

Supekar says the strategy allows tuning of the level of friction based on business goals and, at the same time, smartly targeting users in order to obtain a lift in engagement and conversion rate compared to a purely random policy.