Bitly grew up with the social web. Going on five years in operation, your favorite URL-shortener now sees tens of millions of links shared per day. Those links see hundreds of millions of clicks.
That’s a ton of data—and it’s the job of Hilary Mason, the New York company’s chief scientist, to figure out what to do with it. Making useful things out of data sometimes requires what seems like an unexpected creative leap—the ability to see how a mandate for research on one track can turn into a product on another. In other words, data scientists solve business problems that aren’t immediately apparent, turning research into something unexpected.
Technology, she’s proclaimed, should give us super powers—and she’s a prime example. Having studied computer science and algorithms at Brown and written the book on machine learning, she has also algorithmically uncovered the mediocrity of New York restaurants.
At Bitly, her job, as she describes it in her bio, is a mix of pure research, exploring, and engineering. "My role is chief scientist," says Mason. "What I really do is push potential forward."
The solution-creating work the data science team does opens up opportunities for businesses to solve a problem in the market, Mason says, even it wasn’t immediately apparent. Such was the case for Bitly’s attention-ranking product, Realtime.
Realtime, a product of Bitly-labs, shows what’s being clicked on and shared across the Bitlyverse. There you’ll find a front page with a range of stories that are getting a "disproportionate spike of attention." These will often be celebrity stories—look out for the Kardashians and Biebers—but also unexpected ones. The afternoon of our interview, the trailer for a movie called Elysium came out, generating a huge spike in interest and sending it to the front page.
"The research goal behind that was to know what is happening in the world right now," Mason says. "The product goal was to be able to make that really complicated set of information useful to people."
The interface is simple: If you click on a front-page link, you’ll be taken to the links "story bundle," which is Bitly for taking in a story in aggregate, rather than individual URLs.
You can also see geographic distribution of readers: In that story bundle, you’ll see what percentage of readers are in which country—and the cities within the country that are paying attentions. That geographic awareness also lets you make local queries: In a few clicks you can see what people are talking about in New York, food in Chicago, or in Los Angeles. It’s like a living atlas of the Internet’s attention, born out of years of Bitly’s data development.
"We built a system that mathematically tells us what the world is paying attention to," she says. "Then we built a product that exposes that."
"(Realtime)'s very useful for a problem we didn’t even think about in the beginning," he says, "which is that brands often want to share content around ideas that they want to associate their brand with."
Brands want to be associated with certain ideas or scenes—be it extreme sports, art, or celebrities—and Realtime is a tool that can allow them to see how people are paying attention to those dimensions. The technology beneath the tech demo will be integrated into Bitly’s enterprise suite—which has users from BirchBox to the New York Times to the Dalai Lama. In this way, the insight-generating work that the data science team does builds potential for business.
That’s the art and craft of data science: taking a hunch you have about the world and pursuing it in a structural, mathematical way to understand something new about the world.
"You have to be creative about the choices you make and how you construct those models to get a useful answer and a truthful answer," she says. "Data scientists, as professionals, have that task."
Mason leads those professionals. To do that, she has to know what is and what could be: a mixture of understanding what’s in Bitly’s data, in their engineering, and in the research community—and combining all those possibilities into something that hasn’t been built before.
"We see behavior from billions of people a month," she says. "It really is the scale of human behavior on the Internet."