Big data and analytics advice from an operations pro

More and more IT shops have to figure out how to manage big data -- not to mention where to store it all.

With nearly 450 million app users every month, you can bet Tapjoy Inc. has a lot of information to sift through. But proper planning and smart buying helps the company's IT team use big data to improve gamers' experiences.

The mobile app technology company, headquartered in San Francisco, is a platform for mobile developers that ensures the right ad pops up at the right time on online game applications. Tapjoy determines if the application needs modification based on the information it receives through the data lifecycle.

Weston Jossey, head of operations at Tapjoy, runs the team that builds and maintains its infrastructure. Here, he shares best practices to implement and maintain big data and analytics technology.

What do you use big data analytics for?
Weston Jossey: There are 450 million monthly [game application] users and there is tons of information. There is an endless stream of data. We need to consume as much information about customer behavior as possible, and we can solve that in big data solutions.

It is important to show the right ad at the right time and provide insight to the publishers. [With big data analysis], we can find out where customers are getting frustrated in the game.

What type of server infrastructure supports data analytics systems? 
Jossey: We use a high-density setup; 12 nodes, 3U configuration in our facility in D.C. for our big data infrastructure. [Tapjoy uses] technologies from big companies like Hadoop and Spark. The Intel 1265 [processor] series is low-power and allows us to put more servers in a rack, for example.

What are some best practices for IT infrastructure that supports big data?
Jossey: Flexibility. We knew that infrastructure changes frequently due to [equipment] refreshes, and we wanted to maintain horizontal scalability, so we designed with that in mind.

Weston Jossey

Our data center is next to AWS, so we can go hybrid if we need to expand.

How do you plan for storage needs of big data analytics?
Jossey: Look at congestion patterns. Extrapolate what the user acquisition is and build a model 12 to 18 months in advance.

In general, we take our current usage and find a key metric -- or set of key metrics -- that influence growth. We then create a formula based off of sales or revenue projections that might influence that key metric to make 12- and 18-month projections. 

With those projections, we create scenarios. One: Stay the course, don't buy anything new, just keep doing what we're doing. Two: Buy for a 12- to 18-month projection. Three: Buy for a 36-month projection or expected massive growth.

One almost always ends up picking [the second] because it's a decent hedge that is the most likely to hit. Sometimes we'll choose [the first] because we don't feel confident about our projections, so we'll come up with workarounds to deal with any capacity issues. Rarely we'll pick [the third], mostly because the cost is so expensive and the bet is so large that any potential savings are outweighed by dramatic risks.

Do you need an IT person who specializes in big data support?
Jossey: Nobody at Tapjoy falls under that IT role; we have DevOps engineers who run the infrastructure. Our engineer knows how to run Hadoop and install things in different components. It's a collaborative process; everyone works together. I think it's a better model.

How do you control IT equipment costs?
Jossey: Extremely diligently. We are always looking at how to trim costs and optimize. [We also control costs with] virtualization. We mix and match technologies and don't overbuy anything. We have a three- to five-year plan and buy the gear all at once. We bought what we knew we needed for day one. The flexibility of public cloud drives down cost.

Most modern companies need a strategy for why they use big data. They scour over data and they hire data scientists and analytics [specialists]. Big data helps us make every decision.