NicoElNino - Fotolia
Wayfair's chief architect talks AI-driven innovation, impactful IT
Wayfair sells home furnishings, but under the covers, it's a tech juggernaut. Chief Architect Ben Clark explains how AI-driven innovation propels the business.
To compete in today's fierce retail environment, Wayfair Inc. has to think and act like a tech company, investing deeply in artificial intelligence technologies and other emerging tech to power its business.
Case in point: The Boston-based company in 2017 launched its Search with Photo feature allowing shoppers to use photos of desired items to find the same and/or similar products on its website. This visual search feature, although easy for consumers to use, needs a complex neural network to power it -- one that's robust and reliable, yet also capable of supporting such innovative functionality.
Delivering this AI-driven innovation falls to Wayfair's chief architect, Ben Clark.
Clark joined Wayfair in 2011 as head of customer recommendations. In that position, he worked on the consumer-facing side. He became chief architect in 2014, and, as such, now has responsibility for operational engineering as well as storefront-side and infrastructure-side engineering. He oversees 250 staffers, nearly all of whom are systems engineers and programmers and who work in an environment that spans three colocation facilities and the Google cloud for a few specific workloads.
The company employees about 6,800 people total, which includes more than 1,200 engineers and data scientists.
Clark recently shared his insights on his position, the company culture, software tipping points and the pursuit of AI innovation.
What's your role as chief architect?
Ben Clark: Chief architect is the most fluid of the commonplace chief titles in the industry. It can mean a wide variety of things at different companies, and I have responsibilities not frequently tied to that title. But it's basically about architectural direction, road-mapping and constant probing for soundness of the ideas that we're turning into living, breathing systems at Wayfair.
There are chief architects who are very controlling, but that's not really my style. I think we get a lot of benefits from the bottom-up innovation that comes from all the members of our team. But we do need to get some focus and synergy and make sure all the different threads are coming together to participate in our strategic goals of delivering a terrific experience to all our customers.
Quick aside: IT often uses the word customers for multiple groups of end users, so which group do you mean?
Clark: I mean the people who are shopping on Wayfair. Sure, a lot goes on the back end of that. There are all kinds of things we need to do; it's not just the [shopping] surface [Wayfair teams] interact with. But we try to make sure what everyone is doing is connected to the [consumer] customer. It might be indirect for some groups, more direct for others. If they don't understand that, they might miss opportunities to keep their work aligned with that.
Can you explain more about how you achieve "architectural direction, road-mapping and constant probing"?
Clark: This is the way I think about it: As we grow bigger and bigger, our testing efforts become more and more important to everything we do -- and we do them on a year-round basis. We run two sets of things: what you could call pressure tests and tests where you remove a piece of technology infrastructure to make sure your systems can handle a situation where a component becomes unavailable. That's basically how we make sure that we don't introduce a weakness into the things we're building. If you don't test this way, you can be blind to these things and they can bite you in unexpected ways.
Are the pressure tests and the tests when you remove a piece of infrastructure all about making sure the Wayfair systems are always on?
Clark: Yes, it is, but the tricky part of that is you can have an extremely available system where you don't move very fast. If you don't make any changes, you would be available, but that's not what you want. The trick is to have a continuously available set of systems, for our customers and our internal people and partners, yet to keep up the pace of releasing software many times a day.
So high availability and innovation?
Clark: Yes.
Manager-doer culture
Wayfair advertises that it has a strong 'manager-doer culture.' How does that work in your organization?
Clark: There's nothing mysterious about the term. It means we have an expectation that our managers can sort of ladder down into what their groups are doing, that they're able to not just manage by numbers or their interpersonal relationships but also dig deep into the technology stack and everything going on in their groups.
Not all technologists make great managers, so how do you ensure yours make the transition?
Clark: If we're going to do that manager-doer thing, we start with a pool of people who are excellent technologists. There are some who want to stay on a pure technology track and some who want to go into management over time. We don't promote someone and expect that they'll know how to do it. There's a whole other set of skills that we take seriously, and we have a training program, a curriculum, and informal groups where they learn how to do that [for example] by learning to run a good staff meeting and raise their game in terms of both verbal and written communications.
Boston's biggest tech company
Wayfair promotes itself as a 'tech company that happens to sell furniture,' and the company was recently named Boston's biggest tech company. Why identify as such?
Clark: First and foremost, it's very sort of fact-based; Wayfair has been an e-commerce company with no brick-and-mortar stores since 2002. It's always been this kind of collaborative partnership between the business strategy side and tech side on an equal basis.
In terms of what else makes us a tech company, we have a very strong sense of continuous integration/continuous deployment, of releasing software many times a day and embracing these families of techniques that have become very widespread in recent years. We started doing these very early in our development as a company, and it's really branded the kind of feel of the place.
I think it is an accurate descriptor and it is certainly true when we're presenting ourselves to the group of people who might consider taking a job at Wayfair. We want to portray the way things are, because we do think it's more appealing than if we were a different kind of company behind the scenes.
What kind of workers do you hire to fit your corporate culture?
Clark: We're looking for people who really feed off of the impact that their code has, or the impact the other work they do will have, and so although we have some technology that's very much in the forefront, that's very hot right now, that's kind of an outgrowth of the focus on impact.
So, we try to attract those people. We try to reward those people when they come to Wayfair and when what they do has a high impact. That's kind of a virtuous circle: rewarding people for thinking about the work they do and thinking about the impact it will have on the lives of the customers and those at the company and the suppliers.
AI-driven innovation high on 2018 to-do list
What's on your agenda for 2018?
Clark: We're certainly going to continue to push the consumer-facing side of things in areas of visual search and AR/VR -- augmented reality and virtual reality. But there's a broad range of things we're working on [that enable those areas]. And I'm going to continue to push the testing based-program both for capacity planning and soundness of the systems we're standing up.
Is there a disruptive technology on the horizon you're watching?
Clark: I already mentioned visual search, and you do that with a deep learning-based approach. That's a very fast-moving area we're taking good advantage of already and will continue to keep an eye on and use elements from the research community that become practical.
Speaking of AI-driven innovation and emerging technologies, how do you determine when you need an IT refresh?
Clark: First of all, we're never done. Do we rest? Maybe the day after Cyber Monday we rest a little, for 24 hours, and then we're ready to go for another year of intense activity.
Joking aside, what do we think about when we're going to switch? Consider our recent move to ReactJS (a JavaScript library). We've switched JavaScript frameworks three times since I've been at Wayfair, so that's a thing we do relatively frequently, but in all cases, we came to the decision that while the [existing] framework helped us solve the problems at that time, things change so fast in both the problem space and the technology that you get to a tipping point where there's a big upside to switching and the downsides of switching are starting to diminish.
If it's a large effort, you have to be deliberate because any time you spend switching out, it's time you're not spending on high-value features. But there are plenty of times when we think about those factors and we make a decision to make a move.
How do you think about legacy in the enterprise?
Clark: I don't use that word much or think about it that way; I don't find it adds much to the conservations about technology. The choices you made in the past are either going to continue to line up well with the emerging problems you have or not. There's some code at Wayfair that's been here since 2002, and if it ain't broke, we're not going to fix it. But on the other hand, the wants and needs of our customers and the competitive landscape change so rapidly that we are constantly adding new things and making significant modifications to other things that have been working well for a long time.
