KOHb - Getty Images
AI hardware and software: The key to eBay's marketplace
At the AI Hardware summit 2021, an eBay exec discusses how its similarity system enables it to build an AI-driven marketplace for better latency and throughput.
The most efficient way to build an AI-driven marketplace is to integrate AI software and AI hardware.
That was the message delivered on the first day of the AI Hardware Summit 2021 in a virtual keynote given by Selcuk Kopru, eBay's head of machine learning (ML) and natural language processing.
For a giant vendor like eBay with 159 million active buyers and 19 million active sellers, AI is depended upon to give both buyers and sellers a seamless e-commerce experience.
According to Kopru, to build a global marketplace driven by modern, powerful and scalable AI services, enterprises must optimize systems after training models specifically for their AI hardware.
"Especially for online use cases, we need to be extra careful in building efficient and lightweight models that are optimized for the right infrastructure," Kopru said.
A similarity system
The marketplace vendor uses a similarity system to integrate AI hardware, including GPU and CPU chips, with its machine learning and other AI algorithms.
The reason for building a similarity system is to achieve scalability and performance. Similarity is the relationship between entities based on their search rankings and other variables.
For the consumer, the similarity can be between a text query and a title. For the seller, it can be between an image and a title. The similarity system improves the listing appearance for sellers because it helps eBay suggest accurate titles to sellers based on the image they have uploaded to the platform.
The first step in building a similarity system is to learn the representations or the embeddings using large transformer models, or a type of deep learning system.
After that, eBay builds a kNN (k-nearest neighbors) index -- a type of similarity system that is used offline during index building.
In the online retrieval stage, the query goes through the learning model and vector embedding is computed. The vector is the input into the kNN index. From there, the kNN search is retrieved from the index.
Using the similarity system helps accelerate latency and throughput performance.
Representation learning
EBay starts the representation learning process with the E-BERT ML model, using a large amount of data. To train the transformer model, eBay uses GPUs. EBay also uses AI chips for inferencing.
Kopru said one way of controlling the cost of training the transformer model is to ensure applied researchers across the company have access to all pretrained models.
"Every applied researcher, every engineer can access and start fine-tuning the pretrained models with only a few lines of Python code," Kopru said.
For quick online transactions, eBay uses a teacher-student ML model, which enables it to build much smaller models with some accuracy tradeoff.
"This tradeoff is a good one, based on the result we're getting," Kopru said.
