PyTorch 1.0 brings developers closer to cloud services

While TensorFlow has taken a leadership role in open source deep learning frameworks, PyTorch has gained momentum among developers to build machine learning into their applications.

Facebook provided details on the preview release of PyTorch 1.0, the machine learning library for Python. The 1.0 release brings deeper cloud service support from AWS, Google and Microsoft, as well as tighter integration with technology from Arm, Intel, IBM, Nvidia and Qualcomm.

Facebook developed PyTorch and released it in 2016, a year after Google released TensorFlow. Both frameworks are open source. "Early last year TensorFlow was the presumptive winner of the deep learning framework wars," said Sam Charrington, principal analyst at CloudPulse Strategies, based in St. Louis. "Since then, PyTorch has grown dramatically in developer mindshare."

PyTorch's popularity is driven in large part by a gentler learning curve compared to TensorFlow, in particular a more dynamic and "Pythonic" approach to build neural networks is easier and more intuitive for Python's large community of developers and data scientists, Charrington said. "TensorFlow, on the other hand, while it's most often used with Python, was originally written in C++ and adopts conventions that can feel unnatural to Python developers," he said.

Sam Charrington

A big boost to PyTorch 1.0's momentum is its integration with the top three public cloud platforms. The Amazon SageMaker platform for building machine learning models now provides preconfigured environments for PyTorch 1.0, and Google now supports PyTorch on Google Cloud Platform, as well as integrations across its hardware and software tools, including support for PyTorch 1.0 on Google's custom hardware, tensor processing units. Microsoft also supports PyTorch via its Azure Machine Learning (ML) service, offers preconfigured Data Science Virtual Machines preinstalled with PyTorch, and integrates Azure ML and PyTorch via Visual Studio Code Tools for AI.

Lots of different machine language tools all compete for developer mindshare, many of which are freely available with no vendor relationship, said Ronald Schmelzer, principal analyst at Cognilytica, in Ellicott City, Md. "Keras, TensorFlow, Apache Spark ML, the Jupyter notebook and supporting technology are all free and open source, and the number of developers and data scientists using those platforms continue to grow quickly," he said.

Early last year TensorFlow was the presumptive winner of the deep learning framework wars [but] PyTorch has grown dramatically in developer mindshare.

Meanwhile, the TensorFlow and PyTorch communities jockey for position via their respective project roadmaps.

TensorFlow has a much larger ecosystem with supportive tools such as Keras, a simplified API that makes TensorFlow much more consumable, and TensorBoard, a visualization tool that makes it easy to plot a wide variety of model metrics, Charrington said. "And TensorFlow 2.0 will prioritize 'eager execution,' a mode of operation that's, well, more like PyTorch," he said.

PyTorch 1.0, on the other hand, adds much stronger support for running models in production, previously an of considerable weakness relative to TensorFlow, Charrington said. The fast.ai library, which for PyTorch is analogous to Keras for TensorFlow, also has achieved 1.0 status. And to seed PyTorch's adoption among developers, Facebook has created a PyTorch Scholarship Challenge initiative to make 10,000 seats available in an "Introduction to Deep Learning with PyTorch" course on the Udacity online learning site.