OpenStack release sees drop in storage vendor contributors
OpenStack's Cinder, Swift and Manila storage projects experience a decline in vendor contributors heading into the upcoming release of the latest iteration, Pike.
OpenStack's Swift, Cinder and Manila storage projects are forging on with new capabilities, but contributions have declined, as vendors turn their attention to hotter technologies.
At the OpenStack Summit last week, John Dickinson, the project technical lead (PTL) for the Swift object storage service, said Swift remains an active and large open source project, and it had 120 unique contributors for Ocata, the latest OpenStack release. But he noted the slope has recently trended downward for Swift, one of the founding OpenStack projects seven years ago.
Dickinson, who is director of technology at San Francisco-based SwiftStack Inc., in addition to serving as the Swift PTL, said there could be many explanations, but the most obvious one is several companies have shifted their focus away from Swift and OpenStack.
"And that has hit the Swift community contributor base hard. We've had people who have lost their jobs or have been reassigned to other tasks," Dickinson said. "So, what are we going to do about that? It is something that is one of those things that keeps me up at night, so to speak."
Sean McGinnis, PTL of the OpenStack Cinder block storage service and principal architect at Huawei, said there are also fewer vendors contributing to the project he leads. McGinnis speculated that storage vendors are either refocusing their attention on newer technologies or lack interest in continuing to maintain their drivers as Cinder evolves.
"I think vendors are getting pulled in multiple directions now. Certainly, Docker and Kubernetes are the new hotness," McGinnis said. "I know of a few developers that have at least temporarily been pulled off of working on OpenStack to do things like write volume drivers for containers. Other vendors are just deinvesting, it seems, as some of the hype around OpenStack dies down."
He said many vendors have established support for OpenStack, while they may have no customer options yet for containers. Cinder has more than 100 volume drivers -- a total that includes multiple options from a single vendor that might have a Fibre Channel, an iSCSI and an NFS driver, according to McGinnis.
McGinnis estimated 60 unique vendors have Cinder drivers, but only about 20 remain actively involved in the OpenStack project through code reviews, driver updates and other activities. He said he expects to see less involvement from storage vendors in the next OpenStack release cycle, called Pike.
The OpenStack community releases new software versions twice a year, and Ocata is the 15th release of the open source software for building public and private clouds. The upcoming Pike release is due on Aug. 30, and the Queens release is expected in the first half of 2018.
Ben Swartzlander, PTL for the OpenStack Manila shared file systems service and architect at NetApp, based in Sunnyvale, Calif., said the "depressing trend" has seen a few vendors drop out of the Manila community entirely and others reduce their involvement. He said waning involvement could be due to the "market shaking out the winners and losers in the OpenStack space.
"Companies that don't see an ROI from their involvement will leave, while those that do merely continue with flat investment," Swartzlander said.
Swartzlander said new contributors are joining the Manila project, but net contributions are "a bit down" in Ocata, compared with the prior Newton and Mitaka OpenStack releases. On the plus side, he said the newcomers are largely involved with the distributions, and their focus tends to be stability and usability more than "cramming a bunch of new features in."
OpenStack storage adoption
The OpenStack Foundation's April 2017 user survey showed Cinder had the most production use among the three OpenStack storage projects, followed by Swift and Manila. Swartzlander said adoption of Manila and Cinder continues to increase, although there tends to be consolidation on the vendors that users select.
Companies that don't see an ROI from their involvement will leave, while those that do merely continue with flat investment.
Ben SwartzlanderPTL of OpenStack Manila and architect at NetApp
Overall OpenStack adoption showed no sign of decline in the April survey. The latest user survey reported 538 deployments, compared to 405 in April 2016, when the survey had its largest input to date. The emerging technologies of greatest interest to OpenStack users were containers, software-defined networking, bare metal, hybrid cloud and platform as a service.
It's unclear if declining vendor contributions are having an impact on OpenStack storage deployments. But one company that presented a case study at this week's OpenStack Summit noted problems with some Cinder drivers.
China Mobile's Gangyi Luo, who has done OpenStack-related work since 2014, said the company encountered "serious problems" with data corruption with its OpenStack deployment. He said China Mobile uses OpenStack to manage its VMware, SAN and networks, and to integrate with its existing monitoring system.
"We have aggregated almost all popular SANs," Luo said, "and we can say none of these SAN drivers is perfect." He cited SAN problems such as incomplete functionality, the wrong multipath device and overloading the storage port.
Luo said China Mobile primarily uses the Kilo version of the OpenStack release cycle, but he said the company tested Ocata and found "some problems still remain" with Cinder drivers. He said the company worked with its SAN providers to fix most of the issues. China Mobile uses OpenStack-based storage for its public cloud, and some internet users suffered data loss, he said.
After viewing the China Mobile presentation, McGinnis said that several bugs have been fixed in the multipath area since the Kilo OpenStack release, and a known issue is being addressed with the attach-detach interaction between the Nova compute and Cinder block storage services. He said other issues at China Mobile could be related to a misconfiguration or a misunderstanding of OpenStack functionality. He also acknowledged that no Cinder driver is perfect.
“We can fix some things in the general Cinder community if we can see what an issue is in the driver’s code,” McGinnis said. “But if it’s something specific to how that SAN device works, then they really need to talk to the vendor.”
Swartzlander said Cinder and Manila are the two OpenStack storage projects with drivers for different storage controllers, whereas Swift provides a self-contained service on its own. He said he could not comment on the SAN drivers in question, but he has seen the quality vary among drivers, with some good and others bad.
"It's open source software. A lot of it is supported by the community. And in some cases, a vendor's involvement in OpenStack is one engineer. Sometimes, it's even a contractor or a part-time guy who got hired to go write the driver, and he did that and contributed it, and then he disappeared. We've seen that kind of thing happen," Swartzlander said. "It's 'buyer beware.'"
Swartzlander said that's why many organizations opt for a supported OpenStack distribution, such as those from Red Hat or SUSE, and use only the drivers the vendor has certified.
"That's one of the values that distributions provide. They do additional testing above and beyond what the community does, and they'll stand behind it," Swartzlander said. "When something goes wrong, you have someone else you can go to who's an expert who can provide the additional quality and support."
