Emerging database technologies hit DBMS mainstream, add to choices

A handful of unconventional database technologies are starting to have a noticeable impact on the IT industry while challenging the traditional ways that relational database management system (RDBMS) software is developed and used, according to analysts and IT professionals.

NoSQL databases, column-based database software and Database as a Service (DaaS) technologies may not be entirely new, but they have been getting an increasing amount of attention recently from software vendors and potential buyers alike. And the main reason for the growing interest in the emerging database technologies can be summed up in one word: speed.

Many IT departments are finding themselves having to balance the complexity of managing fast-growing information stores with the need to deliver reliable business-critical transactions and business intelligence (BI) query results more and more quickly. In the burgeoning era of “big data,” experts say, columnar database engines, NoSQL software and cloud-based DaaS technologies hold the promise of relief in the form of faster performance, greater flexibility and, potentially, lower total cost of ownership.

Analysts caution that the much-hyped technologies are still very much in their early adopter phases and that they aren’t right for everyone. But interest in them is expected to continue growing as information management requirements become even more complex and time-sensitive – and as the performance and scalability ceilings associated with traditional relational database software drive users to seek alternative options.

“The limitations of relational databases have been a great driver to think differently,” said Noel Yuhanna, a database technology analyst at Cambridge, Mass.-based Forrester Research Inc.

Column-oriented databases, which typically are used in BI and analytics applications, store data vertically in table columns instead of in rows, as conventional relational databases do. Analysts say the columnar approach minimizes the time it takes to read information stored in the database, potentially resulting in faster query times.

Emerging database technologies in action: columnar DB opens new doors
The use of a column-based database is opening up possible new business opportunities for Provisio Inc., according to Sean Harrison, the Nashville, Tenn.-based company’s chief security officer and senior information architect.

Provisio is the creator of iTrials, a service that enables doctors and pharmaceutical makers to search the medical records of about 70 million people in an effort to find candidates for clinical trials. Harrison said it’s a highly data-intensive process that generally involves searching for a set of people with a specific illness, then comparing the records in an effort to find individuals or geographic areas that meet certain criteria.

The limitations of relational databases have been a great driver to think differently.

Noel Yuhanna, analyst, Forrester Research Inc.

Not long ago, the iTrials service hit the limits of its traditional database infrastructure. Harrison said that queries were taking longer than usual to run and that performance bottlenecks were popping up all over the place. Instead of throwing more row-based relational databases at the problem, Provisio opted to purchase a columnar database and some new servers. “We were looking for solutions that would let us get to the data faster,” Harrison said.

It took a great deal of effort to get ready for the new hardware and software combination, he noted. For example, internal applications had to be customized in preparation for the change, and the company’s data center had to be updated to accommodate the new hardware. But overall, Harrison said, the project has turned out to be a rousing success.

The new technology allowed Provisio to reduce the number of tables in its database from about 230,000 to just 12, and Harrison said queries that used to take 10 minutes to run now take 30 seconds. The increase in computing speed has Provisio thinking about new ways to create revenue, he added. For example, company officials are considering the possibility of offering a service to help lawyers and alleged victims in medical-related legal cases to locate other people who might want to take part in class-action lawsuits.

Forrester’s Yuhanna said Provisio is a prime example of a company that is right for column-based database technology because its processing needs are so data-intensive. “Traditional databases are not really meant for their level of scale,” he said.

NoSQL among key emerging database technologies, despite lack of frills
Yuhanna and other analysts say the growing category of NoSQL databases includes DBMS products that don’t make use of the popular SQL programming language or that simply differ from traditional relational databases in a significant way. Some are entirely non-relational, while some avoid selected relational functionality such as fixed table schemas and join operations.

NoSQL databases tend to be “no frills,” Yuhanna said; as a result, users can often get up and running for a relatively low cost. But he cautioned that there is still much room for innovation in the NoSQL realm, particularly in areas such as ease of use and manageability.

“Right now, it’s a bit difficult to implement NoSQL,” Yuhanna said. “The application developer needs to do a lot of work around making sure that this data is all accessible and retrieved correctly.”

NoSQL databases are primarily designed for very high-performance situations, such as online financial services or gaming uses, according to David Menninger, an analyst at Ventana Research Inc. in Pleasanton, Calif. The technology is best suited to “companies that have to process data so quickly and at such high volumes that they can’t incur any additional overhead beyond just exactly what they need to be able to do,” Menninger said.

How Database as a Service and data services differ
Two other database-related technologies that are making waves these days include Database as a Service tools and data services offerings. While those two categories sound like the same thing and are similar in nature, analysts say that it’s easy to get confused about their differences.

Data services, also known as data virtualization or Information as a Service, involves the process of virtually housing data from multiple systems in a semantic or middleware layer that can be accessed by business applications. The idea is to reduce error-prone data replication processes and increase data integration levels without requiring the involved information to be combined in a single database.

DaaS involves taking the database itself and running it in a virtualized environment, as in the case of a cloud database. Both approaches can provide increased flexibility and help reduce an organization’s computing footprint because database workloads aren’t confined to any specific hardware and can be allocated more efficiently based on available resources, Yuhanna said.