The RDBMS is Dead

If you suspected the title of this blog is click-bait, then you would be correct.  But bear with me anyway on a little retrospection of the relational database market, where it has been, and where I think it is going to end.

What is an RDBMS?

Anyone reading this blog surely knows the definition already, but let us clarify what relational data management systems mean in contrast to all the data management systems.

Relational database refers to the organization of datasets into discrete tables that are related to each other by common data fields.  This evolved from efforts to improve performance from earlier flat data management systems that took a long time to search.  Modern RDMS vendors really took off after ACID took hold as the governing principle of RDBMS development. Andreas Reuter and Theo Härder coined the acronym ACID as shorthand for Atomicity, Consistency, Isolation, and Durability in 1983.  Oracle, Sybase, and Informix all trace their earliest success to that period as the original RDBMS market leaders.

The ACID standard was key because it made for reliable data transactions that could be the basis of business applications requiring both speed and data reliability and integrity.  Durable transactions was the key ingredient that enabled ERP applications built on these enterprise-level databases.

So, why would anyone entertain the thought of even considering whether the relational database management system is dead given that success?

Hype and Reality… and the Reality of Hype

I began thinking of this subject a couple of years ago when a colleague in a group meeting nonchalantly declared that none of the hip voices in application development were talking about databases anymore (at least not the traditional variety).  No, the new hotness was around NoSQL, vastly scaling architectures that didn’t impose the tiresome restrictions on data stores for developers.   There was lots of chatter and excitement about this technology that were growing as people experimented and rolled out a different types of applications made for this purpose.

It was hard to contradict this exuberance for the obvious growing popularity among technology thought leaders without seeming to be a curmudgeon.  And his expressed opinion came from the chatter among developers eager to be at the forefront of the next technology explosion. There was definitely something there, but how would it play out and when would it play out?

Gartner coined the concept of the technology hype cycle from the repeated patterns of introductions of any revolutionary technology.  And the central idea is that the hype on these new concepts always gets ahead of productive usage of the technology.  The buzz is there, usually spawning start-ups galore in a technology gold rush to be the first to monetize that promise and ride a wave of popularity to riches.

Of course, the cycle part of the hype cycle implies that though the excitement may persist, the efforts to monetize the new thing can fail to meet inflated expectations.  The trough of disillusionment comes from failure to make the revolution a reality as fast as hoped.  There is usually a shake-out of the start-ups as the market doesn’t grow fast enough to sustain the initial enthusiasm. 

If the technology in question isn’t overtaken by something better to replace it, and if investors can weather that trough, then we gradually learn productive ways to make the technology cost effective.  Only then does the technology hit the plateau of productivity which implies it will spread and have a lasting effect.

NoSQL to NewSQL to Hybrid to “Nontraditional”

Although the NoSQL term was first coined by Carlos Strozzi in 1998 for his open-source relational database, the NoSQL databases that gained such popularity after 2009 were different.  Where Strozzi NoSQL merely excluded the Structured Query Language interface, most databases called NoSQL in 2009 to present depart from the relational model completely.  (Strozzi suggested they would more properly be called NoREL for ‘No Relational’ but the NoSQL name was already in common usage.)

The excitement over these NoSQL databases came from how they overcame scale limitations of traditional RDBMS, enabling applications that could also scale and without the restrictions of a fixed schema.  It enabled frequent changes to the databases adaptive to changes in the calling applications.  The downside was transaction integrity wasn’t as reliable and applications had to be written with a particular data store in mind with no standard interface or language.

NewSQL was coined for the class of RDBMS seeking to get the scalable performance of NoSQL but maintaining the ACID transaction standards and SQL interface of traditional RDBMS.  Soon NewSQL and NoSQL were being lumped together in market discussion with columnar databases, in-memory databases, and termed as Hybrid databases or currently “Nontraditional Databases”.  That term covers a wide variety of specific-purpose data stores uniquely suited to solve various problems where traditional RDBMS didn’t work as well.

So, while the hype and buzz of these new options to traditional RDBMS is not to be denied, it is getting hard to identify a unifying and superior aspect to them like RDBMS had with the ACID standard.

Familiar Patterns

I am not sure of the original source, but I once read that one sign that you are getting old is that everyone starts to look familiar. And if you have been embedded in particular markets long enough then you will start to recognize other familiar patterns.

History doesn’t repeat itself; but it rhymes.

Back in the day when the “Big Three” in the RDBMS world referred to Oracle, Sybase, and Informix there was a time when OODBMS (Object-oriented Database Management Systems) was the new hotness and all the rage in discussion groups.  IBM held a legacy position because of mainframe DBMS and Microsoft was an upstart selling rebranded Sybase SQL Server to the entry market.  Both IDC and Gartner covered OODBMS as a new category and the chatter in some discussion groups was whether OODBMS would completely overtake traditional RDBMS with the advantages it had for object oriented application development.

This chart outlines the evolution of database systems through the last decade.  But it doesn’t cover the individual vendors’ success or failure over this time.

The yearly major analyst reports for DBMS tracked the rapid growth of Microsoft SQL Server as it reshaped SQL Server into its own vision of a Microsoft-centric IT back office.  Sybase and Informix marketing failures and missteps took them out of the leadership group with Oracle and the Big Three became Oracle, Microsoft, and IBM; leaving Sybase and Informix with large installed bases and particular vertical markets but little growth.

As these DB market shifts happened, due more to marketing successes or failures than technical superiority, the leaders adopted OODBMS features into the traditional RDBMS offerings so they could easily accommodate new demands for data types.  The pure OODBMS offerings fell back into a few niche offerings for special applications.

Since that time, IBM bought Informix to consolidate further and SAP bought Sybase, primarily in a defense of Oracle aggressively pursuing SAP ERP customers. SAP ASE is still a solid technical offering as a back end for SAP applications, but unlikely to recapture a leading independent RDBMS market position again.

Meanwhile, back at the ranch…

Let’s try to put some quantitative perspective to the ‘new hotness’ hype I mentioned earlier about all the popularity around new data store offerings like NoSQL.  It is tough to measure “hype”, but DB-Engines does as good a job as any I have found.   DB-Engines Ranking

This recent graph from the DB-Engines site shows the trend of scores of all sorts of data store offerings.  Rather than attempt to measure revenue results or analyst opinions, it relies on scoring from publicly available information, searches, job offers, mentions on websites, etc.  It doesn’t measure concrete factors such as installations or revenue and therefore positions the rank as an early indicator of future success.  (More detail on how the score is calculated and corrected for comparison is on the DB-Engines website.)

Current DB Market Landscape

Interpreting the chart and the trend of “popularity” is revealing.  Three traditional RDBMS engines are flat, unchanging, and boring.  Oracle, Microsoft SQL Server, and MySQL don’t show much change in popularity over the past few years.  But they all are clustered together far above all other entries.  I suspect if we did find a way to incorporate revenue or actual IT installations that this separation would be even more pronounced.

But the growth in the trends indeed comes from NoSQL offerings like MongoDB, Cassandra, etc.… 

Meanwhile, back at the ranch …Part Deux

So what do the traditional market analysts think about the future of RDBMS vendors?  I must admit that I had not followed them as closely as the year-to-year reports became very similar with Microsoft and Oracle solidifying positions as the clear gorillas in the database market.  They use that leadership position to probe adjacent markets in applications and infrastructure.

A few years ago, Gartner even stopped reporting on the “RDBMS market” in favor of a category called “Operational Database Management Systems”.  This new market category encompassed both traditional RDBMS offerings and all the non-traditional DBMS that had enjoyed the increase popularity in technical discussions.

The Gartner Magic Quadrant shows how Gartner ranks all these vendors relative to one another based on not only short term results but also vision and strategy.

The first thing that sticks out in this Magic Quadrant ranking is that the old RDBMS market leaders are also the Operational Database market leaders.  As would be expected, they rank above many of the newer offerings in ability to execute.   But what is striking is both are also far ahead of the new contenders in completeness of vision as well.  Now this only represents one analyst company’s consensus opinion, but they clearly don’t see any in the Challenger group being an immediate threat to the big RDBMS vendors, why?

Summary and Prediction

The answer to my question is only my own opinion, but my guess goes to echoes of what happened with OODBMS and RDBMS.  The big vendors gradually found ways to incorporate the best parts of the OODBMS while maintaining the stability of the RDBMS for businesses.  While niche players persist there for special purposes; there are few applications that cannot be supported by one of the big RDBMS offerings.

I predict Microsoft and Oracle will continue to mitigate the RDBMS limitations by adding features that were advantages of NoSQL and NewSQL contenders, just enough to make the other transactional advantages of traditional RDBMS to keep the majority of market revenue.  Acquisitions and consolidation is inevitable and these two leaders are the most likely to consolidate power that way as well.

 As the distinction between different DBMS offerings are blurred, the old leaders will keep that leadership until they make major mistakes.

Long Live the Operational Database

Pardon any skepticism about all the new developments in NoSQL (NoREL), NewSQL, etc.   It isn’t that I don’t believe they are important developments that enable capability beyond what we had.  But I doubt we will see new DBMS market leaders coming purely from technical superiority.  The RDBMS leaders are Operational Database market leaders.  And I don’t see evidence of that changing anytime soon.

The RDBMS is not so much dead as rebranded.

Assimilation has begun.