When planning a data warehouse, define before you design
Storage options abound when it comes to processing, managing and deriving the most value from floods of structured and unstructured data -- among them, data warehouses, marts and lakes. Each of these repositories has reasons for being, depending on the data's source, type, status and purpose.
Take data warehouses. When opting for a data warehouse to support fast analytical queries and reporting, one size does not fit all. So a clear sense of purpose and therefore custom design are critical. Poor data warehousing design can lead to the collection and use of inaccurate, inconsistent or incomplete source data, placing a company behind the proverbial business intelligence eight ball in strategic planning and operational decision-making.
A well-designed data warehouse, according to independent BI consultant and data warehouse developer Chamitha Wanaguru in a recent Toptal blog post, includes support for self-service BI to avoid dependency on IT and empower business users, low maintenance costs to offset high initial costs, flexibility to adapt to changing business demands and that all-important value by demonstrating early on the benefits of a data warehouse to key decision-makers and stakeholders.
Yet data managers at times struggle to define the repository that best fits their storage and analytics needs. That indecision is not entirely their fault. Oftentimes, the lines between storage options are blurred, and part of the blame falls on the shoulders of vendors as they jockey for a favorable position in a fiercely competitive and shifting horserace.
This handbook promises to solve the great data management identity crisis so companies can derive the most value from their warehouse. First, we examine companies that favor and opt for data warehouses, why they do and how they use that processed data. Next, the cloud is redefining the approach to warehouse design. Finally, a major vendor's big data platform reflects the burgeoning demand-then-supply approach to data warehousing in the cloud.
