What is Deep Analytics?Decision support experts know that deep analytics is about more than only “big data” or “predictive analysis.” Rooted in scientific, financial services, pharmaceutical, biomedical industries and routinely encountered in telecommunications, ad services, and behavioral modeling, it is commonly found across the entire spectrum of decision support disciplines, data warehousing environments, and Business Intelligence methodologies. In fact, it is likely you are already dealing with the challenges of deep analytic queries today.
Deep analytics is about micro-targeted, granular investigations of millions or billions of individual data elements to discover patterns and behaviors. It addresses expanded dimensionality of interrelated data variables to provide laser-focused insights for analysts and decision makers. Therefore, the notion is equally relevant when either analyzing, monitoring and reporting what has happened across many dimensions or when predicting what might happen across and within them in the future. These needs are pushing decision support professionals to adopt a new database paradigm, as traditional RDBMS’s fail to scale.
Challenges of Traditional Database Technologies
Database and data warehousing experts are facing increasing challenges in adapting older database technologies to rapidly growing data requirements and the never ending demands of analysts. These experts create and recreate data partitions, application shards, and table indexes; all while constantly tuning the performance of their BI and analytic solutions, load balancing their database and storage servers, and defending expensive maintenance plans and upgrades to skeptical executives.
The New Database Paradigm
Calpont’s column database, InfiniDB, is helping organizations dive deeply into and across their data to successfully meet their needs for reporting, monitoring, and advance analytics. InfiniDB combines the simplicity of symmetric multiprocessing (SMP) and the power of massive parallel processing (MPP) into a column-oriented architecture for superior load and query performance. InfiniDB is designed expressly for the uniqueness of data warehouses that support deep analytics. But How?
- Column vs. Row. Because traditional RDBMS’s are row-based, DBA’s must use indexing, horizontal partitioning, materialized views, summary tables and parallel processing to get around inherent row-based performance inefficiencies. The InfiniDB columnar design eliminates this effort and overhead; with not only significantly faster query and load performance, but also without indexes, aggregates or partitions to build. Read more at “Why Choose a Column Database for Business Intelligence”.
- MPP. Until now, expensive SMP solutions were needed for large data sets and many users processing deep analytics. InfiniDB’s MPP, parallel query processing is far more scalable than SMP environments, and is easily scaled with commodity hardware for data capacity and user concurrency needs, without degrading query performance. This makes it an ideal solution for deep analytics environments. See Calpont’s whitepaper “Divide and Conquer – Why MPP is Best for BI Databases” for more information.
- Parallel Recursive Computations. Blindingly fast query and load performance is not enough. Many deep analytic computations are recursive, requiring multiple database passes. To work around this problem, many analysts create a separate data set against which they process queries and reload back into the data warehouse – a very time-consuming and expensive process. InfiniDB’s self-regulating, multi-threaded vertical and horizontal partitioning automatically distributes recursive computations across multi-CPU/core based hardware, solving the problems of multi-step, deep analytics while providing flexible load balancing. See Calpont’s whitepaper “A Technical Overview of the InfiniDB Analytic Database” for more information.
Learn how Calpont is solving industry performance challenges in:
Is your organization experiencing Deep Analytics performance challenges?
Try InfiniDB Free