Different Architectures of Big Data Application!

The steady rise in the significance and demand for Big data makes it no longer a new software technology to the enterprise information technology (IT) infrastructure. As more and more complexity arises with larger and larger data storages created, enterprises are demanded to learn Big Data application architectures that are invented and sourced from operational systems and are accumulated and queried by business intelligence (BI) applications. These data are stored on multiple hardware platforms and exist to support different user communities. Depending on the complexity, volatility, and usage of the data, different Big Data applications can be categorized as follows,

Type 0 Data - The Archive

Type 0 Data, also known as ‘The Archive’ is the fundamental stage involve in integrating big data applications into the data warehouse of the enterprise. It generally involves the extraction of information from operational systems to store them in the data warehouse. This process is generally keyed data referring to accounts, customers, products, and associated dimensions such as sales territories. As the amount of data grows over time, users have a greater ability to detect trends and establish future forecasts. As the amount of data grows, the DW support staff needs to add resources to the system. These usually take the form of hybrid hardware and software ‘appliances’ that combine large data storage capacity with specialty processors.

Type 1 Data - Structured Big Data at Rest

Structured Big Data at Rest or Type 1 Data is a Big Data architecture which specifically defines various classic big data applications. This also involves operational data is accumulated and stored. In addition to keyed data, this type of big data architecture will involve transactional data consisting of product shipping, product purchase and customer interface information. This greatly expands the BI analysts’ ability to query and analyze the complex interplay between customers and their purchases of products and services.

Type 2 Data - Unstructured or Unmolded. As operational systems matured, companies moved their applications closer to the customer. In-store kiosks, on-line ordering, and internet-based product catalogs became common.  As a result, data about customer transactions was no longer generated solely in structured records and files from operational systems. Instead, the norm now is to expect weblogs, social network data, click streams, machine sensors, and other solutions to generate unstructured data. These data can then be analyzed in new ways such as reputation management or competitive intelligence gathering. The challenge of unstructured data is that each selection or manifestation of a data stream may require not only a one-time decision on how to process it but also a unique hardware or software solution in order to analyze it.

Type 3 Data - Data in Motion. Data in motion is the latest variation of big data processing. Here, data from customer transactions, product shipping and receiving, medical patient monitoring or hardware sensors is analyzed as it is generated, and the results used as part of the transaction.  For example, if an on-line customer orders a product, the transaction may be analyzed before it completes by comparing to historical customer purchases in order to detect possible fraud. Analysis of multiple customer purchases may allow algorithms to derive product popularity in certain geographical areas, leading to possible price or shipping cost changes.

Data in motion require analytics at the point of data creation. It may be too late to accumulate this data in a big data appliance or data warehouse for later processing (although this may be done for historical analyses). Instead, the intent is to use the data immediately, do necessary analytics, and implement decisions that can both increase customer satisfaction and increase profits.