In preceding articles, I elucidated the pivotal features and manifestations of the knowledge economy, aligning with significant technological advancements globally. This encompassed various branches such as knowledge engineering, knowledge management, knowledge societies, and knowledge cultivation, alongside pertinent aspects like cryptocurrencies, blockchain, the Internet of Things, augmented reality, artificial intelligence programs, nanotechnology, clean energy, and the focal point of this article—big data technology.
Let’s first define ‘data’ and discern its distinction from ‘information,’ elucidating its essential characteristics. Data constitutes the raw representation of information prior to sorting, organizing, and processing. It inherently lacks utility in its initial form and requires processing. In contrast, information denotes processed, analyzed, and interpreted data that can be employed for establishing diverse relationships between phenomena and facilitating decision-making.
Raw data can be categorized into three types: structured data, organized in tables or databases for processing; unstructured data, constituting the majority generated daily from text writings, images, videos, messages, and website clicks.
Big data is delineated as a dataset of such magnitude and complexity that conventional programs struggle to capture, manage, process, store, and analyze it within an acceptable timeframe. This intricate task is adeptly handled by artificial intelligence programs. Notably, big data technology expedites processes that traditionally demanded extensive time investments. For instance, the decoding of the human gene, which typically consumed a decade, can now be accomplished in a mere week through the application of big data technology, as exemplified by the Silwan Digital Sky Survey project.
This U.S. research endeavor, focused on astronomical sky surveying, accumulated more data in its initial weeks in 2000 than the entire historical record of astronomy. The transformative impact of big data is further illustrated by the acquisition of WhatsApp, where the technology unveiled staggering user statistics and message circulation metrics, highlighting its efficacy in handling vast amounts of information.
The term ‘big data’ specifically refers to datasets so vast and intricate that conventional database management tools or traditional data processing applications become insufficient. This includes aspects such as capture, duration, storage, research, participation, transportation, analysis, and visualization, enabling the identification of crucial business trends, legal citations, crime prevention, and real-time traffic flow data.
Gartner, a leading industry authority, revised its definition of big data in 2012, emphasizing its role as a high-volume, high-speed, or high-diversity information asset necessitating innovative processing methods to enhance decision-making, comprehension, and process improvement.
Effectively harnessed and analyzed, big data technology confers a competitive advantage to organizations by providing deeper insights into customer behavior, facilitating more informed decision-making based on data extracted from customer databases. Notably, various sectors and institutions employing big data analytics tools have reported significant savings, with some achieving annual surplus reductions of over $300 million, equating to two-thirds of their budgets through a 48% reduction in spending costs.
The application of big data extends beyond enterprises to diverse sectors, encompassing energy, education, health, and large scientific projects, exemplified by the Human Genome Project—a comprehensive study of human genetic material.
In summation, big data technology stands as a linchpin in the emerging industries of the knowledge economy, serving as a potent tool for measurement, analysis, and research that yields transformative insights. It emerges as a pivotal technological milestone in human history, especially pertinent when contemplating the future of human roles in the era of automation. In this context, the role of a big data analyst emerges as one of the most crucial functions that the future will provide for humanity.
Author : Manahel Thabet
Published January 22, 2018
Al Bayan Newspaper
