Data Mining and Big Data Analytics Applications

Data mining is the process of extracting and analyzing “relationship and patterns in stored transaction data to get information which will help for better business decisions.”[1] It is typically done using regression, clustering, or classification algorithms that operate on small or large relational databases and structured data.[1] Data mining is rooted in statistics. It is used to unmask potentially useful trends underlying a particular dataset. Big data analytics is closely related but different in that it emphasizes processing and storing very large heterogeneous datasets, those over 1 terabyte large, that contain structured, semi-structured, and unstructured data.[1][2] Big data analytics in some regards is data mining on steroids. Companies and governments around the globe are leveraging both data mining and Big data analytics to make more informed decisions using the data they have at their disposal.[3] It is estimated that by 2020 there will be “5,200 Gbs of data on every person in the world” leaving no shortage of data to be mined.[1]

In the healthcare field, data mining and data analytics are used to quickly and effectively discover the unintended side-effects of certain medications, specifically when multiple medications are taken together. “By mining tens of thousands of electronic patient records, researchers at Stanford University quickly discovered an unexpected answer: people who take both drugs [Paxil and Pravachol] have higher blood glucose levels.”[4] Clinical trials are expensive and they take a long time to complete due to patient health and safety regulations. Without data mining, it could have taken years if not a decade for researchers to discover that mixing drugs like Pravachol and Pravachol is dangerous.

In national defense, data mining and data analytics are used to increase the effectiveness and efficiency of tactical and surveillance systems. Most of the data mining software used by the military is developed by the Defense Advanced Research Projects Agency (DARPA), the same agency that created ARPANET, the precursor to the Internet.[5] One such example is the Chaff Electronic Countermeasure System, which is used to “detect missile near the ship, calculate the missile’s trajectory towards the ship and release chaff rockets to deflect the incoming missile to another calculated trajectory away from the ship.”[5] This data mining-powered anti-missile system can react with greater speed and accuracy than a human can, increasing the overall safety of a combat ship and its crew.[5]

Businesses utilize data mining and big data analytics to increase operational effectiveness and strategic advantage over rivals. Netflix is one prominent example. Every time a Netflix user scrolls through a row of movie titles, clicks on a trailer, or stops watching a particular television show to find another one, their action is recorded by Netflix. Netflix uses this data to create personalized video-rankings, group similar movie and television titles, and decide what content to license or produce in-house.[6] These features have allowed Netflix to save $1 billion in customer retention and surpass Disney as the world’s largest entertainment company with only 5,000 employees compared to Disney’s 200,000 employees.[7] Similarly, Amazon uses big data analytics and a subfield of that called Predictive Analytics “for targeted marketing to increase customer satisfaction and build company loyalty.”[8] Specifically, Amazon employs a robust collaborative filtering engine (CFE) that “analyzes what items you purchased previously, what is in your online shopping cart or on your wish list…to recommend additional products that other customers purchased when buying those same items.”[8] Amazon has found that their CFE generates 35% of their annual revenue.[8] Amazon also uses analytics to dynamically update the prices of products sold through its website every 10 minutes in order to increase annual profits.[8]

Aided by powerful supercomputers, NASA stores, processes, and distributes “12.1TB of data every single day from thousands of sensors and systems dotted across the world and space.”[9] It uses this data to “help analyze the challenging projects, from solar flare and space weather scenarios to detailed space vehicle designs.”[10] At the present, NASA along with several other US government agencies are working on sophisticated models to monitor and predict the long-term effects of global warming and climate change. The accuracy of these climate change models is only possible because of the advances in computational power, Big Data Analytic techniques, and the vast amount of recorded climate data.[11]

Data mining and big data analytics also play a major role in cybersecurity. Traditional cybersecurity defensive measures include firewalls, role-based access controls, and strong passwords.[12] While these measures are still necessary to eliminate low-level threats, they are not enough. New cybersecurity defensive measures include the use of analytics software that can detect suspicious user behavior and cut-off access to a potential attacker before they can compromise an organization’s network.[13]

Research shows that companies that utilize some form of data mining or big data analytics to drive business strategy are “twice as likely to rank in the top quarter for financial performance, five times more likely to make timely decisions, and three times more likely to execute their decisions and plans.”[13] For this reason the big data analytics market is expected to be worth over $200 billion by 2020.[14]

Sources:

1. “Big Data vs Data Mining – Find Out The Best 8 Differences.” EDUCBA.com, EDUCBA, 5 Oct. 2018, www.educba.com/big-data-vs-data-mining/.
2. Labbe, Mark, et al. “What Is Big Data Analytics? – Definition from WhatIs.com.” SearchBusinessAnalytics, Sept. 2018, searchbusinessanalytics.techtarget.com/definition/big-data-analytics.
3. “5 Data Mining Applications.” ExpertSystem.com, Expert System, 7 July 2016, www.expertsystem.com/5-data-mining-applications/.
4. Savage, Neil. “Mining Data for Better Medicine.” MIT Technology Review, MIT Technology Review, 30 Dec. 2013, http://www.technologyreview.com/s/425466/mining-data-for-better-medicine/.
5. “Data Mining in the Military.” MIS Class Blog, MIS Class Blog, 14 Mar. 2017, misclassblog.com/data-analytics/data-mining-in-the-military/.
6. “How Netflix Uses Big Data to Drive Success – Predictive Analytics Times – Machine Learning & Data Science News.” Predictive Analytics – The Power to Predict Who Will Click, Buy, Lie, or Die, 18 Sept. 2018, www.predictiveanalyticsworld.com/patimes/how-netflix-uses-big-data-to-drive-success/9693/.
7. Dans, Enrique. “How Analytics Has Given Netflix The Edge Over Hollywood.” Forbes, Forbes Magazine, 30 May 2018, www.forbes.com/sites/enriquedans/2018/05/27/how-analytics-has-given-netflix-the-edge-over-hollywood/#5017d8546b23.
8. Wills, Jennifer. “7 Ways Amazon Uses Big Data to Stalk You.” Investopedia, Investopedia, 20 Oct. 2018, www.investopedia.com/articles/insights/090716/7-ways-amazon-uses-big-data-stalk-you-amzn.asp.
9. Gorey, Colm. “The Volume of Data NASA Has to Manage Is Mind-Boggling.” Silicon Republic, 26 Oct. 2017, www.siliconrepublic.com/enterprise/nasa-data-figures.
10. Skytland, Nick. “What Is NASA Doing with Big Data Today? | OpenNASA.” NASA, NASA, 4 Oct. 2012, open.nasa.gov/blog/what-is-nasa-doing-with-big-data-today/.
11. Northon, Karen. “NASA Releases Detailed Global Climate Change Projections.” NASA, NASA, 8 June 2015, www.nasa.gov/press-release/nasa-releases-detailed-global-climate-change-projections.
12. “10 Basic Cybersecurity Measures: Best Practices to Reduce Exploitable Weaknesses and Attacks .” Ics-Cert.us-Cert.gov, Water Information Sharing & Analysis Center (WaterISAC), June 2015, ics-cert.us-cert.gov/sites/default/files/documents/10_Basic_Cybersecurity_Measures-WaterISAC_June2015_S508C.pdf.
13. “The Growing Importance of Business Analytics.” Villanova.edu, Villanova School of Business, taxandbusinessonline.villanova.edu/resources-business/article-business/the-growing-importance-of-business-analytics.html.
14. Press, Gil. “6 Predictions For The $203 Billion Big Data Analytics Market.” Forbes, Forbes Magazine, 20 Jan. 2017, www.forbes.com/sites/gilpress/2017/01/20/6-predictions-for-the-203-billion-big-data-analytics-market/#354935c20838.