Data mining is primarily used by industries that cater to the consumer, like retail, financial and marketing companies. If you’ve ever shopped at a retail store and received customized coupons, that’s a result of mining. Your individual purchase history was analyzed to find out what products you’ve been buying and what promotions you’re likely to be interested in. Netflix uses data mining to recommend movies to its customers ,Google uses mining to tailor advertisements to internet users and Walmart uses data mining to manage inventory and identify areas where new products are likely to be successful. Mining is more likely to be used by larger companies, as enormous computers are required to sift through data. Data mining process is a very simple and easy to understand.
How To Data Mining Works:
Data mining involves exploring and analyzing large blocks of information to glean meaningful patterns and trends. It can be used in a variety of ways, such as database marketing, credit risk management, fraud detection, spam Email filtering, or even to discern the sentiment or opinion of users. The data mining process breaks down into five steps. First, organizations collect data and load it into their data warehouses.
Next, they store and manage the data, either on in house servers or the cloud. Business analysts, management teams, and information technology professionals access the data and determine how they want to organize it. Then, application software sorts the data based on the user’s results, and finally, the end-user presents the data in an easy-to-share format, such as a graph or table. A typical data mining project starts with asking the right business question, collecting the right data to answer it, and preparing the data for analysis. Success in the later phases is dependent on what occurs in the earlier phases. Developing a thorough understanding of the project Parameters, including the current business situation, the primary business objective of the project, and the criteria for success.
Determining the data that will be needed to solve the problem and gathering it from all available sources. Preparing the data in the appropriate format to answer the business question, fixing any data quality problems such as missing or duplicate data. Determining whether and how well the results delivered by a given model will help achieve the business goal. There is often an iterative phase to find the best algorithm in order to achieve the best result.
list of Data Mining Process :-
- Business understanding :- In this phase, business and data-mining goals are established. First, you need to understand business and client objectives. You need to define what your client wants (which many times even they do not know themselves) Take stock of the current data mining scenario. Factor in resources, assumption, constraints, and other significant factors into your assessment. A good data mining plan is very detailed and should be developed to accomplish both business and data mining goals.
- Data understanding :- In this phase, sanity check on data is performed to check whether its appropriate for the data mining goals. First, data is collected from multiple data sources available in the organization. These data sources may include multiple databases, flat filer or data cubes. There are issues like object matching and schema integration which can arise during Data Integration process. It is a quite complex and tricky process as data from various sources unlikely to match easily. For example, table A contains an entity named cust_no whereas another table B contains an entity named cust_id. Therefore, it is quite difficult to ensure that both of these given objects refer to the same value or not. Here, Metadata should be used to reduce errors in the data integration process. Next, the step is to search for properties of acquired data. A good way to explore data is to answer the data mining questions (decided in business phase) using query, reporting, and visualization tools .Based on the results of query, the data quality should be ascertained. Missing data any should be acquired.
- Data preparation :- In this phase, data is made production ready. The data preparation process consumes about 90% of the time of the proiect The data from different sources should be selected, cleaned, transformed, formated anonymized, and constructed (if required).Data cleaning is a process to “clean” the data by smoothing noisy data and filling in missing values. For example, for a customer demographics profile, age data is missing. The data is incomplete and should be filled. In some cases, there could be data outliers, For instance, age has a value 300. Data could be inconsistent. For instance, name of the customer is different in different tables. Data transformation operations change the data to make it useful in data mining.
- Data transformation :- Data transformation operations would contribute toward the success of the mining process. It helps to remove noise from the data. Summary or aggregation operations are applied to the data. I.c, the weekly sales data is aggregated to calculate the monthly and yearly total.
- Modelling :-In this phase, mathematical models are used to determine data patterns. Based on the business objectives, suitable modeling techniques should be selected for the prepared dataset. Create a scenario to test check the quality and validity of the model. Run the model on the prepared dataset. Results should be assessed by all stakcholders to make sure that model can meet data mining objectives.
- Evaluation :- In this phase, patterns identified are evaluated against the business objectıves. Results generated by the data mining model should be evaluated against the business objectives. new business requirements may be raised because of data mining. A go or no-go decision is taken to move the model in the deployment phase.
- Deployment :- In the deployment phase, you ship your data mining discoveries to everyday business operations. The knowledge or information discovered during data mining process should be made easy to understand for non-technical stakeholders. A detailed deployment plan, for shipping, maintenance, and monitoring of data mining discoveries is created. A final project report is created with lessons learned and key experiences during the project. This helps to improve the organization’s business policy.
