Moss L.T., Atre S. Business intelligence roadmap: The complete project lifecycle for decision-support applications
Подождите немного. Документ загружается.
- Credit risk: Assess the credit risk of potential loan applicants based on a
predictive model of the database that looks for reasons and patterns
affecting risk.
- Quality control: Find patterns of quality problems on assembly lines to
help reduce the number of products returned due to substandard quality.
Financial services
- Customer retention: Identify loyal bank customers who have many
high-balance accounts and provide each of them a personalized fee
structure. It is much cheaper to retain existing customers than to acquire
new ones.
- Stock performance: Develop models of stock performance to aid
portfolio management. Look for stocks that have performed in ways
similar to certain high-performing securities.
Distribution
- Inventory control: Improve inventory control and distribution by
developing predictive models of which products or parts will be needed at
various distribution points at various points in time.
One good indication of the value of data mining is the secrecy that surrounds its
implementations. Many of the companies that have implemented data mining
hesitate to talk about their successes. Some will not even confirm that they use this
technology.
[ Team LiB ]
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
391/631
[ Team LiB ]
Data Mining Activities
The activities for data mining do not need to be performed linearly. Figure 13.2
indicates which activities can be performed concurrently. The list below briefly
describes the activities associated with Step 13, Data Mining.
State the business problem.
Set goals before starting the data mining efforts, and prioritize the goals (such
as increase profits, reduce costs, create innovative product strategies, or
expand the market share). Time and money have to be invested in order to
reach any of these goals. There also needs to be a commitment from
management to implement a data mining solution at the organization.
1.
Collect the data.
One of the most time-consuming activities of data mining is the collection of the
appropriate types and quantities of data. In order to have correct
representation, first identify all the data needed for analysis. This includes data
stored in the operational databases, data from the BI target databases, and any
external data that will have to be included. Once you have identified the source
data, extract all pertinent data elements from these various internal and
external data sources.
2.
Consolidate and cleanse the data.
Redundantly stored data is more of a norm than an exception in most
organizations. Therefore, the data from the various sources has to be
consolidated and cleansed. If the internal data is to be supplemented by
acquired external data, match the external data to the internal data, and
determine the correct content.
3.
Prepare the data.
Before building an analytical data model, you need to prepare the data. Part of
data preparation is the classification of variables. The variables could be
discrete or continuous, qualitative or quantitative. Eliminate variables with
missing values or replace them with most likely values. It provides great insight
to know the maximum, minimum, average, mean, median, and mode values for
quantitative variables. In order to streamline the preparation process, consider
applying data reduction transformations. The objective of data reduction is to
combine several variables into one in order to keep the result set manageable
for analysis. For example, combine education level, income, marital status, and
ZIP code into one profile variable.
4.
Build the analytical data model.
One of the most important activities of data mining is to build the analytical
data model. An analytical data model represents a structure of consolidated,
integrated, and time-dependent data that was selected and preprocessed from
various internal and external data sources. Once implemented, this model must
be able to continue "learning" while it is repeatedly used by the data mining
5.
6.
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
392/631
tool and tuned by the data mining expert.
Interpret the data mining results.
Once the data mining operations are run and results are produced, the next
major task is to interpret the results. Important things to consider during this
interpretation are how easily the results can be acted upon and whether the
results can be presented to business executives in a convincing, business-
oriented way.
6.
Perform external validation of the results.
Compare your results with published industry statistics. Identify the deviations
from those statistics and determine the reasons for the deviations. Be sure you
are using updated industry statistics since they change from time to time.
Compare the selection criteria of your data to that of the industry statistics, and
compare the time frame during which your data was selected to the time frame
covered by the industry statistics. The selection criteria and time frame of your
model and of the industry statistics must be similar.
7.
Monitor the analytical data model over time.
Industry statistics are usually established by using very large samples. It is
important to validate your analytical data model against industry statistics at
regular intervals. Industry statistics change over time, and some industries
have seasonal changes. In that case, adjust your internal analytical model.
8.
Figure 13.2. Data Mining Activities
[ Team LiB ]
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
393/631
[ Team LiB ]
Deliverables Resulting from These Activities
Data mining database
The data mining database is designed and built for a specific analytical data
model and a specific set of data mining operations. This database will be
populated with data from either an operational system, a BI target database, or
a combination of both.
1.
Analytical data model
The analytical data model is developed and tested so it can be used by the
algorithms of the data mining operations in the data mining tool.
2.
[ Team LiB ]
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
394/631
[ Team LiB ]
Roles Involved in These Activities
Business representative
In the end, it is the business representative (or his or her management) who
will benefit from the results of data mining. Therefore, the business
representative has to work very closely with the data mining expert (unless the
business representative on this BI application is the data mining expert) in
order to understand and interpret the data mining results.
Data mining expert
The data mining expert is a statistician who really knows the data and is
familiar with the data mining techniques. He or she is usually responsible for
selecting the most appropriate data mining tool for the organization. He or she
is also the primary person who builds the analytical data model and analyzes
the data mining results.
Database administrator
The database administrator must have a good understanding of the data
content in order to design the data mining databases for the data mining
activities. He or she works very closely with the data mining expert.
Subject matter expert
The subject matter expert can help analyze, define, cleanse, and prepare the
source data for the data mining databases. He or she will work under the
direction of the data mining expert.
[ Team LiB ]
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
395/631
[ Team LiB ]
Risks of Not Performing Step 13
Most organizations are sitting on top of a gold mine—the "gold" being all the data
collected about their customers and the products their customers buy. Embedded in
this data is information about their customers' spending styles, likes and dislikes,
and buying habits. This data is a wasted resource if it is not mined to expose the
hidden business intelligence. In addition, executive management must take notice of
data mining performed by competitors. If the competitors end up increasing their
profits, reducing their costs, creating more innovative product strategies, and
expanding their market shares, the organization may lose customers at a very quick
rate, which may put its survival in jeopardy.
[ Team LiB ]
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
396/631
[ Team LiB ]
Bibliography and Additional Reading
Abramowicz, Witold, and Jozef Zurada. Knowledge Discovery for Business
Intelligence Systems. Kluwer Academic Publications, 2000.
Bajaj, Chandrajit. Trends in Software: Data Visualization Techniques. New York: John
Wiley & Sons, 1999.
Bashein, Barbara, and M. Lynne Markus. Data Warehouse: More Than Just Mining.
Morristown, NJ: Financial Executives Research Foundation, 2000.
Berson, Alex, and Stephen J. Smith. Data Warehousing, Data Mining, and OLAP. New
York: McGraw-Hill, 1997.
Berson, Alex, Stephen Smith, and Kurt Thearling. Building Data Mining Applications
for CRM. New York: McGraw-Hill, 1999.
Cabena, Peter, Pablo Hadjinian, Rolf Stadler, Jaap Verhees, and Alessandro Zanasi.
Discovering Data Mining: From Concept to Implementation. Upper Saddle River, NJ:
Prentice Hall, 1997.
Edelstein, Herb. Introduction to Data Mining and Knowledge Discovery. Potomac,
MD: Two Crows Corporation, 1998.
———. Data Mining: Products and Markets. Potomac, MD: Two Crows Corporation,
1997.
Kudyba, Stephan, and Richard Hoptroff. Data Mining and Business Intelligence. Idea
Group Publishing, 2001.
CRoss Industry Standard Process for Data Mining: http://www.crisp-dm.org
Data Management Association: http://www.dama.org
[ Team LiB ]
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
397/631
[ Team LiB ]
Chapter Fourteen. Step 14: Meta
Data Repository Development
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
398/631
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
399/631
Chapter Overview
This chapter covers the following topics:
Things to consider about meta data repository development
The many potential sources of meta data and the difference between active and
passive meta data repositories
The need to develop two types of meta data repository interface processes: a
tool interface process and an access interface process
The four types of testing that apply to meta data repository development: unit
testing, integration testing, regression testing, and acceptance testing
Six activities to perform before the meta data repository goes into production
How to organize a directory for the meta data repository
Brief descriptions of the activities involved in meta data repository
development, the deliverables resulting from those activities, and the roles
involved
The risks of not performing Step 14
Things to Consider
Meta Data Repository Product Support
What interfaces are required for the meta data repository product?
Will we need to contact the meta data repository vendor for
assistance with writing these interfaces?
Do we expect any problems with extracting meta data from the
tools that originate it (e.g., computer-aided software [CASE] tool,
extract/transform/load [ETL] tool, or online analytical processing
[OLAP] tool)? Do we know if other companies experienced any
problems?
Will we need to contact the tool vendors and ask for their
assistance?
Can we embellish the meta data reports that are provided with the
meta data repository product?
Does the meta data repository product have a context-sensitive
online help function?
Custom-Built Meta Data Repository
Is our meta data repository design flexible enough to be expanded
in the future?
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
400/631