Moss L.T., Atre S. Business intelligence roadmap: The complete project lifecycle for decision-support applications
Подождите немного. Документ загружается.
Never develop a logical data model in isolation.
Business people must drive the data requirements:
what information they need, how they need it, when
they need it, and how clean it has to be.
When reviewing the business meta data with
business people from other departments, resist
including their data requirements in your scope.
Remind those other business people that you only
want them to validate the correctness of your data
definitions, domains, business rules, and so on.
Don't ask programmers or systems people to create
the business meta data. (Technicians generally do
not enjoy analysis, just as business analysts
generally do not enjoy programming.) Find people
who enjoy detailed business analysis, such as data
administrators, business analysts, and subject
matter experts.
Don't assume that the more current the data, the
higher the accuracy. If quality processes are not
enforced at an organization, new data can get
corrupted within months.
Don't underestimate the time it will take to resolve
the many battles among the business people to
arrive at a consensus on valid data definitions, data
domains, business rules, and interpretations of
data.
Never suck and plunk! In other words, don't plan to
move all the source data as is. One of the main
reasons for a BI decision-support initiative is
gaining access to reliable and clean data. Simply
giving access to dirty data through a BI application
will not help business executives make better
decisions.
Don't attempt to cleanse every piece of data
because the cleansing task can become
overwhelming without providing real business
value. Triage your data-cleansing activity.
Tips and Rules of Thumb
The most effective techniques for both data
integration and source data analysis are
normalization rules and logical data modeling.
Involve a data administrator on your project to
create the logical data model. Data administration
should then compare and merge your project-
specific logical data model with the enterprise
logical data model to find and resolve data
discrepancies.
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
571/631
In regards to business rules, use top-down logical
data modeling to define the business rules and
bottom-up source data analysis to discover where
the source data violates the business rules.
While the data is being modeled, have the source
data analyzed. Since there is never enough time to
analyze each data element, prioritize the data
elements, and concentrate on the most critical ones.
The data analysis effort on BI projects can be
greatly reduced if logical data models and business
meta data already exist for the operational source
systems. Unfortunately, these models and meta
data are rarely created for operational systems and,
therefore, they rarely exist. The business
representative and business sponsor should urge
the owners of the operational systems (line-of-
business managers) to consider creating logical data
models and collecting business meta data for their
operational systems in the future.
Business managers (and many IT managers) who
have not been through data cleansing before and
who are unfamiliar with the effort often
underestimate the time required by a factor of four.
Source data analysis is primarily an intensive
manual effort. Although tools can help with the
effort, they cannot eliminate it.
Solicit help from the IT staff when needed. Systems
analysts, developers, and database administrators
often know the technical aspects of the data more
intimately than the business representative or the
data owners do. The IT staff knows how and where
the data is stored, processed, and used. They often
have in-depth knowledge of the accuracy, the
relationships, and the history of the data.
Be sure that you have a mechanism for capturing
the business meta data during this step. At a
minimum, capture:
- Data names
- Definitions
- Relationships
- Cardinality
- Data domains
- Business rules
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
572/631
Source data quality will only be as good as the
enforcement of quality processes in the operational
systems. Quality enforcement should include data
entry rules, edit checks, and training. Improving the
organization's quality of data is a holistic approach
and cannot be achieved through data cleansing
alone.
6. Application
Prototyping
Decide on the type of application prototype to
create:
- Show-and-tell
- Mock-up
- Proof-of-concept
- Visual-design
- Demo
- Operational
Get business people to participate in the prototype
from the beginning, especially during needs
assessment and graphical user interface
construction. Remember, it is their definition of
success and failure that counts, not yours.
Make sure that time limits are set and followed for
each prototype iteration.
Limit each prototype to a specific subject area or a
specific application function.
Review and renegotiate the project constraints
whenever the scope of the prototype changes.
Communicate daily with the project core team
members, especially the database administrator and
the ETL lead developer.
Make sure that the business people realize that only
a prototype is being built, not a full-scale,
production-worthy BI application.
Learn about the reporting dimensions, the level of
aggregation needed, and the variety of data
accessed. Test the database design during the
prototype.
Stop prototyping when you reach the point of
diminishing returns.
Don'ts
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
573/631
Don't promise what you cannot deliver. Unfulfilled
promises can damage your reputation.
Don't include all the project requirements in the
scope of the prototype. Even when using the
operational prototype as a development method for
the access and analysis portion of the BI
application, each prototype should deliver only a
small piece of the final BI application.
Don't keep adding new functionality to the end of
the prototype when the original objectives of the
prototype have been reached. If you want to
continue prototyping, set new objectives, and revise
the project plan.
It is best not to use an arbitrary sample of source
data. Source data is usually voluminous, and an
arbitrary sample may not necessarily represent all
the possible combinations that need to be tested in
that particular prototype.
Don't contaminate the prototype with poor-quality
data, and don't spend any time cleansing the data
unless you are testing an ETL function.
Don't address too many data integration
requirements. The more integration, the higher the
complexity, and the longer the prototype will take.
Data integration requirements should be kept to a
minimum.
Don't ignore the front-end interface design. Be sure
to have the business people involved.
Don't work only with the business people who are
vocal or who are your favorites. You need to have
some nonbelievers as well because they will tell you
more readily when something is not working. You
want to find out during prototyping what is working
and what is not. Most importantly, include the
business people who "sign the check" for the
prototype since the funding is coming out of their
budget and they form the core group for whom the
prototype is being built.
Don't underestimate the time required to implement
suggested changes.
Don't use the prototype to test technology
performance. This is not a stress-testing
environment.
Tips and Rules of Thumb
To test ease of use, let the business people have as
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
574/631
much hands-on experience with the prototype as
possible so that you can see whether the application
will be easy to use the way you are planning to
build it.
Use the prototype activity to build and maintain a
coalition comprised of line-of-business managers, IT
managers, and senior business executives.
Ensure that an appropriate number of business
people participate in the prototype (more than one
but no more than eight).
Avoid using a large project team to build the
prototype. Don't add to the team size if deadlines
are missed. Instead, shrink the team size!
"Bloating" the team will increase the time required
for staff communication and slow things down even
more. Shrinking the team size will reduce required
communication among team members and will
enable the team to get things done faster.
Business managers often think they need only
summary data, but sooner or later they end up
asking for detailed data. Depending on your design
and the tools you are using, providing detailed data
may not be as trivial as it sounds. Be sure to test it
in the prototype.
Consider building an operational prototype for the
access and analysis portion of the BI application.
Operational prototypes are robust enough, and
access and analysis tools are flexible enough, that
this type of prototype could naturally evolve into the
final access and analysis application after several
tightly controlled iterations. The activities of Step
12, Application Development, could be applied to
the final iteration of the operational prototype.
Define ease of use. When is a system considered
easy to use? Some measurements include the
following:
- Learning curve: One or two days is the
maximum that a business person can usually
set aside for learning a new application.
- Speed of task accomplishment: By using the
new BI application, knowledge workers and
business analysts have to be able to finish
their analysis tasks at least 25 percent faster.
- Subjective satisfaction: The business people
should be looking forward to using the new BI
application and not avoiding it.
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
575/631
- The help function: The business people
should be using the help function at least once
a day for the first month rather than avoiding
it because it is too complicated or too
confusing.
- Difference in usage and needs: Not all
business people are created equal.
Understand the different needs of executive
managers versus business analysts.
7. Meta Data Repository
Analysis
Have the business representative participate in the
meta data requirements definition process. Meta
data is an important part of every BI project and
needs to be given the same amount of attention as
business data.
Establish data ownership and allow those data
owners to control the meta data for the business
data over which they have authority. Data
ownership can be assumed by business managers
individually or by a representative committee.
Work with the data administrators to develop or
revise cross-organizational data standards and
publish those standards.
Plan to consolidate all existing meta data from
various tools into one enterprise meta data
repository.
Capture and validate the meta data requirements
through a logical meta model using the entity-
relationship modeling technique, even if you later
decide to license a meta data repository or to build
one based on an object-oriented design.
Pay equal attention to business meta data and
technical meta data.
Don'ts
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
576/631
Don't try to do everything at once, but don't forget
the big picture either. The meta data repository will
evolve over time just like the other components of
the BI decision-support environment.
Don't forget to analyze the interface requirements
for the meta data repository. There are two types of
interfaces to consider: the access interface for
business people and technicians, and the tool
interface to the ETL, OLAP, CASE, and other tools.
Don't consider meta data to be just documentation.
Meta data provides the context for business data
and is used as a navigation tool in the BI decision-
support environment. It is therefore an integral
deliverable of every BI application.
Tips and Rules of Thumb
If your organization has no meta data solution and
this is the first time you are addressing meta data,
triple your time estimates for the development steps
in the Meta Data Repository track.
Prioritize the meta data components into three
categories:
- Mandatory
- Important
- Optional
All mandatory meta data components should be
captured and stored. Also try to capture as many
important components as you can, and postpone the
optional ones if you run out of time.
When creating the logical meta model, draw the
entity-relationship diagram for the most critical
meta data components:
- Entity
- Attribute
- Relationship rules (cardinality and
optionality)
- Table
- Column
- Keys (primary and foreign)
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
577/631
- Domain
- Data type and length
- Definition
- Transformation rules
Add additional components to your logical meta
model as time allows.
8. Database Design
Work with multiple business people to understand
the types of analyses they need to perform and the
ways they will access the data. Choose the
appropriate database design schema based on those
requirements and access patterns.
Use the entities on the logical data model and the
business meta data as a starting point for designing
the conformed dimensions and the normalized
snowflake dimensions.
Review the lessons learned from the prototyping
activities. See how many reports and queries can be
run in parallel, what dimensions are needed, and
what security is needed.
Identify similar reporting patterns among business
analysts in order to minimize the number of star
schemas needed.
Plan to monitor the performance of queries and
reports on a regular basis.
Expect to continually refine the BI target database
designs.
Make decisions about clustering tables, partitioning,
data placement, and indexing with performance in
mind.
Index those columns that are searched on
frequently and that have a high distribution in
values.
Don'ts
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
578/631
Never attempt to implement a fully normalized
logical data model! The purpose for a normalized
logical data model in any environment (operational
or BI) is to facilitate business analysis. Even if an
entity-relationship design schema is chosen, the
logical data model must still be denormalized into a
physical data model (database design).
Don't assume that one size fits all. Different
business people have different access requirements;
they need different levels of detail and
summarization; and they have different
requirements for timeliness, availability, and
quality.
Don't assume that all BI target databases must be
multidimensional. There are occasions where an
entity-relationship design is appropriate. Be sure to
let the requirements drive the database design
decision, not the latest technology or design trends.
Don't count on being able to reload your BI target
databases from scratch after a catastrophic
database failure. It would take a very long time, if it
could be done at all. Plan on taking frequent
backups (full, partial, or incremental).
Don't blindly use a physical data model (database
design) developed for a different organization. Often
those models reflect compromises made for a given
organization, which could easily lead to having to
redesign your database later.
Tips and Rules of Thumb
Database administrators—not programmers—should
design databases. Do not use database
administrators as "data entry clerks" who simply
type up and run DDL handed to them by the
programmers. The job description of database
administrators includes database design for the
main reason that database administrators are or
should be trained in the DBMS-specific optimizers as
well as in multidimensional design techniques.
Understand the differences between operational
databases and BI target databases. Key features in
well-designed operational databases are
normalization, utilization of DBMS referential
integrity, and judicious use of indexing. Key features
in BI target databases are just the opposite:
denormalization, reliance on program-enforced
referential integrity, and heavy indexing.
Many BI target databases will almost inevitably fall
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
579/631
into the VLDB category.
- Small databases are in the range of 10 to
100 GB
- Medium databases are in the range of 100 to
300 GB
- Large databases are in the range of 300 to
800 GB
- VLDBs are in the range of 800 GB to many
terabytes
Since a BI target database is grow-only, today's
small database is next year's medium (or large)
database.
Clustering is a very useful technique for sequential
access of large amounts of data. Since sequential
access of data is the norm in BI applications, using
this technique can dramatically improve
performance.
Physically co-locate related tables on the disk drive.
When indexing the dimension tables, either of two
extreme approaches may pay off.
- Index everything in sight.
- Leave the table as a heap (no physical index
or ordering whatsoever).
The decision depends on the makeup, usage, size,
and distribution of data values in the dimension
tables.
When should an index not be built?
- When value entries in the index are more
than 15 percent, building an index does not
pay off. For example, if an index is built on
the column Gender_Code there will be about
50 percent entries in the index for the value
"female" and 50 percent entries for the value
"male."
- When the index database is searched
sequentially.
- When performance still does not improve.
- Or worse, when as a result of building the
index, performance is degraded even further
北斗成功社区 BeiDouWeb.com 教育音视频/电子书/实用资料文档/励志音乐影视 仅供免费试用/版权原著所有
580/631