Shop News Jobs WorldWide Contact Us Search C-Byte Home Page Big Open Systems That Work
      Data Center Ready/  

decision advantage whitepaper  

Attention, Warehouse Shoppers:
The Output is in the Decision.


Decision-Driven Warehouse Architecture

An effective approach to designing a successful warehouse is to view it as a set of services that model the decision-making requirements of your enterprise. These services are not driven by data requirements. They are driven by business requirements of the enterprise. Within this framework, you can embody the goals of each organization. You identify the means of achieving those goals through the use of internal or external information. Finally, you provide the feedback mechanism to manage the change that inevitably comes with evolving goals, growth, improved technology, and experience developed during the life of the project. By defining these services, you will have a logical view of a decision-driven working environment. Now, you are in a position to select the physical view— the Data Warehousing methodology that will lead to real success.

Three services are required for success in the Decision-Driven Warehouse:

(1) Decision Support Services are user centric and are closest to the decision-making process. Decision Support Services provide the data interface to the decision maker—the presentation, the data navigation mechanism, analysis, information discovery, and knowledge about the data, such as the data quality.

(2) Information Services are data centric and supply information to satisfy the needs of the Decision Support Services by merging, enriching, scrubbing, archiving, and mapping internal and external data. Information Services are held accountable for providing the necessary data with the appropriate quality for decision making.

(3) Decision Support Management Services are responsible for security, accessibility, availability, performance tuning, quality control, and change control. Decision Support Management Services provide the feedback mechanisms for managing quality control and performance issues.

Each service has service agents, service providers, and service requirements. Agents are people or entities that play a role in producing each service. For example, a human or a neural network could be a decision maker (agent), whose role would be to produce the decisions within a Decision Support Service. Service agents use providers to meet the requirements defined for the service. For example, a decision maker (agent) might use a query tool (provider) within a Decision Support Service to make decisions. Ideally, providers within each service operate in an integrated fashion. For example, if query tools and metadata are part of Decision Support Services, then the query tools need access to the metadata for help in data navigation.

While each service provides a generic deliverable (like clean, high quality data), the service requirements describe those unique deliverables for a given enterprise (e.g. how to merge customer data from three separate operational systems). The requirements for a given service can impact requirements for another service. For example, a decision maker’s requirements for accessing a Decision Support Service until 8 p.m. would be reflected in the database availability requirements of Decision Support Management Services.

Agents and requirements are defined first. Providers should be selected after a “physical” design (warehouse implementation strategy) is determined. For example, you would select an extraction tool after you know how much data, how often and how many places data has to be transferred. You should select a query tool only after deciding what database(s) you plan to use.

In the discussion that follows, each service is described in detail for a fictional insurance company, All Care Mutual (ACM).

ACM has built a successful Decision-Driven Warehouse, after moving away from cumbersome reports that took from two weeks to one month to turn around on legacy systems. ACM has no direct sales. It works entirely with a sales channel of insurance agents who sell several product lines—auto (both personal and commercial), life, and numerous specialty lines like mobile homes and boats. ACM built its warehouse using 10 years of history gathered on operational systems, including claims and sales data for millions of policies. The goal of ACM’s Data Warehouse project was simply to better manage current product lines by analyzing the profitability of product lines, policy holders, and agents. This would allow ACM, for example, to offer discounts to low claim groups or incentives to agents to sell into high margin business. In the beginning, ACM believed it had the data to do this but was lacking a way to get the data out in a way that enabled real analysis.

Back to Top

Decision Support Services
Decision Support (DS) Services give targeted decision makers (agents) the user-friendly, clear view of the business they need to make decisions. The decision makers are described as “targeted” because you need to understand their goals and information requirements. Each Decision Support Service is associated with a collection of targeted decision makers who have the same goals or are measured similarly. These decision makers use similar terms and definitions to describe their business. In some cases, it is possible to use a computerized process, such as neural networks, to take the place of the decision makers as agents in the DS Service.

Several DS Services might use overlapping data (such as marketing and sales from the same division) or their data might not intersect at all.

In the case of ACM, the fictional insurance company, the targeted decision makers were the sales and marketing departments responsible for the management of their insurance agents and policy coverage strategies. Each group required different views of somewhat overlapping data, resulting in two different kinds of decisions. Thus, ACM had two Decision Support Services.

In other industries, examples of a Decision Support Service might be the product marketing department of a manufacturing division or the point-of-sale operation for a drug store chain.

Decision Support Service

The DS Service requirements document describes the business goal of the service, the general class of business questions that need to be answered, and the decision-making methodology. The service requirement shows where the decision makers for that service are located on the enterprise network, what measures they need for decision making, how they view their business according to meaningful business attributes, how current their data needs to be, and so on.

The requirements document for ACM followed this framework. ACM had to manage two important business processes: policy production (sales and income) and policy claims (losses). Theoretically, ACM was interested in all of its operational data, at the very lowest level of detail. The most important business concerns were insurance agents, policies, products, and time, although marketing was more interested in product details, and sales was more interested in details of the agents (and at a lower grain—policy holders). ACM wanted a design that made sense to all of marketing and sales, not just to the power users or the IS department, who up to this point had been generating the legacy reports for everyone. Decision makers had to know the state of each policy at the end of the week (for example the coverage amounts, the policy limits, the income they earned and so forth), but they needed access to every transaction against the policy that occurred during the week. From a connectivity standpoint, all the users were located on a campus consisting of two buildings connected via Ethernet with TCP/IP.

A DS Service provider is classified as a tool that helps organize and conduct a decision maker interview process or any other tool that aids in the development of the decision-making requirements. Providers of Decision Support Services could include executive information systems (EIS) or query tools, parameter-driven queries, tool training, analysis (OLAP), data mining tools, and metadata capabilities that support data navigation or provide information about data.

ACM used ad-hoc PC query tools to access the data, and developed parameterized queries to answer anticipated business questions. They built their own tables to store metadata describing a promised degree of data quality and a data quality owner (a contact if the data in a field was of poor quality). In addition, ACM integrated the use of the metadata with query tools for data navigation and developed tool training classes for their decision makers who used real data and metadata in the class. During training, decision makers learned how to use the tools to get answers to their business questions.

Back to Top

Information Services
Information Services are the part of the Decision-Driven Warehouse infrastructure that provide the information necessary to create Decision Support Services. Information Services extract the operational data and map it to the business view needed for decision makers in DS Services. Information Services have two agents: the Staging Agent, where raw data is scrubbed, enriched, aggregated or archived for historical storage and the Knowledge Base, where staged data is ready for use by the Decision Support Services. The Knowledge Base is the source of consistent, high quality data for DS Services, mapping the raw data from staging. It may hold data at a much lower grain than is needed for a particular DS Service, but it keeps no lower grain data than is necessary for decision making in the enterprise. The Knowledge Base embodies the business requirements of all the DS Services and is adjusted as those requirements change.

Examples of Information Service providers include extraction tools, conversion tools (to convert the DS Service requirements into a legible logical schema), metadata for mapping and extraction control, sorting and transformation software, and relational database management software (RDBMS).

The Information Servicesrequirements document

bullet.gif  includes descriptions of the sources, frequency, and volumes of data received by staging.

ACM had just one data source, the application that input the daily policy and claim transactions. Initially, they decided to package all transactions into weekly warehouse updates, leaving the possibility of updating the daily transactions on a daily basis when the warehouse was mature.

bullet.gif  provides the logical design for the data that will be accessible by each DS Service.

For the marketing department, ACM designed two large denormalized tables, one for claims and one for sales. Each had a unique key consisting of time, policy, product, and transaction type. This would enable a manager, for example, to tally the income for an insurance agent for a particular product over the last three months. Losses were determined in a similar way using the claims data.

bullet.gif  identifies data overlap among the DS Services

For ACM, there was so much data overlap in the two DS Services, it made sense to have one logical design and reflect the differences in access at the tool level.
Enterprises will find a wide degree of variation here. Look for commonality where possible.

bullet.gif  establishes the logical mapping of source data received in staging to the Knowledge Base.

ACM had to map its daily transactions entered over several record types to the new design. Weekly summaries were calculated along with some generated fields, like the income earned for the week for a policy.

bullet.gif  identifies the data manipulation and data quality requirements for staging.

ACM’s initial requirements document called only for old legacy codes to be converted to legible textual descriptions. ACM did discover during a small-scale implementation that the data had not been filled in as completely as expected. ACM instituted more screening on its operational system and added scrubbing routines to the extraction program to flag missing and inconsistent data.

bullet.gif  describes steps for resolving data discrepancies found among multiple sources.
ACM fortunately did not have the problem of having multiple sources for the same data, where, for example, different operational systems might hold possibly conflicting customer information.

Back to Top

Decision Support Management Services
Decision Support (DS) Management Services provide the security, quality control, availability, accessibility, performance tuning, change control, query management, and the feedback required to maintain and improve the chosen logical design. It provides features that support the iterative processes of Decision Support System physical design, like query tracking and metadata version control. This service offers more than support to handle the platform management issues surrounding the operating system and database layers. It includes backup, accessibility, and an availability strategy because it is tied to the requirements of the decision-making process.

Examples of Management Service providers include tools that administer the security and backup strategies, query tools that monitor for query resource utilization, help desks supplying data quality feedback, and Metadata providing version control. Database administrators, data architects, system administrators, and system operators are the agents for this service. Management Services, more than any other, are usually overlooked in the execution plan for DSS. These elements are absolutely vital if you want to deliver a real solution. The DS Management Requirements define these services for a particular enterprise and are driven largely by the results of the Decision Support Services and Information Services requirements analysis. In DS Management Services, query patterns are monitored so that useful aggregate tables to improve performance are introduced into the schema when identified. Useful queries are stored and cataloged, and data that is not used, is archived or removed. Security IDs are also maintained. A mechanism to report and investigate data quality issues must be established. Impact on any existing communications network is evaluated. Changes to the source systems or the Knowledge Base mapping are managed, possibly with version control for data items as well as the software.

At ACM, the system was available from 9 p.m. to 7 a.m. for maintenance activities and the weekends for weekly loads. In addition, ACM identified three or four slow times during the year that they could rely on to reload or reorganize the databases. To classify the types of access required, they used several canned reports previously made against their operational data. This gave ACM a good idea of what the queries would be like, but did not define what aggregations would be the most popular (and the most useful for summary tables). As part of the tool evaluation, they looked for a mechanism to track this. ACM wanted to ensure high quality data, so they planned to implement a help desk for query problems, which could simultaneously be used to catch performance issues. ACM planned to use database security but wanted to decentralize the assignment of security levels as much as possible, so they devised a way to assign security managers in marketing and sales. They chose not to embed security into the query tool(s) because they wanted to do it once at the database level and have the flexibility of using several tools. With 10 years of data, ACM also had a set budget based on a “Return on Business Value” evaluation. They knew that at least two years of data had to be readily available, but they could consider aggregating the data, for example quarterly, or placing the remainder on near-line storage if necessary to meet the budget requirement. ACM wanted to recover from a disk failure very quickly and recover the most current few months if necessary within 24 hours. They were willing to be flexible on backup time to keep within budget constraints.

Back to Top

Previous | Table of Contents | Next

C-Byte Company, Inc.
  Good Morning it is 12:38 am in Las Vegas, Nevada, United States.    E-mail This Page  |  Digg This  |  Facebook  |  Twitter  |  Delicious

Ask Us a Question - Call Inside US 1-800-393-5804, Outside US 1-403-770-7818, or Contact Us
Copyright © 1989 - 2017 C-Byte Company, Inc.  ::  C-Byte Privacy Statement   ::   P.R.O. Program ::  RSS 1.0 Feed
This site is protected by copyright and trade mark laws under U.S. and International law. All rights reserved.
SecurityGroup Audits Systems at C-Byte Tackling...