<![CDATA[
<BODY BACKGROUND="../../../images/bk_teal_corp.gif" LINK="#006666" ALINK="#009999" VLINK="#339999">
<table border="0" cellpadding="0" cellspacing="0" width="631">
<tr>
<td width="169" valign="TOP">
<table border="0" cellpadding="0" cellspacing="0" width="138">
<tr>
<td colspan="2"><img src="../../../images/clearpixel.gif" width="12" height="10" align="BOTTOM" border="0"></td>
</tr>
<tr>
<td colspan="2"><a href="../../data_org.html"><img src="../../images/but_sol_org1.gif" width="138" height="40" align="BOTTOM" alt="Data Warehousing" border="0"></a></td>
</tr>
<tr>
<td width="17" height="6"><img src="../../../images/clearpixel.gif" width="1" height="1" align="BOTTOM" border="0"></td>
<td width="121" height="6"><img src="../../../images/clearpixel.gif" width="1" height="1" align="BOTTOM" border="0"><img src="../../../images/clearpixel.gif" width="1" height="1" align="BOTTOM" border="0"></td>
</tr>
<tr>
<td width="17" valign="TOP"> </td>
<td width="121"><a href="../warehouse.htm" target="_top"><b><font size="1" face="Verdana, Helvetica, Arial">Data Warehousing</font></b></a></td>
</tr>
<tr>
<td width="17" height="6"><img src="../../../images/clearpixel.gif" width="1" height="1" align="BOTTOM" border="0"></td>
<td width="121" height="6"><img src="../../../images/clearpixel.gif" width="1" height="1" align="BOTTOM" border="0"></td>
</tr>
<tr>
<td width="17"> </td>
<td width="121"><a href="../metadata.htm" target="_top"><b><font size="1" face="Verdana, Helvetica, Arial">Meta Data Management</font></b></a></td>
</tr>
<tr>
<td width="17" height="6"><img src="../../../images/clearpixel.gif" width="1" height="1" align="BOTTOM" border="0"></td>
<td width="121" height="6"><img src="../../../images/clearpixel.gif" width="1" height="1" align="BOTTOM" border="0"></td>
</tr>
<tr>
<td width="17"><img src="../../../images/arrw_teal_corp.gif" width="17" height="10" border="0"></td>
<td width="121"><b><a href="../../../company/artcls.htm" target="_top"><font size="1" face="Verdana, Helvetica, Arial">Articles & White Papers</font></a></b></td>
</tr>
<tr>
<td width="17" height="6"><img src="../../../images/clearpixel.gif" width="1" height="1" align="BOTTOM" border="0"></td>
<td width="121" height="6"><img src="../../../images/clearpixel.gif" width="1" height="1" align="BOTTOM" border="0"></td>
</tr>
<tr>
<td colspan="2"><img src="../../../images/but_sol_mda0.gif" width="138" height="35" align="BOTTOM" alt="Multi-dimensional Analysis" border="0"></td>
</tr>
<tr>
<td colspan="2"><a href="../../ebusiness/ebusiness.htm" target="_top"><img src="../../../images/but_sol_port0.gif" width="138" height="35" align="BOTTOM" alt="Portal and Web-enabled Solutions" border="0"></a></td>
</tr>
<tr>
<td colspan="2"><a href="../../data_mov.html"><img src="../../../images/but_sol_ei0.gif" width="138" height="35" align="BOTTOM" alt="Data Movement" border="0"></a></td>
</tr>
<tr>
<td width="17"><img src="../../../images/clearpixel.gif" width="17" height="1" align="BOTTOM" border="0"></td>
<td width="121"><img src="../../../images/clearpixel.gif" width="121" height="1" align="BOTTOM" border="0"></td>
</tr>
</table>
</td>
<td width="12"> </td>
<td width="430" valign="TOP">
<p><b><font face="Verdana, Arial, Helvetica" color="#006666">Planning for a Data Warehouse</font></b><font size="2"><br>
<br>
</font><a href="planning.pdf" target="_blank"><font size="2"><img src="../../../company/images/icon_pdf.gif" width="17" height="18" align="BOTTOM" border="0"></font><font size="1" face="Verdana, Arial, Helvetica">Download a .pdf version of this article</font></a><font size="2"><br>
<br>
</font><b><font size="1" face="Verdana, Arial, Helvetica" color="#006666">By Michael Haisten</font><font size="1" color="#003366" face="Verdana, Helvetica, Arial"><br>
<br>
</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica">Few publications on the data warehouse deal with the substance of the planning process. They almost make it seem like you can pick a subject area, make some technology decisions, hire a data modeler, and hit the ground running. I have worked with numerous data warehouse projects and I can say, without qualification, that the quality of the forethought and understanding of your development team has a far greater impact on success than anything else. It isn't the technology. Few of you are reading your first data warehouse article. I kept this firmly in mind from the outset. This article does not cover the basics of data warehouse 101 and the chaotic history of information management. What it does cover, instead, is an outline of the pivotal issues to consider before committing to action.<br>
<br>
</font><b><font size="1" face="Verdana, Arial, Helvetica" color="#006666">Clarify Your Intentions</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Moral: Rationale is the foundation of your plan and a determinant of your potential.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
So you plan to build a data warehouse. Why? The concept has become so popular that it is treated as a self-justifying proposition. As in the hit film </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Field of Dreams</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica">, normally sound-mined IS professionals firmly believe: </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">If we build it, they will come!</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
<br>
One factor shared by all successful data warehouse efforts is pragmatic rationale. Articulation of the potential benefits is a minimum starting point. The ability to paint a rosy picture of future possibilities is a valuable skill. But, real success comes from effectively associating your warehouse proposal with the daily reality of the enterprise.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Don't confuse vision with rationale. Vision defines the path. Your rationale pays for the journey.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
The most powerful justifications for a data warehouse investment are qualitative goals. Typical objectives culled from actual executive presentations are:</font></p>
<ul>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">Improving information access</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">Bringing the user in touch with their data</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">Enhancing the quality of decisions</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">Providing cross-function integration</font>
</ul>
<p><font size="1" color="black" face="Verdana, Arial, Helvetica">Statements like these help illuminate your vision. A common failing, though, is to present these statements as if they are, in fact, the rationale.<br>
<br>
What does it mean to </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">improve information access</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica">? How do we ensure this has the same meaning for everyone? Every goal should not simply point out a direction. Each should tell a consistent story. Each must carry specific deliverables and must, somehow and in some form, be measurable.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">A data warehouse can rarely be justified based (solely, primarily) on operational cost reduction.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
Where do you turn to add substance to these fuzzy statements of direction? The normal tendency of IS professionals is to revert to a tactical pitch involving cost reduction. </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Reduce reporting costs by 30%</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> is typical of these cost-centered arguments. It sounds pretty specific, doesn't it, and it comes equipped with a number. What more could you ask for? Well, for one, it helps if it is based on even a shred of truth.<br>
<br>
Let's follow this chain of logic: Give me some money now and I promise to eliminate a batch of operational reports at some time in the future. I will accomplish this by creating a pool of data and training the users to wade in on their own to siphon off what they need.<br>
<br>
This is only an elaborate shell game which management will see through, sooner or later. In the best possible outcome, you simply transfer costs from the very visible IS budget into a user cost center. If lucky, you produce a slim IS-only return on investment. In a more typical case, the saving from eliminating an operational expense does not even cover the development costs. And in any scenario, the business as a whole experiences a negative return.<br>
<br>
Ironically, a data warehouse can return real benefits that go unnoticed when the promised objective is not achieved. Good work can not rise above bad rationale. Help your project build on a solid foundation.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">The best-cost objective: Achieve a permanent reduction in the full lifecycle cost of delivering information.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
A data warehouse solution can become a highly efficient process for responding to new information needs. Traditional means of extracting information from our data systems are highly labor intensive and, at best, linear in cost. Dealing with older legacy systems can result in exponential costs; it costs more and more every time you go back to the well. The objective of data warehouse implementation is to initiate a data acquisition and delivery process that offers lower marginal cost with each new user over time.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">The best overall objective: Align your goals explicitly with a strategic business initiative.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
Link your data warehouse to the strategic plan of your enterprise. Not the IS business plan - I am talking about the business business plan. I use the motto </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">AIM Higher</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> to remind me to search these plans for opportunities in three areas:</font></p>
<ul>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Analysis</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - Capture 20% market share by increasing volume and reducing the price on high margin products.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Integration</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - Decentralize by elevating the divisions to autonomous profit centers while retaining corporate-wide visibility of product movement and customer interactions.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Migration</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - Re-engineer the order process to ship 90% of orders the day they were taken.</font>
</ul>
<p><font size="1" color="black" face="Verdana, Arial, Helvetica">These strategic goals often suggest the right subject areas and business context to pursue. For instance, the analysis example implies a market subject area that commingles internal data on sales volume, pricing, and profitability with external data on market share. The integration example explicitly cites a product and customer subject area. Indirectly, it implies a financial data warehouse spanning the newly independent divisions.<br>
<br>
Re-engineering, renewal, or re-whatever often involves significant application migration activity. Migration spawns many productive justifications for a data warehouse. One can be implemented in advance to shield information consumers from disruption due to the transition. A data warehouse can simultaneously serve a forward conversion role as well as its normal information access function. A data warehouse can also supplement information access and analysis deficiencies in new applications.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">The worst (and frighteningly frequent) objective: a technical proof of concept for data warehousing.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
This is the goal of way too many pilot projects. Suffice it to say data warehouse projects justified exclusively on technical grounds have the highest failure rate.<br>
<br>
</font><b><font size="1" face="Verdana, Arial, Helvetica" color="#006666">Create a Program</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Moral: Offer a bold, comprehensive, and rigorous program with a pragmatic bias.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
With a strong, credible rationale, you are off to a solid start. The next step is to frame what you intend to do.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Spread the vision . outline a multi-year, multi-phase plan, and propose a pilot supporting this direction.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
When you articulate what the data warehouse will become, always position it as evolving incrementally to match the changing needs of the enterprise. In early releases, you will be perfecting techniques and building technical infrastructure. Make sure management realizes that each early release will tackle a new business subject area and will offer additional information deliverables. Then prepare to fulfill these promises.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Vision background; not just another database, not just a project, more than infrastructure.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
A data warehouse is not just another database. Most database designs are derivative . intended to serve the processing requirements of a specific application. A data warehouse database is both primitive and primary . intended as the definitive source for a broad spectrum of basic business facts. The prime directive of traditional database development is to </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">optimize efficient handling of a known set of transactions</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica">. The data warehouse prime directive is to </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">optimize availability of cross-functional data in support of future needs</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica">.<br>
<br>
A data warehouse is not just a project. A project is the </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">managed organization of resources to produce a defined set of deliverables in a specified timeframe</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica">. A data warehouse effort does involve several time-sequenced deliverables: raw atomic data acquired from operational systems, collections of data prepared for use, and executable procedures to perform this processing. However, these deliverables are produced using exploratory techniques that are radically different from traditional methodologies. And, though there is a definite beginning, there should be no end to the data warehouse. Sourcing, packaging, and delivering data will continue and evolve as the business changes.<br>
<br>
But the differences go deeper than this. When you initiate a data warehouse implementation, you are investing in a whole array of new services supported by the deployment of new technical infrastructure. You select and introduce analysis tools. You plan and implement training programs. Support procedures need to be designed and support mechanisms must be put into place. This effort takes place amidst an evolving set of new information management practices.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">A data warehouse is a program to manage sharable information acquisition and delivery universally.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
A data warehouse, like your neighborhood library, is both a resource and a service. The value of library resources is deter-mined by the breadth and depth of the collection. The value of library services is based on how quickly and easily they can assist you in finding and using what you need. Unlike a library, a data warehouse must take on the role of manufacturer and distributor as well. As in a factory, raw materials are collected from operational systems and packaged for use by information consumers. Similar to a public utility, a data warehouse uses a common distribution network to deliver products to the point of use. The value of a factory is determined by how well it produces the finished goods that consumers most need today and into the future. The value of a utility is measured by the efficiency and timeliness of delivery.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Avoid the false promises of the "just do it" advocates.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
Whew, this sounds like you must propose an expensive mega-project. Don't all the other data warehouse advisors say that you can - in fact, that you should - start on a small-scale, limited-exposure, pilot project?<br>
<br>
If you follow the advice of the </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">quick start</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> or </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">just do it</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> advocates literally, you are signing up for a quick trip to nowhere. Many people are duped into thinking that a pilot project is the limit of their planning horizon. There is nothing wrong with moving out fast if you know where you are going. Starting with a real plan need not take forever, only two or three months. It will be time well spent.<br>
<br>
Take the time to look for strategic opportunities. Create the outline of a long-term plan. Define potential future destinations and alternate paths to get there. Pitch your pilot as the logical first step.<br>
<br>
</font><font color="#006666"><b><font size="1" face="Verdana, Arial, Helvetica">Outline Information</font></b></font><font size="1" face="Verdana, Arial, Helvetica" color="#006666"><br>
</font><font color="#006666"><b><font size="1" face="Verdana, Arial, Helvetica">Deliverables</font></b></font><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Moral: Learn the differences about data warehousing since commonly held beliefs will cause you to fail.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
Now let's turn our attention to the outputs of a data warehouse. First, it helps to understand the various types of data that should be stored on your warehouse shelves. Then we can discuss how you decide what to put into these containers.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">A data warehouse contains a far wider variety of containers than traditional applications. </font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
Expect to design four major types of containers: core data, collections, archive, and meta data. (see box, below)<br>
<br>
<table border="1" cellpadding="0" cellspacing="0" width="100%" bgcolor="#FFFFCC">
<tr>
<td width="100%">
<p><b><font face="Verdana, Arial, Helvetica" color="#006666">Data warehouse containers</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
<br>
</font><b><font size="1" color="black" face="Verdana, Arial, Helvetica">CORE DATA:</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> Raw data acquired and lightly refined form the operational system.</font></p>
<ul></ul>
</td>
</tr>
</table>
</font></p>
<li><b><font size="1" color="black" face="Verdana, Arial, Helvetica">Base</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> - basic elemental business data at the most granular level (ideally atomic)</font>
<li><b><font size="1" color="black" face="Verdana, Arial, Helvetica">Reference</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> - descriptive or identifying information for based data (i.e., master files)</font>
<li><b><font size="1" color="black" face="Verdana, Arial, Helvetica">Pre-derived</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> - calculated or summarized values retained to ensure auditability of results</font></UL>
<p><b><font size="1" color="black" face="Verdana, Arial, Helvetica">COLLECTIONS:</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> Packaged sets of information prepared to satisfy consumer needs.</font></p>
<ul>
<li><b><font size="1" color="black" face="Verdana, Arial, Helvetica">Aggregate</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> - accumulation of related data into a single table to improve ease of access (i.e., pre-joined)</font>
<li><b><font size="1" color="black" face="Verdana, Arial, Helvetica">Summary</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> - consolidated data summarized on one or more dimensions from base tables</font>
<li><b><font size="1" color="black" face="Verdana, Arial, Helvetica">Prepack</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> - packaged data in the unique structure (e.g., starjoin, hypercube) required by a specific tool</font>
<li><b><font size="1" color="black" face="Verdana, Arial, Helvetica">Export</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> - unloaded flat file data created for a specific use and requiring delivery to another location</font>
</ul>
<p><b><font size="1" color="black" face="Verdana, Arial, Helvetica">ARCHIVE:</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> Active extension of a warehouse that retains enterprise history for on-demand retrieval.</font></p>
<ul>
<li><b><font size="1" color="black" face="Verdana, Arial, Helvetica">Atomic history</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> - atomic level data placed in an archive immediately after base tables acquisition</font>
<li><b><font size="1" color="black" face="Verdana, Arial, Helvetica">Time fragment</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> - dataset containing data for a defined timeframe and optimized for selective retrieval</font>
<li><b><font size="1" color="black" face="Verdana, Arial, Helvetica">Context history</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> - periodically stored snapshots of reference, definition, and navigation data</font>
<li><b><font size="1" color="black" face="Verdana, Arial, Helvetica">Strategic results</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> - saved set of any collection deemed strategic for "corporate memory."</font>
</ul>
<p><b><font size="1" color="black" face="Verdana, Arial, Helvetica">META DATA:</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> Data about data to facilitate access, integrated tools and utilities, and automated processing.</font></p>
<ul>
<li><b><font size="1" color="black" face="Verdana, Arial, Helvetica">Definition</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> - data names, formats, meanings, source, usage, and other defining characteristics</font>
<li><b><font size="1" color="black" face="Verdana, Arial, Helvetica">Navigation</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> - catalog of available data, cross-reference summary, and help/support information</font>
<li><b><font size="1" color="black" face="Verdana, Arial, Helvetica">Administration</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> - control data for process management, monitoring, and integration across the data warehouse</font>
<p><b><font size="1" color="black" face="Verdana, Arial, Helvetica">Core data</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> is the raw material from which all other data tables are built. This data should be brought in with minor cleanup and conversions to make it more useful. Avoid selective acquisition and take all rows from the source. Avoid summarization on the way in and keep the detail. Avoid creating new calculated values at this point, but it is okay to import derived data when it is essential to maintain auditability. New derived data is created later when the collections are produced. Core data is typically stored in a normalized form.<br>
<br>
</font><b><font size="1" color="black" face="Verdana, Arial, Helvetica">Collections</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> are manufactured in the warehouse environment to satisfy known and anticipated needs. Aggregations are produced to support repetitive access of related data drawn from multiple base tables. Summaries are created based on usage experience, not by </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">a priori</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> analysis. Prepacks and exports satisfy specific needs of your consumers or the tools they use. By definition, all collections are denormalized.<br>
<br>
Though the terms used here may be unfamiliar, virtually every warehouse effort builds tables containing the kind of data described above. Unfortunately this is not the case for archive and meta data. Though extensive lip service is given to the need to maintain history, few projects go beyond adding timestamps to online data warehouse tables. A virtual avalanche of words is lavished on the essential role of meta data. Yet in the daily grind, this is always something that can be addressed in the next release.<br>
<br>
The central role of the archive and comprehensive use of meta data are two features that clearly set the data warehouse concept apart from prior information access initiative. Build them into your plan at an early stage. As with everything else, you can start small, but do include these features.<br>
<br>
The </font><b><font size="1" color="black" face="Verdana, Arial, Helvetica">Archive</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> contains data packaged for easy retrieval and reintegration with the online warehouse data. Raw data at the most granular level (atomic) is stored to ensure no loss of information over time. In some cases, the archive is the only place where this level of detail is available. Context history is stored to ensure that you can recreate the data environment that existed at any period.<br>
<br>
</font><b><font size="1" color="black" face="Verdana, Arial, Helvetica">Meta Data</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"> is the most discussed but least understood. Most people think of a data dictionary first, but this is the least valuable form of definition meta data. Within this subcategory, knowing the ultimate source for a data element, meanings rather than descriptions, and references to how the data is commonly used are more important than the basic data dictionary-style information.<br>
<br>
Navigation data is intended to help the consumer find what exists and how to use it. The card catalog analogy suggests some of the possibilities of navigation. The least understood form of meta data, administrative, is possibly the most powerful in the long run. It plays an essential role in process integration and automation.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">A data warehouse must be designed to support unknown future needs of unidentified consumers.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
When you turn your attention to selecting the data for your core tables, you immediately run into a wall. How can we possibly plan for the unknown? The most difficult hurdle is to realize you must try.<br>
<br>
There are three classic ways to fail when an information consumer searches for data: 1) not having the data in the form they desire, 2) not having the required detail, or 3) not having the data element they need at all. The first problem is a matter of design that can be fixed on the fly. A collection can be built to satisfy this need from core data already in the warehouse. The latter two, however, are a factor of advance planning. It takes considerable time to acquire missing data - time the consumer does not have.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Traditional requirements gathering methods must be abandoned or extensively modified.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
Traditional practices and commonly held beliefs lead us to:</font></p>
<ul>
<ul>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">Focus too concretely on what existing users know they need and give them only what they ask for;</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">Eliminate data not known to be required in order to control the scope of the project or size of the table;</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">Consider users closest to the source systems to be the primary information consumers;</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">Fixate on response time and storage issues and other operational concerns.</font>
</ul>
</ul>
<p><font size="1" color="black" face="Verdana, Arial, Helvetica">Only getting what is a known requirement leads to repeated, costly trips to the well. Try, instead, to:</font></p>
<ul>
<ul>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">Anticipate data needs based on business trends, all potential usages, intuition, etc.</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">Include data not known to be worthless in order to eliminate future costs which will be greater;</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">Seek potential consumers widely, particularly those not close to the source in space or function;</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">Concentrate on availability and flexibility while creatively managing performance and volume issues.</font>
</ul>
</ul>
<p><font size="1" color="black" face="Verdana, Arial, Helvetica">Start from the premise that you will drain every potentially usable nibble of data out of the source systems. Fight at all turns the tendency to accept summarized data rather than the ultimate atomic level of detail. Make your associates justify not including data rather than the other way around. You will have to back off, to some degree, but you are coming from a more defensible position for the long run.<br>
<br>
</font><b><font size="1" face="Verdana, Arial, Helvetica" color="#006666">Outline the Technology</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Moral: How you build it is at least as important as what you build.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
Much attention has been devoted to Information Architecture in data warehouse articles and conferences. The vast majority of data warehouse speakers and writers are (former) database analysts, data modelers, data analysts or other practitioners of data-centric fields. Almost no attention is given to technical architecture or the practice of defining the optimal way to build a warehouse.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Process components are the building blocks of a data warehouse.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
A data warehouse is more than a computer, a DBMS, a network, and a client/server query tool. These things are the infrastructure services that form the backbone upon which you build a data warehouse. But what is the essence of the data warehouse itself?<br>
<br>
From the perspective of technical architecture, a data warehouse is a set of processing components that interact to make enterprise information available to information consumers:</font></p>
<ul>
<ul>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Acquisition</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - bringing data into the warehouse form the operational environment.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Preparation</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - packaging the data for a variety of uses by a diverse set of consumers.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Operations</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - controlling repetitive tasks and managing warehouse inventory for optimal processing.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Navigation</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - assisting the information consumer in finding and using warehouse resources.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Access</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - supporting direct interaction with data warehouse data by consumers using a variety of tools.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Delivery</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - providing subscription delivery of data as an alternative to interactivity.</font>
</ul>
</ul>
<p><font size="1" color="black" face="Verdana, Arial, Helvetica">To build a complete data warehouse solution you need to build, buy or manually perform these services.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Create a technical plan for your data warehouse that is technology-free.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
A well-defined technical plan allows you to specify how your data warehouse will function and what services it will provide. The plan can define what technical components you intend to offer initially, and with subsequent releases, thus providing a common framework defining growth over time.<br>
<br>
A technical plan that does not list any vendor-specific products by name (technology-free) provides an excellent foundation for making the decision of what to buy, build or perform manually. It can be used to frame the requirements for product selection and, later, to provide purchase justification for essential components.<br>
<br>
</font><b><font size="1" face="Verdana, Arial, Helvetica" color="#006666">Determine Automation Targets</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Moral: Failure to automate a data warehouse leads to failure.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
The basic development tasks are so deceptively simple that most people resort to our normal manual practices. This includes writing all the code by hand and writing a separate custom procedure for each activity.<br>
<br>
To continue to add value, a data warehouse must adapt to changing needs. New data must be continually acquired. Available raw data must continually be packaged and repackaged to support new demands, tools, and forms of analysis. The key to survival is to remain dynamic. The key to remaining dynamic is to automate repetitive tasks and reuse the prior deliverables whenever and wherever you can.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Go from design to implementation in one step.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
The holy grail of Computer-Aided Systems Engineering is to eliminate all coding. For the most part, CASE has been a monumental failure. Sophisticated upper CASE modeling tools and methods failed to reduce development time, and resource requirements often increased. Now in the specialized field of data warehouse development, CASE techniques are finally beginning to pay off. I will give you just two examples.<br>
<br>
First, it is now possible to use graphical modeling tools to design, implement and maintain a data warehouse database. In fact, this is the preferred method. Maintaining synchronization between the graphical image and the physical database is essential. You accomplish this by eliminating all manual manipulation of the warehouse database data definition language statements.<br>
This can and does work for a data warehouse.<br>
<br>
The second high productivity area involves the new breed of </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">extract and load</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> code generators. Using very different approaches, these products allow you to define the data source(s), the warehouse target table, and the mappings and transformations to get from one to the other. They then generate code to operate on the specific platforms in your environment. Productivity is increased by a factor of two to five times for initial development, and is considerably greater for revisions to existing feeds. Given the number of data feeds, the rapid change, and the ability to reuse prior definitions, these products pay for themselves very quickly.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Buy generalized utilities whenever and wherever possible. Otherwise, consider building them.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
Though the current concept of a data warehouse is new, many tasks necessary to develop and operate one are well understood. Existing tools and utilities can be harnessed for data warehouse use. New products are regularly introduced to simplify data design or management tasks. Where a repetitive activity exists, there is an opportunity to develop a custom utility. Examples abound. Here are just a few:</font></p>
<ul>
<ul>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">A reverse engineering utility can be used to create a logical model of the source application.</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">There is a growing category of products that scan the data itself to help you define one-time cleanup operations or ongoing conversions necessary to repair integrity or consistency error.</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">Transformation products exist for most platforms and format combinations to simplify data movement.</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">Some analysis tools require data to be structured in a very explicit fashion. Rather than write a unique piece of code for each occurrence, write a generic utility to prepare data for these products.</font>
</ul>
</ul>
<p><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Provide the glue to integrate off-the shelf products into a seamless process.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
Many labor-intensive data warehouse tasks can be streamlined by simply stitching together existing products and procedures. For instance, the dynamic nature of a data warehouse means there are many frequent changes in the batch processing cycle. Wouldn't it be nice if you could update a diagram of the batch dependency plan to change the schedule? How about automating the process of error escalation? Think how much easier it would be if all interested parties were automatically notified by e-mail when a data load fails.<br>
<br>
One solution incorporates a PERT charting applications, the DBMS of your choice, your existing schedule management software, one or more mail packages and a mail gateway along with a client/sever development tool and a bit of code. All this functionality, and more, can be made available by creative use of building blocks.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Plan for automation and reuse in your first project and expand from there.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
No data warehouse program can operate cost-effectively without introducing a high degree of automation and reusability. Many opportunities exist. Just search for repetitive tasks that can be reduced or eliminated. Don't wait. Begin in the pilot.<br>
<br>
</font><b><font size="1" face="Verdana, Arial, Helvetica" color="#006666">Select the Tools</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Moral: Abandon completely even the desire to find a silver bullet.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
Many data warehouse project teams waste enormous amounts of time searching in vain for a silver bullet. They believe their mission is to find the one access tool that will handle all the needs of all their users. Don't even try. It can't be done. One size does not fit all.<br>
<br>
There is a wide and ever-growing diversity of tools for data access, exploration, analysis, and presentation. The appropriate mission of a data warehouse or decision support team is to understand these diverse alternatives and properly match the tool to the intended usage.<br>
<br>
The names typically applied to tools are ambiguous and confusing. Generally every new name only obscures the issue more. What is the definitive definition of </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">ad hoc query tool</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica">, </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">decision support system</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica">, </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">executive information system</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> or </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">online analytic processing</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica">? Some terms, like EIS, carry the burden of years of overzealous selling and underwhelming results.<br>
<br>
To evaluate tools, you need to slot the alternative into categories that allow for meaningful comparison. Since the traditional terms add little discriminatory power, where do you turn? The first part of the answer is to create purely functional categories. For example:</font></p>
<ul>
<ul>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Browser</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - simple direct access to DBMS tables or views without join or business semantic layer support.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Query</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - two-level client/server tool providing complex direct access to data with full SQL support.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Multi-dimensional analysis (MDA)</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - provides unhindered data explosion, drill-down, and surfing capabilities.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Syntax</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - language-based (4GL) direct access to data with full SQL support.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Fixed</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - pre-planned data access with generally pre-computed options (EIS tools often fit here.).</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Custom</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - custom-developed data access exploration, analysis or presentation application. These applications may be developed using a specialized DSS toolkit, 3GL, or 4GL.</font>
</ul>
</ul>
<p><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
One category missing from this list includes tools such as PC spreadsheets, charting and presentation tools. In most enterprises today, they are the final packaging vehicles for management presentation. However, they are rarely the tools of choice for direct access to the data warehouse. Also, this taxonomy only includes interactive tools. Many mature technologies exist for batch reporting. A new class of non-interactive tools will come on the market over the next several years. They will use agent-processing techniques to scan remote databases and pass result data back to the client asynchronously.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Match the tool to the consumer, context, and usage.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
The next step in selecting access tools is to survey your potential consumers. You need to collect information in three broad interrelated areas: 1) who they are and what their role is, and how they operate, 2) what the business context is, and 3) their typical usage of data. This information will help you select the appropriate tool category(s) for each consumer. Then rank order your potential consumers according to who will be the most involved, productive, and powerful participants. The priority tool categories should become apparent. Several examples follow:</font></p>
<ul>
<ul>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">A </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">product manager</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica">, who interacts extensively with data, identifying patterns of purchasing behavior to plan new marketing campaigns, will probably be best served by a multi-dimensional analysis tool.</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">A </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">marketing analyst</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> studying store site demographics to profile differences between selected geographic areas is more likely to need a query tool. Analysts who are very business savvy, but not very technically inclined, will be better off with a query tool that provides a business-like, semantic interpretation layer.</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">When you see repetitive activity involving the same selection criteria and domain, the power of query and MDA tools is overkill. A </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">financial analyst</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> who regularly produces a listing of the top and bottom performing products should use a browsing tool.</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">When a </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">manufacturing quality assurance group</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> routinely studies defects using standard variables and analyses, a fixed mode tool or a custom application is suggested.</font>
</ul>
</ul>
<p><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Build a best-in-class tool set of supported products.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
Of course you can provide quality support for only a small, finite number of tools. After identifying the tool categories most appropriate for your consumers, you are prepared to evaluate what the market has to offer. The objective is to select the tools you intend to support in each category.<br>
<br>
You will never be able to eliminate pirate activity. Some consumers will always find justifiable reason to pick a non-supported product. We have found that it is far more productive to explain the value of selecting a supported product than to wage a battle to enforce a </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">standard</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica">. You will lose customers and goodwill.<br>
<br>
Remember to plan regular surveys of the market. You need to remain aware of new features of existing products, new products in each category, and the advent of wholly new categories. Be ready to change your supported product mix when there is genuine need. Resist the tendency to move capriciously to the next best thing. You and your customers need time to become proficient and successful before moving on.<br>
<br>
</font><b><font size="1" face="Verdana, Arial, Helvetica" color="#006666">Lay Out the Project</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Moral: A data warehouse project is more like scientific research than anything in traditional IS!</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
The normal IS approach places considerable emphasis on knowing what the expected results are before committing to action. In scientific research, the results are unknown up front, and emphasis is placed on developing a rigorous, step-by-step process to uncover the truth. The activities are hands-on, involving regular interaction between the scientist and the subject and among the participants on the project.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Adopt an exploratory, hands-on process involving cross-disciplinary participation.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
The following situations indicate a project in trouble because these techniques are not being used:</font></p>
<ul>
<ul>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">The project has proceeded for two months and nobody has touched the data.</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">The future consumers are not involved hands-on from day one throughout the program.</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">The team members doing data design (modelers and DBAs) have never used the access tools.</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">The summary tables are defined before the raw atomic data is acquired and base tables have been built.</font>
<li><font size="1" color="black" face="Verdana, Arial, Helvetica">The data design is finished before participants have experimented with the tools and live data.</font>
</ul>
</ul>
<p><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Initiate multiple parallel activity tracks simultaneously.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
A data warehouse project involves a wide range of deliverables that can be worked on in parallel. The track scheme we advocate is:</font></p>
<ul>
<ul>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Management</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - obtain sign-off, document milestone activity, and coordinate track deliverables.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Consumer</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - discover the consumer needs and design the packaging requirements.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Model the database</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - define and iteratively implement the database.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Source data</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - define the technical architecture, select products, and perform the build and implementation tasks.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Tools</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - evaluate, select and make available the access tool set.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Operations</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - plan operational infrastructure and processing cycles.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Support</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - develop documentation, and design and implement training programs and help-desk services.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Integration</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - integrate the meta data and deliver navigation services.</font>
</ul>
</ul>
<p><font size="1" color="black" face="Verdana, Arial, Helvetica">This scheme offers a division of labor proven to work in practice. You do not need nine or more people to make this scheme work for you. But it is hard to gain the advantages of parallel activity with only a few participants.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Forcing closure on specific results too soon leads inexorably to disaster.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
A general rule of thumb is to break the initial project roughly into thirds: </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">initiation</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica">, </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">exploration</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica">, and </font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">implementation</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica">. We assume that the project begins after the planning activity, when you assemble the bulk of the team for action.<br>
<br>
</font><b><font size="1" face="Verdana, Arial, Helvetica" color="#006666">Initiation Phase</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
This phase continues through delivery of the first trial database of live core data. On the consumer track, you outline the expected information deliverables. On the source data track, basic mapping of data from the source systems is defined. The model database track delivers a data warehouse data model containing only core tables and then proceeds to create and load the initial data warehouse database.<br>
<br>
Ideally, the access tools are selected by the end of this phase. Optionally, you may identify finalists that are to be evaluated during the exploratory phase. Design deliverables are created for the other tracks (technical plan, support, operations, and integration). All the technology to support the development of the data model, database, and data acquisition procedures must be ready when needed during the initiation phase.<br>
<br>
</font><b><font size="1" face="Verdana, Arial, Helvetica" color="#006666">Exploration Phase</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
This phase involves iterative revision of the database and the preparation of procedures that maintain it. On the consumer track, a team of technical and consumer participants uses the access tools suite to develop representative samples of anticipated results. Based on the experience, the model database team members define aggregates and summaries that offer a good balance of user flexibility, production efficiency, and runtime performance. On the source track, the participants refine the mapping and transformation logic based on hands-on experience with the data. They clean up the source data and define conversion logic to correct consistency or integrity problems. They modify the core tables based on tradeoffs between acquisition and preparation activity. All technical infrastructure and tools work concludes during the exploration phase. The operations track prepares for implementation and production migration. The support track prepares for initial consumer training in the next phase. Navigation services, the meta data tables, and the meta data maintenance procedures are developed at this time.<br>
</font><font size="1" face="Verdana, Arial, Helvetica" color="#006666"><br>
</font><font color="#006666"><b><font size="1" face="Verdana, Arial, Helvetica">Implementation Phase</font></b></font><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
This phase starts with a freeze on source investigation and consumer experimentation. It concludes with formal release of the current version of the data warehouse. The final data definition and data manipulation and procedural changes are made. All component services (acquisition, preparation, operations, navigation, access, and delivery) are installed and unit tested. The production schedule is implemented and a full system test is conducted. Initial consumer orientation and training are offered during the production migration activity.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Continue to look over the horizon at the future possibilities.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
To ensure that the data warehouse program evolves continuously, management must maintain focus on future opportunities.<br>
<br>
By the beginning of the implementation phase of the current project, management must already be planning the subsequent project. What new subject area will be tackled? What new consumers will be recruited? What new techniques will be tried? What new technical components will be implemented? And very importantly, how has what you are offering been received and adopted?<br>
<br>
</font><b><font size="1" face="Verdana, Arial, Helvetica" color="#006666">Solicit a Team</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Moral: A cross-functional program must be inter-disciplinary. Use maximum leverage.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
The question I am most often asked is: . How many people are needed to build a data warehouse?. My typical answer is: Anywhere from two and a third to a cast of hundreds. From project to project, there are substantial differences in objective and scale. But the resource question is also biased by how you define participation.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">When is 2.33 equal to 4 or more?</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
The smallest project in my experience used two systems analysts and a part-time database programmer to build a data warehouse from scratch in eight month. Their subject area, R&D resources, was quite small. Their technical objectives were limited and they supported only one access tool. Yet, they are successful enough to move into other subject areas and add new consumer groups while expanding their direct staff to five. When you look closer, the 2.33 full-time equivalents significantly undercounts real participation. At least four full-time equivalents were expended on development alone. This includes developer on the source systems who implemented new and modified data feeds, reviews by a DBA, vendor contractors, an advisory consultant, and others. An unknown number of consumers, developers, and managers were involved before, during, and after development.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Consciously manage multiple spheres of involvement. This is the only real way to "get more from less."</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
Everyone wants to know the minimum number of people it will take to get the job done. When pressed for a single number, my answer is seven. This is roughly the critical mass needed to sustain a successful program for a medium to large business. But how this resource is structured varies too wildly to draw any general conclusions. The creative manager builds a small core team and borrows the rest. This is the key to delivering more with less. Our definition of levels of involvement is:</font></p>
<ul>
<ul>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Core</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - dedicated individuals whose sole mission is the data warehouse program.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Extended</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - team members form developer or consumer groups with specific roles over the entire project.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Participant</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - individuals who are assigned specific deliverables at one or more points in the project.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Involved</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - people consulted for expertise, managers solicited for influence, etc.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Impacted</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - consumers or developers whose work patterns will be changed by the introduction of the data warehouse.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Aware</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - anyone who is relatively current in their knowledge of the data warehouse objectives and progress.</font>
<li><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Oblivious</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"> - those who know little or nothing about the data warehouse program.</font>
</ul>
</ul>
<p><font size="1" color="black" face="Verdana, Arial, Helvetica">Actively solicit extended team members and participants from other groups, departments, and cost centers. A data warehouse requires broad expertise from multiple disciplines. Sign on developers from the source systems to acquire the data. Enlist consumers and the IS support personnel. All the IS infrastructure groups need to be on your side. If not, they will be in your face.<br>
<br>
Reaching out to other groups for expertise increases the management hassles, no doubt. But it can be immeasurably valuable in solidifying cross-functional support. Your data warehouse project should be the future direction for information access. Competition is wasteful and deadly. Don't even try to do it all yourself. Keep the core team small and recruit a cast of hundreds.<br>
<br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">The talent you need on the core team is the hardest to find.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
So, who should be on this small core team? Typically, the first talent hired is a data modeler and the second is a database analyst or DBA. While you need these data specialists, you don' t need them as full-time, permanent participants of the core team. The best individuals at the core of your team are people who intimately understand the broad issues of a complete data warehouse program. Ideally, they studied the field extensively and participated in at least one successful project. These folks are hard to come by.<br>
<br>
Your next choice should be those with experience of packaging data for consumer use: a data analyst with DSS experience, a 4GL programmer with information center or end-user support background, a data. centric developer with real experience using spiral or interactive methods. You need data engineers to get started and to add new data resources to the warehouse. They create the potential. You will need data analysts or data usage specialists to stay in business. They deliver results.<br>
<br>
</font><b><font size="1" face="Verdana, Arial, Helvetica" color="#006666">Evangelize Endlessly</font></b><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
</font><i><font size="1" color="black" face="Verdana, Arial, Helvetica">Moral: Tell them what you are going to say, say it, then tell them what you said.</font></i><font size="1" color="black" face="Verdana, Arial, Helvetica"><br>
The story of a successful data warehouse program is of a story often told. You need to keep the vision alive while delivering results. When you begin planning for your own data warehouse, plan to spend a considerable amount of your time on the campaign trail selling and reselling your concept.</font></p>
</ul>
</td>
</tr>
</p></UL></TD></TR>
</table>
<TABLE BORDER="0" CELLPADDING="0" CELLSPACING="0" WIDTH="608">
<TR>
<TD WIDTH="25"> </TD>
<TD COLSPAN="3">
<CENTER>
<P>
<HR ALIGN="CENTER" SIZE="1" NOSHADE>
<BR>
<A HREF="../../../index.html" target="_top"><FONT SIZE="1" FACE="Verdana, Helvetica, Arial">home</FONT></A><FONT
SIZE="1" FACE="Verdana, Helvetica, Arial"> | </FONT><A HREF="../../../aboutus.html" target="_top"><FONT SIZE="1"
FACE="Verdana, Helvetica, Arial">about us</FONT></A><FONT SIZE="1" FACE="Verdana, Helvetica, Arial"> | </FONT><A
HREF="../../../solutions.html" target="_top"><FONT SIZE="1" FACE="Verdana, Helvetica, Arial">solutions</FONT></A><FONT
SIZE="1" FACE="Verdana, Helvetica, Arial"> | </FONT><A HREF="../../../products.html" target="_top"><FONT SIZE="1"
FACE="Verdana, Helvetica, Arial">products</FONT></A><FONT SIZE="1" FACE="Verdana, Helvetica, Arial"> | </FONT><A
HREF="../../../partners.html" target="_top"><FONT SIZE="1" FACE="Verdana, Helvetica, Arial">partners</FONT></A><FONT
SIZE="1" FACE="Verdana, Helvetica, Arial"> | </FONT><A HREF="../../../clients.html" target="_top"><FONT SIZE="1"
FACE="Verdana, Helvetica, Arial">clients</FONT></A><FONT SIZE="1" FACE="Verdana, Helvetica, Arial"> | </FONT><A
HREF="../../../careers/home.html" target="_top"><FONT SIZE="1" FACE="Verdana, Helvetica, Arial">careers</FONT></A><FONT
SIZE="1" FACE="Verdana, Helvetica, Arial"> | </FONT><A HREF="../../../contact.html" target="_top"><FONT SIZE="1"
FACE="Verdana, Helvetica, Arial">contact us</FONT></A><FONT SIZE="1" FACE="Verdana, Helvetica, Arial"> | </FONT><A
HREF="../../../search.htm" target="_top"><FONT SIZE="1" FACE="Verdana, Helvetica, Arial">search</FONT></A><A HREF="../../../sitemap.html"
target="_top"><FONT SIZE="1" FACE="Verdana, Helvetica, Arial"><BR>
</FONT></A><FONT SIZE="1" FACE="Verdana, Helvetica, Arial"><IMG SRC="../../../images/clearpixel.gif" WIDTH="1"
HEIGHT="10" ALIGN="BOTTOM" BORDER="0"></FONT>
</CENTER>
</TD>
</TR>
<TR>
<TD WIDTH="25" VALIGN="BOTTOM"> </TD>
<TD WIDTH="121" VALIGN="TOP">
<P ALIGN="RIGHT"><IMG SRC="../../../images/logo_sm.gif" WIDTH="120" HEIGHT="40" ALIGN="BOTTOM" BORDER="0">
</TD>
<TD WIDTH="12"> </TD>
FACE="Verdana, Helvetica, Arial">Texas</FONT><FONT SIZE="1" COLOR="white" FACE="Verdana, Helvetica, Arial">.</FONT><FONT
SIZE="1" COLOR="#666666" FACE="Verdana, Helvetica, Arial">78746 tel(512)329-6646,</FONT><FONT
SIZE="1" COLOR="white" FACE="Verdana, Helvetica, Arial">.</FONT><FONT SIZE="1" COLOR="#666666" FACE="Verdana, Helvetica, Arial">fax(512)329-0767<BR>
<IMG SRC="../../../images/clearpixel.gif" WIDTH="12" HEIGHT="5" ALIGN="BOTTOM" BORDER="0"><BR>
</FONT><I><FONT SIZE="1" COLOR="#999999" FACE="Verdana, Helvetica, Arial">© Copyright 1999-2003, Daman Incorporated. All rights reserved. All trademarks are the property of their respective owners.</FONT></I></TD>
</TR>
<TR>
<TD WIDTH="25"><IMG SRC="../../../images/clearpixel.gif" WIDTH="25" HEIGHT="1" ALIGN="BOTTOM" BORDER="0"></TD>
<TD WIDTH="121"><IMG SRC="../../../images/clearpixel.gif" WIDTH="121" HEIGHT="1" ALIGN="BOTTOM" BORDER="0"></TD>
<TD WIDTH="12"><IMG SRC="../../../images/clearpixel.gif" WIDTH="12" HEIGHT="1" ALIGN="BOTTOM" BORDER="0"></TD>
<TD WIDTH="450"><IMG SRC="../../../images/clearpixel.gif" WIDTH="450" HEIGHT="1" ALIGN="BOTTOM" BORDER="0"></TD>
</TR>
</TABLE>
</BODY>
]]>