Self Service Data Warehousing

We have all heard of self service reporting. What about self service data warehousing as an approach to sharing the responsibility of building data warehouses across the enterprise, while centralizing hardware resources and technology?

Imagine IT putting forward a data warehousing appliance, setting up database schemas, setting up the ETL and the reporting platform, and telling the rest of the enterprise, “Go ahead, build the Data Warehouses and BI you need”?

Each initiative that requires a data warehouse to be created, then goes ahead and creates one. IT can offer a pool of qualified technical resources, in one resource model. If the initiative owner can offer their own dedicated resources that come out of their own budget, they do so to their advantage, in terms of more control of prioritization of resources.

The individual data warehousing efforts need to coordinate with each other, perhaps, from a standards perspective, or from an optimal data storage perspective, but I see this as a lesser problem. The advantage it provides is easing the burden on the IT department of trying to meet the constantly evolving information demands of competing initiatives – addressing the perennial problem of not being able to respond quickly to changing analytic needs. It puts the onus right back on the group that is demanding the analytics. IT will assist any way it can, with expertise and even resources, but it is up to the initiative owner to figure out how to engage the right resources who would figure out the optimal design for their needs, driven entirely by the end users.

And how about a vision for the next generation of data warehousing technology to make this easier? Imagine a world where we don’t have to “design” data warehouses – no more debate on dimensional vs. third normal form modeling. No worrying about tables, columns, or indexes. The platform stores data optimally for a variety of queries. It monitors queries for performance and decides how to cache or index them accordingly. A power user is able to build up complex reusable ETL processes from simpler building blocks intuitively, and BI visualizations as well. In other words, take the “technical” out of the process and put it into the technology. Administration, of course, still belongs in IT,and can be centralized for the whole platform.

We are not there yet. When we do, building a DW/BI effort would not require more than an Administrator. The rest of the effort would come from power users that belong to various analytics groups across the enterprise. The role of a Data Warehouse Architect would be minimized, or altered, perhaps, to more of a documentation role rather than a technical one.

Blue sky? Hopefully for not much longer.

1 Comment

  1. Old wine, new skin, you know.

    I like the idea descibed here, but I suppose that this what we could already have. Imagine a central Staging Area or Operational Data Store, the place source data is stored, maybe in historized form. Then any user group could connect and build up its own application. I avoid the usage of Data Mart, because this is often linked to Cubes and Dimensions etc, but it is this.

    Marketing department need sth. more relational to perform data mining, Controling need fixed reports. What you will do is moving IT knowledge into the business departments, wouldn’t you? These are no longer Jeans/Shirt/Pizza/Coke guys, but from now the smart ones, but under the shelf still IT guys, doing mappings or what ever needed to build the application. There may be tools offering a nice GUI and less programming to do so, but sth. needs to be set up.

    Could this be a new definition of Enterprise Data Warehosue? Central Staging and connected Applications, but no central corporate wide consolidation?


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s