We have all heard of self service reporting. What about self service data warehousing as an approach to sharing the responsibility of building data warehouses across the enterprise, while centralizing hardware resources and technology?
Imagine IT putting forward a data warehousing appliance, setting up database schemas, setting up the ETL and the reporting platform, and telling the rest of the enterprise, “Go ahead, build the Data Warehouses and BI you need”?
Each initiative that requires a data warehouse to be created, then goes ahead and creates one. IT can offer a pool of qualified technical resources, in one resource model. If the initiative owner can offer their own dedicated resources that come out of their own budget, they do so to their advantage, in terms of more control of prioritization of resources.
The individual data warehousing efforts need to coordinate with each other, perhaps, from a standards perspective, or from an optimal data storage perspective, but I see this as a lesser problem. The advantage it provides is easing the burden on the IT department of trying to meet the constantly evolving information demands of competing initiatives – addressing the perennial problem of not being able to respond quickly to changing analytic needs. It puts the onus right back on the group that is demanding the analytics. IT will assist any way it can, with expertise and even resources, but it is up to the initiative owner to figure out how to engage the right resources who would figure out the optimal design for their needs, driven entirely by the end users.
And how about a vision for the next generation of data warehousing technology to make this easier? Imagine a world where we don’t have to “design” data warehouses – no more debate on dimensional vs. third normal form modeling. No worrying about tables, columns, or indexes. The platform stores data optimally for a variety of queries. It monitors queries for performance and decides how to cache or index them accordingly. A power user is able to build up complex reusable ETL processes from simpler building blocks intuitively, and BI visualizations as well. In other words, take the “technical” out of the process and put it into the technology. Administration, of course, still belongs in IT,and can be centralized for the whole platform.
We are not there yet. When we do, building a DW/BI effort would not require more than an Administrator. The rest of the effort would come from power users that belong to various analytics groups across the enterprise. The role of a Data Warehouse Architect would be minimized, or altered, perhaps, to more of a documentation role rather than a technical one.
Blue sky? Hopefully for not much longer.