In the data producer-consumer dynamic, dashboards get disproportionate attention while valuable analyses are routinely de-prioritized, and step changing self-service enablement is perpetually tabled or ignored.
There are at least three reasons for this:
- illusion of speed: in the near-term, it is faster to write some SQL, generate a dataset, and spin up a dashboard/email a report than execute an iterative analysis or explore the true underlying need in order to design and implement a sustainable self-service workflow
- the appeal of tradition: for historical reasons, dashboards have been the primary form factor for data dissemination, and our tooling is geared towards it.
- sense of accomplishment: this may seem minor but when the producer publishes a report or sends out a dataset, they can check a task off their list - we cannot underestimate the satisfaction from completing tasks, however small, in our daily grind.
As to the first reason (speed), I call this an illusion because if we slowed down just a bit to build the right models and abstractions, the next N questions and requests can be answered with significantly higher velocity.
The second reason is rooted in history, and is equal parts culture and tooling. Historically, when data collection was scarce, creating a report that accurately calculated a few select metrics was sufficient. But in a world where tracking has become abundant, we have not fully adapted to the new reality that mere calculations and displays of data is not as valuable as before.
Unfortunately, our tools reinforce this paradigm as we keep writing ad-hoc code, generating datasets and proliferating outputs - and we push the responsibility to the consumer to sift through the deluge of dashboards and reports, and to export, and further manipulate and utilize the data for their needs - all of which is a massive drag on their productivity.
Finally, as to the feeling of accomplishment - there is no greater feeling for a data producer/ analyst when data illuminates a new insight or decision and steers the business in the right direction. A far greater feeling than the near-term dopamine hit from checking off a task.
So, let’s slow down to move fast - by building the right data models and abstractions like a business semantic layer.
Let’s adopt tooling that works on the new abstractions, and gives producers (data teams) enormous leverage while empowering consumers (analysts and business teams) to truly self-serve their analysis and operational needs.
And, let’s increase those moments when data illuminates the right insight, the right decision, or the right experiment to run - the right next step to steer the business.
