Keeping Up With Data #103
5 minutes for 5 hours’ worth of reading
The image above is from an article by Bain about how automation is rehumanising work by eliminating routine work. We see a lot of grey colour representing physical and information processing (like compiling data or maintaining records) in the chart. And there is a lot of dark red for problem solving too.
We often say that data creates value by being used to make better decisions (problem solving), automate (information processing), and create new revenue streams.
But it is not only about the economic value it creates. Working in a data-driven organisation allows people to focus on the interpersonal and creative tasks. And thus making the work environment more human.
I know that in the flood of unsympathetic chatbots, intrinsically inhuman metaverse, and scary-looking humanoid robots it’s hard to see the humanising aspects of automation (or technology or data). But it is there.
This week’s reading list looks at data engineering, data strategy, and data contracts.
- Let’s Move Fast And Get Rid Of Data Engineers: Given the nature of our work, we often talk to companies starting with data and analytics. Their idea is to engage a technology provider to establish a data lake and ingest individual data sources, and then hire few analysts to make use of the data. The article is quoting Bill Inmon saying: “In order to be a data warehouse an organization must integrate data. If you don’t do integration of data then you are not a data warehouse.” Wise words. Skipping data engineering can be a costly error. But just doing data engineering is also not enough. A common mistake I see is companies integrating data from bottom up — starting from data structure provided by operational applications and hoping to get close to a data model that would reflect the business and its processes — when the opposite direction works so much better. Starting by designing a data twin of your business and diligently replicating it in the data infrastructure (using data engineering) is my way to go. (Ben Rogojan @ SeattleDataGuy)
- Building a Data Strategy — The Essentials: Sharing this is a thirty-minute read says a lot. The article covers data strategy from many angles — what it is, its key elements, how to create one, or challenges of building one to name a few. There is also a section about how market research fits into the process (this is a company blog so I guess that’s what the “MR” in the company’s name stands for). But it’s basically about making sure the data strategy is aligned with the business strategy. Some use market research, some use design thinking, others simply link data initiatives to business ambitions. My experience is that there is often a big difference between what the business wants and what the individual employees want (which says a lot about how well designed and communicated the corporate strategies are). So when using market research, use it wisely. (FlexMR)
- An Engineer’s Guide to Data Contracts — Pt. 1: Data engineering problems caused by changes to upstream operational databases are omnipresent. With the increasing importance of data and analytics, they are being discussed more often (and louder). A possible solution lies in creating data contracts — API-based agreements between software engineers who own services and data consumers. “The term Data Contracts is being used to introduce a concept: There must be a formal agreement between data producers and consumers where one did not really exist before.” I like the concept. Building data infrastructure on top of data sources that don’t even know they are data sources sounds like a crazy idea. But unfortunately, it’s the reality. This article is about implementing data contracts. What requirements should be met and how to define, enforce, fulfil, and monitor the data contracts. (Data Products)
This week, our business — DataDiligence — is turning two years. And almost the same is true for this weekly reading list of mine. It’s time to celebrate!
Enjoy the weekend!
