Keeping up with data — Week 53 reading list
5 minutes for 5 hours’ worth of reading
The time of predictions, planning and resolutions is here. Data influencers are writing about the next big trends for 2021, practitioners decide what they plan to achieve in 2021 and how to go about it, and individuals are sharing their commitment to up-skill, learn and improve as data professionals.
The end of the year provides a natural opportunity to reflect on the industry, your business and yourself, and analyse what should be the next level and how to get there.
It is no surprise that these are the topics of this week’s reading list.
- 2021 Predictions: Data Science: A compilation of data science predictions from various individuals. These are my takeaways: companies are increasingly relying on data. This has been accelerated even more by this ‘year of Covid’. As a consequence, businesses will increase their spendings on data science. MLOps will empower data scientists to put more models in production. The role of all data professionals will shift from building their own tools and systems to pragmatically utilising available technologies to solve the problems of their businesses. Metadata is more valuable than the data itself and data contextualisation will be at the centre of metadata curation. There will also be a strong emphasis on the possibilities of feature engineering. (datanami)
- Back To The Future: Data Engineering Trends 2020 & Beyond: Data engineering is mostly about storing and processing data. The recent trends have imho been driven by the fact that data-powered solutions are moving closer to production, which brings new challenges to tackle. The article outlines the following trends and predictions. When it comes to infrastructure, the cloud data warehouse systems (like Snowflake) will stay dominant. In the architecture space, data lakes will continue to mature and managing real-time data flows will remain a challenge. When it comes to data management, we will see data quality, data lineage and data discovery solutions maturing into unified data management solutions. DBT as a data processing orchestrator will be adopted more and more (either as product, or as a concept). And we will also see adoption of the data mesh principles. (Data Engineering Weekly)
- 21 Predictions about the Software Development Trends in 2021: Software development is very relevant for data so it’s helpful to know what’s happening there. This article provides a lot of insight into the latest trends (and some predictions) so let me just pick few. Centralised data centers are now in cloud. And if not, they are to be migrated soon. Some situations, however, require data processing to happen at the devices and therefore, edge computing will be big in 2021. Containerisation is ruled by Kubernetes while Docker is loosing ground. Quantum computing is again predicted to gain momentum next year. AI (and AutoML) will become even more available. Modern data stores will combine SQL, NoSQL and NewSQL. (Md Kamaruzzaman @ Towards Data Science)
- Data Team Roadmap: Every data team should have a strategy, a road map, a plan. Otherwise the team will just reactively wait for tasks to land on their desks. The post is about creating a road map for a data engineering team but it has many points relevant to any data teams. Firstly, it is immensely helpful to put together a big picture of what you want to achieve and what you need in order to do that. Secondly, you see how things like data infrastructure, data products or data governance are interlinked and one cannot work with the others. And thirdly, never forget about the people element! Both in terms of the data team as well as the rest of the business. In my experience, this exercise should be linked to a data strategy, which contains what you want to achieve, what you need in order to do that and how are you going to do execute it. (Miles 2 code)
Once we take stock of the industry and our businesses it is fair to focus on ourselves too. And so come the resolutions. I’ve always believed that data science is a craft. One needs to keep working on real problems and solve for all the challenges that come along the way. Last year I also learned a lot about habit formation so the initiative that resonates with me is #66DaysOfData. What always worked best for me was to create a habit (or addiction?) of practicing and be guided by my own curiosity or passion. The first gives you energy, the second direction.
My plan for 2021 is to help as many clients as possible to maximise the value they get out of their data. And I’m sure that I’ll have to learn a ton along the way.
Wishing you and yours a happy, healthy, and successful new year!
