How to approach your data management
Managing your data can seem like a daunting task. Especially when it involves cleaning up a data catastrophe that’s been years in the making. But there’s simply no getting away from it. It’s well known that the consequences of poor data management can end up having a significant impact on your business. This translates into poor customer communications, inefficient and time-consuming processes, reduced productivity and ultimately a loss of revenue opportunities.
But you don’t have to fret – staying on top of your data needn’t be that hard! Our tips and expert advice will help keep your data in order. Ultimately, saving you time and resources, enhancing your customer relationships and adding to your bottom line.
Being able to trust your data is key to accurate reporting and confident decision making. That’s why implementing data quality rules (DQR) is so important. Make sure you’re building DQRs that sit across your whole data set rather than fixing individual problems. This way you can prevent the same mistakes from repeating themselves.
When you think about all the data that your company processes every day, it would be an impossible task to manually cleanse it. But once you’ve established your data quality rules, there is software that will help you to implement them.
There are always exceptions to the rule which will require manual input, and this is where data stewardship comes in. Data stewardship solutions draw your attention to and allow you to correct any exceptions to the data quality rules you’ve implemented. Use data stewardship to refine, de-dupe, cleanse and aggregate your data.
The best data stewardship solutions enable collaboration using a team-based workflow. This means people in different teams across your company have the opportunity to define priorities and track progress on the data projects they are involved with.
Ensuring trustworthy data is only one part of data management. When it comes to customer data, you also need to make sure that the right people in your organisation have the correct access to it and that they understand the rules of how it should be processed and stored.
Establish a data governance strategy that comprises standardised rules on who in your business has access to certain data sets, how your data is processed, and what technologies are required to manage and protect your data assets. One example of data governance is establishing how long you need to keep customer data – ensuring that it’s handled in a GDPR compliant way.
The goal of your data governance strategy should cover; minimising risk, establishing internal rules, implementing compliance requirements, improving internal and external communications, increasing your data’s value, facilitating administration processes, reducing costs and managing risk and compliance.
Sounds like a lot, but when you think about the hefty GDPR fines you could incur, (We Buy Any Car, Saga and Sports Direct being recent examples) it is well worth putting the time in.
When you have data passing through multiple teams and systems, it can easily become siloed. Issues caused by people working in silos include departments not implementing the established governance frameworks, not sharing useful data and therefore missing out on important insights, and multiple departments implementing their own integrations separately. This wastes time and resources and creates inconsistency when it comes to integration quality.
The best way of bringing your data together to give you a complete view of your business is through data integration. Data integration involves bringing together data from different sources to create a complete picture of what you are trying to analyse. That could be a complete customer view, or a complete representation of operations efficiency, or accounting records, and so on. It allows teams to draw accurate insights and use this to make informed, evidence-based decisions.
For example, your CRM holds all of your customer comms data. It could be incredibly useful to integrate this with your accounts data, allowing you to see your customer’s purchase history, spot trends and better understand the relationship between the communications you’re sending out and the purchases your customers are making.
One way to effectively integrate your data, is by using the systems APIs to pass the data to and from each system. However, this will require a developer, an expense for companies that don’t have access to one in house. Another option is to use an integration tool (such as Talend) that will do all of the hard work for you, meaning less (or sometimes even no) developer work is required.
Often, the topic of data management comes up because organisations are considering moving data from one place to another. If you are upgrading an existing data platform, or switching to a new platform altogether, you will need to carry out a data migration.
All data migrations will involve an Extract, Transform and Load (ETL) process, or at least the transform and load steps. This means that extracted data will have to go through a series of functions in preparation, after which it can be loaded into a target location. Getting the ETL process right can make or break a data migration project.
The best way to ensure a successful outcome is by using the right data tool. A poor data migration where the process has not been developed tested and reconciled could mean preferences are not being migrated correctly, resulting in inaccuracies, lost data and duplicates. You can even risk breaching GDPR guidelines.
An effective way of reporting on whether you have migrated your data correctly is to produce Reconciliation reports. They will show if 100% of the data expected to be migrated has ended up in the target system.
Creating Your Foundation for Success
Gartner estimated that 60% of big data initiatives would fail in 2017. After just a year, Gartner analyst Nick Heudecker declared the actual figure to be “closer to 85%”. Heudecker went on to say that the issue wasn’t with the technology, it was with “you”.
A key reason for these shocking statistics is that many companies have a poor understanding of source data stores and underspecified integrations. Put the odds in your favour by conducting a data landscape analysis before you start your data project. This helps you to understand what state your data is in and what needs to be done to fix it. Look at it this way, if you were a doctor, you wouldn’t diagnose a patient without knowing their symptoms!
A landscape analysis, driven by your business goals, will give you the foundation to begin your data management journey with confidence.
We’ve mentioned Talend already in this article, and that’s because we think it’s a great tool for data management. Talend has a range of functionalities that allow you to rest easy knowing that your data is processed correctly. It also has a wide variety of connectors, making it platform agnostic.
Talend helps you cleanse incoming data, and makes recommendations for addressing any recurring quality issues. It also facilitates data stewardship, helps you to migrate data without the risk of data loss, and more. To find out if Talend is the right tool for you, get in touch with one of our experts or take a look at our Talend page for more info.
So, to recap – managing your data really doesn’t need to be such a terrible task:
- make sure you have your DQRs in place
- implement your data governance strategy
- use data integration to create a single source of truth
- make sure you take your ETL process seriously if you are migrating
- and don’t forget about that all-important landscape analysis before you change a thing!
By following these tips, you will be able to harness the benefits of being a data-driven organisation. Measure business performance, highlight business issues, improve productivity, better understand your customer base, and the list goes on. Well-managed data can truly become the key to improving your business.
Applying best practice is critical. Our recommended approach to data migration is a simplification of the Johnny Morris model, which is split into four different stages; discover, design, test and deliver. In this blog, we’ll take a look at how to get the first two stages right.
If you’re reading this blog with the view to carry out the ‘test’ and deliver’ stages of your data migration, you should have already completed the ‘discover’ and ‘design’ stages detailed in part one of this blog. Part two will focus on the thorough testing procedures that need to be carried out right up to your ‘go live’ date. You’re nearly there!
In the past 6 months, we’ve seen an explosion in the number of people asking us to conduct Landscape Analysis of data assets and data processes. If you’ve heard people mention Landscape Analysis but have never really been sure what…