Getting the data right: seamless data migration for the public sector
With the National Data Strategy setting out the Government’s desire to create a data-led public sector, as well as an appetite from local authorities to deliver data-driven services, many organizations now see that migrating from legacy systems to a new provider offers the ideal opportunity to improve data quality and use this investment to its fullest potential.
Data migration can be seen as a ‘necessary evil’ when implementing new tech, and may not be given the attention it deserves. The consequence of this is data decay – key data is excluded from transfer, or is clumsily aggregated in an attempt to retain information with little thought about its future use or accessibility.
Getting data migration right goes a long way to repressing fear and skepticism of change for stakeholders, which is vital in any digital transformation project. With careful planning and execution, data migration can be broken down into a series of manageable areas and councils can succeed in protecting valuable citizen data, with staff able to access it and use it to inform decisions and improve services.
Here are the most important elements for public sector organizations to consider when undertaking a data migration.
Data migrations require people with technical and non-technical knowledge. I’d recommend that teams include an expert on the area to which the system relates, who will take the lead on the initial data mapping exercise. This is a key individual who understands the current business process and legislative requirements for that data. They will also understand and consult with other stakeholders who rely on the data – maybe the local land charges department or an external Government department in need of solid statistics.
This expert should also understand the legal requirements for data retention and be empowered to make decisions about which data should and shouldn’t be migrated. These teams should also include an individual with technical knowledge of the source system. This person will have a good understanding of the source database and work closely with the above expert to convert business rules into technical requirements and can also read into the underlying data, helping to inform mapping decisions.
Identifying anomalies in the data up front lets the team correct them before the actual migration, cleansing records at source. They can also build rules into the migration to resolve issues during the extraction. Identifying data quality issues too late in the migration may result in unexpected issues, and ultimately a delay in the project’s timeline.
Data quality issues may originate from legacy data migrations with poor mappings or minimal data, invalid or duplicated records, accidental deletions, changes to the way staff used the system in the past or the poor administration of records. It’s important to identify any issues in the data before migrating it – side-stepping the potential for precious citizen data to be lost, damaged, or inaccessible.
