Importing a file seems like a convenient way to transfer a large amount of data all at once. However, you’ll want to know more than just whether the file can be opened. Are the entries unique? Do the columns mean what you expect? And what information do you use when sources contradict each other?
I would answer those questions before having the data processed. Even a file that can be imported technically may still require content-based checks.
Start by understanding the meaning of the fields
Two columns can have the same name even though they describe different things. For example, a date can indicate when something was created or when it was last modified. You can't just use those values interchangeably.
Therefore, document what each field represents and where it will be located in the new system. Also discuss which data fields are required and how to handle missing values. An empty field does not necessarily mean that the data was intentionally deleted.
With custom software, this coordination is part of preparing for the import. The process determines what data you need and how you use it.
Agree on how to recognize a listing
A name or description is not always a reliable way to compare two entries. The same entry may be written differently. Conversely, two different entries may have almost the same name.
Therefore, check which fixed attribute the system uses to identify information. Verify that the attribute is present and that it is indeed supposed to be unique.
If the same attribute appears multiple times, don't delete the extra lines right away. Compare the content first. The lines may be identical, but they might also contain information that you want to keep or still need to have checked.
Specify the rules that apply in the event of an overlap
When you merge multiple files, it must be clear which source takes precedence. The file name alone or the order in which you open the files does not always provide sufficient guidance.
Agree on how you will assess the timeliness of information and what you will do if the chosen source does not include a particular field. Are you allowed to fill in that information from another source? And how do you prevent an older value from replacing a deliberate correction?
Save those choices alongside the import file. That way, you can look back later to see why a particular value was included. A colleague won't have to make the same decision again without the original information.
Set aside cases where there is doubt
A value may have been copied exactly from the source and still raise questions. Perhaps the meaning isn't clear, or the value differs significantly from what you'd expect.
Therefore, keep track of what you've checked. Have you only verified that the value appears as is in the source file? Or have you also confirmed that the value is correct for the intended use?
Do not fill in missing information with an assumption that will later be treated as fact. Set aside any unclear or contradictory entries and note which questions remain unanswered. Agree on who can make a decision about them before you process them.
First, test it on a small sample
Start with a limited selection in a test environment or an import sample, if the system supports it. Be sure to include situations that require extra attention, such as a duplicate entry, a missing field, and an existing value that would change.
Check what the system does with that data. Does it go into the correct fields? Is existing information supplemented or replaced? And can you see which records were skipped?
Only then should you determine whether the entire file is ready for processing. Save the original and document how to revert any unwanted changes.
Be sure to check the results after the import
A notification that the import is complete does not necessarily mean that all the relevant information has been processed correctly. Therefore, compare the results with the prepared file. Check the numbers and also review the content of a few entries.
When it comes to automation, the same question applies: How can you tell if the information has been correctly processed? A clear overview of processed, skipped, and rejected rules helps you focus your further analysis.
Would you like to migrate data to an existing or new environment? Feel free to contact us. We’ll discuss what sources you have, what the system requires, and what controls are appropriate for the task.