It’s 2am and you’re reconciling data. Something’s off, and it’s hard to tell where. Regions are slightly off, product amounts are off. You slice the data another way and sales by salespersons are off. It’s not isolated to just one sales order or one type of order. Sound painfully familiar?
I wrote about how to cut data reconciliations by 90% before, but here’s a different spin on reconciliations: look at it from creation to display. There a series of data manipulations that take the data from raw form (creation) to report (display). Your data process or ETL may be different, but you get the general idea:
- Transaction is created
- Transaction is stored in tables
- Tables are moved into views or logical joins
- Data is summarized
- Data is filtered
- Date is put into report format.
What you need to do is jump from the end of the problem to the beginning of the problem. Here’s my suggestion on how to do this:
- Confirm the numbers don’t match at the display level (i.e. report, cube, or whatever the end customer is looking at)
- Jump down to the most detailed level possible (sales order number, tables, or raw files) and compare there.
If your data is accurate in item number 2, then it’s an issue of filters or criteria (usually is), but if it’s off in the source files, then you have a different type of reconciliation to do. This can save you hours of time trying to find the “needle in a haystack” by skipping past the display issues.
What other secrets can you share with the community?
Photo by: philandpam
