Decisions anchored in data can help organizations compete, scale and avoid risk, but only if teams verify the integrity of the data feeding analytics or AI systems before models are trained or dashboards are built. If datasets contain inconsistencies, gaps or hidden biases, the result will likely be a “garbage in, garbage out” scenario.
Robust quality checks, from source validation and schema consistency to labeling, governance and drift monitoring, aren’t busywork; they’re essential risk management strategies that should take place before a dataset reaches production. Below, members of Forbes Technology Council share the key practices they rely on to validate datasets, strengthen trust and ensure reliable results as data volume and complexity grow.
Validate Dataset Granularity
Before analysis, confirm the dataset’s granularity matches the decision you need to make. Clean data can still mislead if it’s aggregated at the wrong level—for example, monthly versus daily, or product family versus SKU. Ensuring the right level of granularity prevents insights that seem correct but aren’t; a forecast may look off at the product family level even though only a few SKUs are driving the deviation. – Gaurav Sharma, Applied Materials
Verify Statistical And Contextual Fit
Before production, verify that the dataset reflects reality both contextually and statistically. Ensure labels, types and distributions are correct; duplicates are removed; and anomalies are addressed. “Garbage in, garbage out” remains the core principle. – Lakshmi Lasya Jonna, Comerica Bank
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
Automate Data Tests Early
Adopt a shift-left approach by implementing automated data testing. Validate data against defined business rules. Check for data quality dimensions. Detect data drift by comparing with previous datasets, if available. Most importantly, ensure automated testing occurs before data ingestion and transformation stages, not just at the final consumption point. – Sandesh Gawande, iceDQ
Secure Provenance Through Data Governance
We need to prove the end-to-end provenance and integrity of a dataset. Through strong access controls, encryption and audit trails, and compliance with existing regulations, we must ensure the data comes from an authorized source and hasn’t been tampered with. This strategy ensures data contributes to your analytical accuracy, customer trust and institutional resilience. – Dr. Anil Lamba, JP Morgan Chase
Enforce Accurate And Consistent Tagging
One essential consideration is ensuring proper tagging. Accurate tagging provides context, meaning and structure to the data, helping models interpret it correctly and produce reliable results. Consistent tagging also ensures that only relevant, validated data is used in analysis, minimizing errors and reducing the risk of data leakage that can compromise data integrity. – Justin Mescher, ePlus
Reassess Data Collection Assumptions
Before using a dataset in production, question the assumptions that shaped it: why it was collected, what behavior it was meant to capture, and what the team expected it to show at the time. Clean, well-structured data still leads to bad decisions if the world has changed since the moment it was gathered. When assumptions drift, insights drift with them. – Philip Moniaga, Billow AI
Align Semantic Definitions Across Sources
The essential check is semantic field alignment. Before using any dataset, one must rigorously verify that the meaning and definition of all core features, like active user or customer churn, are consistent across all source systems. A failure to align these semantic definitions leads to silent feature collapse, generating misleading comparisons and flawed AI models. – Mohan Mannava, Texas Health
Assess The Dataset’s Representativeness
Ensure the dataset accurately reflects the population or phenomena it aims to analyze. A misleading sample can skew results, leading to incorrect conclusions and poor decision-making. Evaluate sampling methods and potential biases in data collection to establish a solid foundation for reliable analysis. – Roman Vinogradov, Improvado
Reconcile Anomalies With Source Systems
Identify anomalies and implement methods to reconcile them with the source. It is vital to detect anomalies and implement solutions in real time. Rather than waiting until the last minute, set up a process to check quality at each step against the source. Focus on key metrics—for example, in financial data, track the total, the number of records, and one or two other key attributes based on the use case. – Hemanga Nath, Amazon
Confirm Structural Consistency
One essential check is for consistency between the data’s structure and its expected use: validating formats, ranges, relationships and completeness against the intended analytical model. If time fields are misaligned, values mislabelled or units inconsistent, even the most sophisticated analysis will mislead. Confirm the data behaves as expected before trusting insights. – Maman Ibrahim, EugeneZonda Cyber Consulting Services
Use Metadata And Documentation
It’s worth paying attention to the metadata and documentation. Good metadata explains what each field means, the units and formats used, how the data was collected, and what assumptions apply. Without proper documentation, even well-structured datasets can be misused or impossible to maintain over time. Clear metadata supports data governance, ensuring that insights from data remain accurate and reliable. – Son Nguyen, Neurond AI
Ensure Data Freshness
Making sure data is “fresh” is essential for accurate analysis and decision-making. Real-time data infrastructure helps to ensure that data is reliable—unlike outdated data, which can lead to mistakes and wasted investment. – Guillaume Aymé, Lenses.io
Map Fields To Product Standards
Translation of data names in order to validate before ingesting is key. Some data fields have names that don’t align with product-required formats. Translating into consumable and usable fields is complex, yet this validation and marrying of data descriptions to product formats, once completed, builds libraries of definitions and mapping opportunities that are reusable. – Jane Mason, Clarifire
Monitor For Data Drift
Checking for data drift is frequently overlooked. For example, an abrupt increase in a particular transaction type or changes in customer demographics could be a sign of upstream modifications to data collection, business procedures or user behavior. Drift scanning compares the statistical characteristics of a dataset to baseline or prior periods, improving long-term reliability. – Kavitha Thiyagarajan, Ascendion
Sanity-Check Distributions For Business Fit
Before any dataset goes into production, sanity-check the numbers. Run basic distribution checks: mean, median, minimum and maximum. Then ask, “Do these values make business sense?” Outliers and missing data will quietly corrupt every downstream model. Fix the logic before you touch the analytics. – Vivek Thomas, AISensum
Validate Ethical Sourcing And Compliance
Before analysis, ensure the dataset is ethically sourced and compliant with relevant regulations. Verify that personal data has been anonymized or collected with proper consent, that collection methods align with GDPR or CCPA, and that provenance and governance documentation is clear. Also, assess representativeness to avoid sampling bias. – Mammon Baloch, Starlight Retail Inc.
Stress-Test Dataset Stability
Check the dataset’s behavior under stress. Before trusting any field, push the data through drift checks, anomaly scans and distribution shifts to see if it “breaks” under real-world conditions. Clean data can still be unstable. If the statistical shape warps with small perturbations, the insights will too. Stability under stress is the real integrity test. – Akhilesh Sharma, A3Logics Inc.
Ensure Reproducibility And Anonymization
Your dataset should be reproducible. You should be able to re-create it from the source. Also, it’s essential to ensure you’re legally allowed to make the data public in the first place. Check your sources: Ask if it is balanced per category or sample type. Lastly, anonymize the data. – Victor Paraschiv, broadn
Confirm The Data Supports Actionable Execution
Beyond accuracy checks, ensure your dataset enables actionable execution, not just analysis. The real test of data quality is whether it drives successful outcomes when optimization and execution operate as one closed loop. Teams often validate data for analysis but miss execution feasibility. Quality data must support both planning insights and operational actions that deliver measurable results. – Richard Lebovitz, LeanDNA
Read the full article here