The performance of machine learning (ML) models heavily depends on the quality of the data they are trained on. While prior work often treats data quality as uniform across a dataset, we investigate whether it varies across different population subgroups within a dataset and examine its implications, a phenomenon we refer to as Data Quality Disparity (DQD). Our analysis reveals that many real-world datasets inherently exhibit DQD, with underrepresented or marginalized groups frequently associated with lower-quality data compared to more privileged subgroups. Consequently, training ML models on raw data or applying standard preprocessing techniques without accounting for DQD can lead to uneven prediction performance across demographics. We introduce metrics to quantify DQD and assess its potential for discrimination, using both structured (tabular) and unstructured (image) datasets. Our work explores the connections between data quality and algorithmic fairness and underscores the need for developing fair data processing pipelines and learning algorithms that are DQD-aware.