Empirical software engineering research plays a critical role in advancing the field, but the reliability of findings depends heavily on the rigor of the methodology. In this keynote, I will discuss the importance of addressing key aspects of validity—internal, construct, and statistical conclusion validity—when conducting empirical studies. Using two prevalent research areas, Mining Software Repositories (MSR) and Deep Learning (DL) algorithms, I will illustrate common methodological challenges. Through these examples, I will explore potential strategies and considerations for improving study design and analysis, aiming to enhance the reliability and scientific contribution of empirical software engineering studies, ultimately driving more robust insights and advancements in the field.