Remote pair programming research indicates benefits for software engineers, including increased productivity, code quality, teamwork, knowledge management, and morale. The COVID-19 pandemic increased the prevalence of remote pair programming. Gender gaps persist in CS classes and workplaces, which may negatively impact the way pairs coordinate, communicate, and collaborate. To understand these effects, we conducted a large-scale survey to investigate differences between men and women as well as same- and mixed-gender pairs. The survey questions were adapted from established literature on gender differences in the fields of education, communication, management, human-robotic interaction, and human-computer interaction. Quantitative analysis of the survey data using ANOVA and pairwise t-tests indicated that women participants reported their men partners made gender-based assumptions about them and felt dominated and interrupted with men partners. Men participants felt their men partners were more rude and gave more negative feedback than women partners. Further, qualitative analysis of interviews gave insights into several challenges software engineers face in same and mixed-gender pairs when programming remotely. Our findings have implications for researchers, practitioners, and educators to promote gender inclusivity in collaborative environments.