The recent rise in the performance and availability of large language models (LLMs) has fueled the adoption of generative artificial intelligence (AI) to support software engineering. Technologies such as Copilot and ChatGPT have become ubiquitous in software engineering, both in academic and professional settings. Nevertheless, the effects of such technologies on how engineers collaborate to build software are relatively unknown, raising questions regarding the impact they have on computer-supported collaborative work (CSCW). To explore these effects, we conducted a within-subjects empirical case study with 24 undergraduate software engineering students. Students were divided into seven groups, completing four brainstorming tasks related to a software engineering project for a course on frontend development. For each group, two of the tasks were supported by LLM-powered bots, while the other two tasks did not include bots. Our findings show that when the brainstorming process was supported by bots, students proposed significantly fewer ideas and reported significantly less sense of authorship and sense of responsibility with respect to the ideas selected. These results motivate the need for frameworks to guide how software engineers collaborate with AI-powered technologies.