With the growing accessibility of generative AI tools, students are increasingly using them in coursework. However, their impact on developing core computational thinking skills remains unclear. This paper investigates whether access to ChatGPT during learning affects students’ short- and long-term transfer of two essential computational thinking skills, specifically, decomposition and abstraction. In a controlled experiment, we divide students into two groups; one with ChatGPT assistance during the learning phase and one using traditional resources. Participants complete learning tasks and testing tasks in a study session, followed by a retention test within a week. We measure task completion time and solution correctness. We also collect post-study questionnaire responses to assess confidence, perceived task difficulty, and experience with ChaGPT. We analyze screen recordings and ChatGPT logs to understand usage patterns. Our findings show ChatGPT provided fast but fleeting advantages, helping students complete tasks faster, perceive them as less challenging, and have better initial awareness of their performance. However, these gains did not carry over in the long term, suggesting that the tool acted more as a crutch than a lasting learning aid.