The prevalent fine-tuning paradigm for large language models (LLMs) has demonstrated strong performance on various code generation tasks. However, these models still fall short when confronted with algorithmic programming problems, where precise algorithmic reasoning is required. Humans typically adopt diverse algorithmic techniques to tackle complex programming problems, enabling general analysis and accurate implementation. Building on this observation, we propose a method that learns compact, LLM-friendly representation of algorithmic knowledge, termed \underline{A}lgorithmic \underline{I}nversion (\textbf{AI}), which aims to aid LLMs in understanding programming problems. Specifically, we apply a lightweight fine-tuning process on code-oriented models to automatically learn algorithm embeddings. When concatenated with the inputs, the algorithm embeddings act as instructive signals, guiding LLMs in generating correct code solutions by providing contextual algorithmic hints. We apply our approach to models of three different parameter sizes and evaluate them on three algorithmic programming benchmarks. Our extensive experiments show that applying \textbf{AI} to small (1.5B parameters) models results in relative improvements of up to $180%$ on Pass@1, while large models (15B parameters) achieve improvements of up to $77.8%$, compared to Prompt-Tuning. Additionally, our method outperforms traditional full fine-tuning approaches by a significant margin across all tested benchmarks. Furthermore, our analysis of the generated code reveals that \textbf{AI} effectively enhances the model’s problem-solving process by providing clear algorithmic guidance.