\textit{Large Language Models} (LLMs) can translate natural language requirements into code, yet empirical analyses of representative models reveal that \emph{semantic errors}—programs that compile but behave incorrectly—constitute the majority of observed faults (e.g., $>$60% on DeepSeek-Coder-6.7B and QwenCoder-7B). Post-hoc repair pipelines detect such faults only \emph{after} execution, incurring latency, relying on incomplete test suites, and often mis-localizing the defect. Since semantic drift originates in the autoregressive decoding process, \emph{intervening while the code is being generated} is a direct way to stop error propagation. Constrained-decoding approaches such as ROCODE attempt this, but still wait until the entire program runs to obtain feedback and use entropy heuristics that do not truly capture semantics. A more effective solution must inject \emph{semantic} signals—early and precisely—into the decoding process.We present \textbf{SemGuard}, a semantic-evaluator-driven framework that performs real-time, line-level semantic supervision. To train the evaluator, we build \textit{SemDiff}, the first dataset with fine-grained annotations that mark the exact line where a correct and an incorrect implementation diverge. The evaluator, once embedded in the LLM’s decoder, flags deviations on partial code, rolls back to the faulty line, and guides regeneration—without executing the program or requiring test cases. Across four benchmarks, SemGuard consistently outperforms state-of-the-art baselines. It lowers the semantic error rate by \textbf{19.86%} on \textit{SemDiff} relative to ROCODE, and lifts Pass@1 by \textbf{48.92%} on the real-world \textit{LiveCodeBench} with CodeLlama-7B. Similar gains hold for StarCoder2-7B on \textit{MBPP} and for DeepSeekCoder-6.7B on the Java benchmark \textit{SemDiff-Java}, demonstrating model- and language-agnostic effectiveness.