Cyber-Physical Systems (CPSs) are composed of computational control logic and physical processes, that intertwine with each other. CPSs are widely used in various domains of daily life, including those safety-critical systems and infrastructures, such as medical monitoring, autonomous vehicles, and water treatment systems. It is thus critical to effectively test them. However, it is not easy to obtain test cases which can fail the CPS. In this work, we propose a failure-inducing input generation approach FIGCPS for CPS, which requires no knowledge of the CPS under test or any history logs of the CPS which are usually hard to obtain. Our approach adopts deep reinforcement learning techniques, which interact with the CPS under test and effectively search for failure-inducing input guided by rewards. Our approach adaptively collects information from the CPS, which reduces the training time and is also able to explore different states. Moreover, our approach considers both continuous action space and large-dimension discrete action space, which are common for CPS systems. The evaluation results show that FIGCPS not only achieves a higher success rate than the state-of-the-art approach, but also finds two new attacks in a well-tested CPS.