The number of Android apps keeps increasing in recent years. Despite the fact that there exist apps for various kinds of purposes, apps that share similar functionalities with existing ones are still emerging on the market. To reduce the effort in testing such apps, previous research has proposed approaches for migrating test scripts across similar apps. However, such test reuse techniques require the existence of a suite of test scripts for migrating, which hinders their practical use in commercial app development. Unlike script-based GUI testing, record-and-replay techniques are more convenient for human testers who are unfamiliar with programming and require no extra test maintenance effort. In this paper, we propose a new testing technique, RIDA, that records interaction sequences on one app (source app) and replays the recorded sequences on another app (target app) with similar functionalities. Such Cross-app record-and-replay is challenging. First, there are no clear mappings between the recorded sequences on source apps and the sequences required to be performed on the target apps. Second, reliable indicators of widgets’ functionalities are not always available from the recorded sequences, which limits the effectiveness of event matching between source and target apps. To address the challenges, we design an on-the-fly searching algorithm for finding target widgets during cross-app replay and leverage multiple semantic descriptors together with image captioning techniques to infer the functionalities of widgets. We have implemented RIDA and evaluated it using both controlled and in-the-wild experiments. The results show that RIDA can effectively perform cross-app record-and-replay for both scenarios and outperform baseline methods in terms of the number of completely- and partially- replayed events.