Artificial Intelligence is a rising topic in the field of emotion recognition, e.g., from facial expressions. However, existing methods often require to be performed in staged set ups and are obtrusive by gathering additional data. Especially, collecting video data includes a high data protection risk. Our approach is to provide an unobtrusive emotion recognition tool based on Keystroke, Mouse and Touchscreen data. Recently, we published a data set for emotion recognition from keystroke, mouse, and touchscreen dynamics. In this paper, we present the challenges we faced during the creation of the data set. This covers collecting User Interface data as well as emotional ground truth data. For each of seven mentioned challenges, we provide our solutions as well as guidelines for other researchers to prevent them. The challenges include possible issues with recorded data as well as issues of automated facial coding engines. We provide a possible approach for manual facial coding and describe aspects attention should be paid to. Furthermore, we indicate issues when using different software tools to collect the data. The paper aims to help other researchers by providing insights into and a guideline for the process of the data set creation. We make these insights available for other researchers who want to create similar data sets or who want to expand ours.