Oscar Pastor, Ignacio Panach

Title: ARTICULATING CONCEPTUAL MODELING RESEARCH CONTRIBUTIONS: AN EXPERIMENTAL PERSPECTIVE

Co-Chairs: OSCAR PASTOR / IGNACIO PANACH

Abstract:

Conceptual modelling aims to abstractly represent information systems from a highly abstract perspective, closer to the problem space rather than to the solution space. It is about conceptualization, a hard task, with a huge amount of articles in the literature, and with a rich diversity of people working in the field, which makes how to articulate conceptual modelling research become a relevant problem. In order to mitigate this problem, a framework to classify the conceptual modelling research contribution of a work has been proposed in previous works. In this session, an experimental validation of that framework will be done in group and with a high interaction. The experimental task will allow to evaluate the level of agreement among subjects in the classification of the paper characteristics; analysing the correctness when classifying the paper; measuring the effort spent in using the framework, and evaluating the satisfaction of the framework for better understating conceptual modelling articles.

Session Goals:

Experimental evaluation of a framework to characterize the contribution of a conceptual modeling-based research paper.

Development of the Session: (How will the session be conducted? How much interaction?)

• Step 1: Demographic questionnaire. Subjects fill in a demographic questionnaire to identify the subjects’ profile. Questions of this questionnaire can be seen in Training Web site. This task takes 5 minutes.

• Step 2: Framework introduction. In the session start, one experimenter describes the framework and how to work with it. Each section of the framework is described in detail. Slides can be downloaded from Training Web site. This task takes 10 minutes.

• Step 3: Training with the framework. Subjects must use the framework to analyse a paper selected as example of application. All subjects use the same paper for training. The protocol and the instruments used in this training exercise are the same as the ones used in the experimental tasks. Subjects must read the paper and fill in all questionnaires. These data are not analysed since they are out of the experiment. The paper to read as training and the questionnaires to fill in the process can be seen in Training Web site. This task takes 25 minutes.

• Step 4: Applying the framework to the selected papers (three). There are three different papers distributed in a balanced way among three groups of subjects. Subjects must first read the paper, and apply the framework, reporting this analysis through a questionnaire. Papers to review and questionnaires to fill in can be viewed in:

⮚ Paper A

⮚ Paper B

⮚ Paper C

This task will take 45 minutes.

• Step 5: Filling satisfaction questionnaire. At the end of using the framework, subjects must fill in the satisfaction questionnaire to get their opinion on the use of the framework. This questionnaire is filled in on the web page of the last paper read. This task will take 5 minutes.

What means for interaction will be used or required?

Group participation following the steps commented before.

Background and recommended reading:

No particular background and recommending reading is strictly required, although one paper presenting the framework to be used in the experiment will be provided in advance, to let participants start with some basic, initial knowledge about it.

Expected Outcomes and Plan for Continuing the Work beyond ISERN:

The experiment will allow to achieve a practical, shared and concrete understanding around how to accomplish a conceptual characterization of research contributions for a paper of any Software Engineering or Information Systems Engineering selected domain.