1. 1Research questions



Download 129.28 Kb.
Date19.10.2016
Size129.28 Kb.
ANNEX A. Resources of the systematic mapping on context modelling

1.1Research questions


In this step we specify the research questions which will allow conducting the review methodology. To formulate them, we have used the PICO (Population, Intervention, Comparison and Outcome) criteria [Kitchenham and Charters 2007]. This framework consists on defining research questions by means of these criteria allowing dissecting useful keywords to structure the main string search in order to obtain more evidence from the study field. The general idea of PICO is suitable to organize any search strategy; however, the comparison criterion is out of the scope of this review since we are not interested on comparing context models, but rather, our interest is focused on the resources provided by each of the context models having a significant impact on the value delivered through services. Therefore, the main research question is finally formulated as shown in Table I. As it can be seen, the main research question is too generic, so that we refined it into specific sub-questions considering interests and motivations of the review.
Table I. Research questions

Main research question

In the field of services (P), do ontology-based context models (I) proposed so far provide an adequate and structured set of context knowledge pieces to be reused in prospective proposals that can define their context (O)?

Specific sub-questions

Research sub-question

Interest and motivation

RQ1.1. What is the chronological overview of the research done so far in ontology-based context models?

Identify the proposals in the field, find their interrelationships and distribute them along time to find any significant trend, considering also their provenance (academy or industry)

RQ1.2. What are the characteristics of the proposed ontology-based context models?

Make explicit the main characteristics of these context models in terms of size, structure and completeness

RQ1.3. Which classes of context information and entities are the most addressed in ontology-based context models?


Identify aspects related to the scope of these context models, such as: what are the contexts and entities attracting more attention from researchers, since it may help to understand their priorities and eventually some research gaps

RQ1.4. What are the most consolidated classes of context information and entities in ontology-based context models?

We aim at identifying the most recurrent definitions of classes of context information and entities, which in some sense could be considered as the starting point of any new future proposal


1.2The Protocol


The definition of the protocol comprises the selection of bibliographic sources where the electronic documents are systematically sought following a well-defined search string by means of the identified keywords, and finally, the selection criteria are established to retrieve the most significant works in the field.

1.2.1. Bibliographic sources. The search process conducted in this systematic mapping comprises automatic and manual searches. The automatic search is carried out by using bibliographic databases, and the manual search through collecting the works from specific journals and conferences of the interest field. Advantages and drawbacks of both approaches are analyzed through a case study in [Kitchenham et al. 2009]. Based on this analysis, we decided to integrate both strategies by accomplishing an automatic search in the selected databases and complementing the results with manual searches to the most relevant conferences and journals if some issue was missing. Finally, to select the bibliographic sources relevant to this review we follow the study and selection criteria provided in [Dieste et al. 2009] to select bibliographic databases. Therefore, the selected databases were Scopus, IEEE Xplore and ACM Digital Library.

In order to ensure the coverage of the review, we identified a list of journals and conferences relevant to this study that should be considered as potential sources of research works. We targeted venues mainly in the field of software services and context modeling such as pervasive and ubiquitous computing; adaptive systems, human-computer interaction, semantic web and information technologies (see Table II). These venues were selected from the top ranked list based on the JCR impact factor for journals and the CORE-A status1 for conferences. It is important to keep in mind that these sources are just for the purpose of checking completeness of the study, which will include other conferences and journals within computer science categories, due the broad spectrum of the selected databases (DB).


Table II. Journals and conferences classified by category

Categories

Journals

Conferences

Surveys

ACM Computing Surveys (CSUR); IEEE Communications Surveys and Tutorials.




Pervasive and ubiquitous computing & Adaptive systems

IEEE Pervasive Computing; Pervasive and Mobile Computing (PerCom); Personal and Ubiquitous Computing (PUC); Adaptive Behaviour; Journal of Ambient Intelligence and Smart Environments (JAISE); ACM Transactions on Autonomous and Adaptive Systems (TAAS).

IEEE International Conference on Pervasive Computing and Communications (PERCOM); International Conference on Pervasive Computing (PERVASIVE); Ubiquitous Computing (UbiComp); International Conference on Mobile and Ubiquitous Systems: Networks and Services (Mobiquitous).

Services

IEEE Transactions on Services Computing (TSC); International Journal of Web and Grid Services (IJWGS).

IEEE International Conference on Services Computing (SCC); IEEE International Conference on Web Services (ICWS); International Conference on Service Oriented Computing (ICSOC).

Human computer interaction

International Journal of Human-Computer Studies; ACM Transactions on Computer-Human Interaction.

British Computer Society Conference on Human-Computer Interaction (HCI); IFIP TC13 Conference on Human-Computer Interaction (Interact).

Modelling

Applied Ontology.

International Conference on Conceptual Modelling (ER); International Conference on Formal Ontology in Information Systems (FOIS).

Web and semantic web

ACM Transactions on the Web (TWEB); Journal of Web Semantics (JWS); World Wide Web-Internet and Web Information Systems (WWW).

International World Wide Web Conference (WWW); International Conference on Web Information Systems Engineering (WISE); International Semantic Web Conference (ISWC); Extended Semantic Web Conference (ESWC).

Information technologies

Information Sciences; Journal of Information Technology (JIT); Information Systems (IS); Computer; IEEE Software; Advanced Engineering Informatics; European Journal of Information Systems (EJIS); Data & Knowledge Engineering (DKE); Journal of Information Science (JIS); Journal of Systems and Software (JSS); IEEE Transactions on Software Engineering (TSE); Journal of the ACM; Communications of the ACM.

International Conference on Information Systems (ICIS); ACM International Conference on Information and Knowledge Management (CIKM); Americas Conference on Information Systems (AMCIS); European Conference on Information Systems (ECIS); International Conference on Advanced Information Systems Engineering (CAiSE); International Conference on Computational Science (ICCS); Enterprise Distributed Object Computing Conference (EDOC).

Conferences and journals shown above were reviewed to verify that all their editions from 2001 to 2014 were indexed and published in the selected DB. This evaluation allowed us to identify when to apply automatic or manual searches, i.e. whether the journal or conference was published with all its editions in some of the selected databases means that it is sufficient to apply an automatic search, otherwise, it is necessary to apply manual searches for each edition missed, leading to searches in a specific Digital Bibliographic Library (DBL) of a conference or journal.



The results obtained are depicted in Table III, we omitted conferences and journals that were indexed and published in all of their editions such as Information Science, IEEE Pervasive Computing, Adaptive behavior, and so on. Hence, it was only specified those conferences and journals involving manual searches such as TSC, IJWGS, WWW, etc., having editions not included in the databases. This issue was found in a higher proportion in conferences than journals: 17 conferences and 7 journals that were addressed with manual searches for each edition missing in the next stage of the systematic mapping.
Table III. Manual and automatic searches

Journal

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

IEEE Commun. Surv. Tutor.

NP

M

M

M

A

A

A

A

A

A

A

A

A

A

IEEE Trans. Serv. C.

NP

NP

NP

NP

NP

NP

NP

M

A

A

A

A

A

A

Int. J. Web Grid Serv.

NP

NP

NP

NP

M

M

A

A

A

A

A

A

A

A

World Wide Web

M

M

A

A

A

A

A

A

A

A

A

A

A

A

ACM Trans. Comput.-Hum. Interact.

M

M

A

A

A

A

A

A

A

A

A

A

A

A

Pers. Ubiquitous C.

M

M

M

M

A

A

A

A

A

A

A

A

A

A

Applied Ontology

NP

NP

NP

NP

M

M

M

M

M

M

A

A

A

M

Conference

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

ICIS

M

M

M

M

A

A

A

A

A

A

A

A

M

NY

PERVASIVE

NP

M

NP

M

A

A

A

A

A

A

A

A

NY

NY

UbiComp

M

M

M

M

M

A

A

A

A

A

A

A

A

M

AMCIS

NP

NP

M

M

A

A

A

A

A

A

A

A

M

M

HCI

NP

NP

NP

M

NP

NP

M

M

M

M

M

M

M

M

ECIS

M

M

M

M

A

A

A

A

A

A

A

M

M

M

ICWS

NP

NP

M

A

A

A

A

A

A

A

A

A

A

NY

Interact

NP

NP

M

NP

M

NP

M

NP

M

NP

M

NP

M

NY

CAiSE

M

M

M

M

M

M

M

M

M

M

M

M

M

M

ICCS

M

M

M

M

M

M

M

M

M

M

M

M

M

M

ER

M

M

M

M

M

M

M

M

M

M

A

M

M

M

FOIS

A

NP

NP

NP

NP

M

NP

M

NP

M

NP

M

NP

M

MobiQuitous

NP

NP

NP

A

A

A

A

M

A

M

M

M

M

NY

ICSOC

NP

NP

M

A

M

M

M

M

M

M

M

M

M

M

WISE

M

M

M

M

M

M

M

M

M

M

M

M

M

M

ISWC

M

M

M

M

M

M

M

M

M

M

A

M

M

M

ESWC

NP

NP

NP

M

M

M

M

M

M

M

M

M

M

M

Legend: A - Automatic, M - Manual, NP - Unpublished, NY - Not Yet Available


1.2.2. Keywords used. To structure the main search string of the review we acquired the keywords from the PICO criteria specified in Section 1.1. Specifically Population and Intervention criteria are used to extract them. Although in [Kitchenham and Charters 2007] it is recommended to consider keywords also from Comparison and Outcome criteria since it is the common procedure in the field of medicine, we have not used them. As stated in [Kitchenham et al. 2007] and identified in other SLRs [Riaz et al. 2009] and systematic mappings [Petersen et al. 2008], this is not always applicable. In our case, Comparison was discarded in the establishment of the research questions and Outcome was not considered because it is not based on a particular measurement in the research questions. Hence, from each term of the Population and Intervention criteria, we identified the keywords used to build the search string as is depicted in Table IV.

Table IV. Keywords and search string



Criteria

Keyword

Variants

Population

Service

“service”, “services”

Intervention

Context model

“context model”, “context models”, “contexts model”, “contexts models”; “context ontology”, “context ontologies”, “contexts ontology”, “contexts ontologies”; “context taxonomy”, “context taxonomies”, “contexts taxonomy”, “contexts taxonomies”; “context hierarchy”, “context hierarchies”, “contexts hierarchy”, “contexts hierarchies”

Search string

(“service” OR “services”) AND (“context model” OR “context models” OR “contexts model” OR “contexts models” OR “context ontology” OR “context ontologies” OR “contexts ontology” OR “contexts ontologies” OR “context taxonomy” OR “context taxonomies” OR “contexts taxonomy” OR “contexts taxonomies” OR “context hierarchy” OR “context hierarchies” OR “contexts hierarchy” OR “contexts hierarchies”)

Note that to build the search string, variants inside Population and Intervention criteria are interconnected through OR connectors (e.g. “service” OR “services”) and finally, these criteria are joined through an AND connector.



1.2.3. Selection criteria. Once the results were obtained we considered the following selection criteria to decide which works could be relevant to the review:

Filter by title. Step to quickly identify and remove noise from results. After this selection, documents whose scope was clearly unrelated to context models were removed.

Filter by abstract. Step used to delete works that although being related to context models, did not present an ontology-based context model as a contribution of the paper.

Filter by fast reading of full paper. Step to discard those papers that did not fulfil properly the following inclusion criteria: (1) defining explicitly the context model; (2) presenting a context model that can be applied from the perspective of the service-oriented computing, i.e., the model conceptualizes a process or some of the entities involved in the service provisioning and consumption (e.g. user or provider of a service, service composition, etc.).

Addition of further work (snowballing). Step that gathers further works during the SM process. To conduct this step, we employed backward snowballing that identifies relevant works from the reference list of the articles. We have included those referenced works that fulfil the previous inclusion criteria.

The results obtained from the searches are described as follows (see Figure 1): 787 papers automatically from Scopus, 293 from IEEE Xplore and 72 from ACM DL yielding a total of 1.152 papers from which 220 were removed as they were duplicated, resulting 932 papers found automatically. Then, 111 papers were added from manual searches in selected venues (as explained in Section 3.3.1), obtaining 1.043 papers from which 712 were deleted by title and abstract, resulting in 331 papers to filter by fast reading. We discarded 17 papers that were not available through our University resources and whose authors didn’t provide a copy under our request. Finally, the resulting papers after filtering by full paper were 145 and after adding 19 papers by snowballing, we obtained 164 papers to include in the SM. Afterwards, when analysing them in detail, we found a set of equivalent proposals, i.e., proposals from the same authors with a similar contribution; in this case we selected the most complete proposal to be evaluated and reviewed all of them to identify relevant information not considered in the chosen representative papers. Figure 1 summarizes the results obtained.


Fig. 1. Primary studies selection process.



References

Barbara Kitchenham and Stuart Charters. 2007. Guidelines for performing systematic literature reviews in software engineering. Technical Report EBSE-2007-01. Keele and Durham Universities, UK.

Barbara Kitchenham, Pearl Brereton, Mark Turner, Mahmood Niazi, Stephen Linkman, Rialette Pretorius, and David Budgen. 2009. The impact of limited search procedures for systematic literature reviews—A participant-observer case study. In 3rd International Symposium on Empirical Software Engineering and Measurement (ESEM’09). IEEE, 336-345.

Oscar Dieste, Anna Grimán, and Natalia Juristo. 2009. Developing search strategies for detecting relevant experiments. Empirical Software Engineering 14, 5 (2009), 513-539.

Barbara Kitchenham, Emilia Mendes, and Guilherme H. Travassos. 2007. Cross versus within-company cost estimation studies: A systematic review. IEEE Transactions on Software Engineering 33, 5 (2007), 316-329.

Mehwish Riaz, Emilia Mendes, and Ewan Tempero. 2009. A systematic review of software maintainability prediction and metrics. In Proceedings of the 3rd International Symposium on Empirical Software Engineering and Measurement. IEEE Computer Society, 367-377.



Kai Petersen, Robert Feldt, Shahid Mujtaba, and Michael Mattsson. 2008. Systematic mapping studies in software engineering. In Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering (EASE’08), 68-77.

1 There is no standard procedure to select the list of conferences. Although the CORE-A index, as any other, can raise controversy, we consider that it is a good indicator for our purposes. http://core.edu.au/index.php/conference-rankings.


Share with your friends:


The database is protected by copyright ©ininet.org 2019
send message

    Main page