5678910118 of 32
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Evaluating Success Factors of Health Information Systems
Blekinge Institute of Technology, Faculty of Engineering, Department of Health.ORCID iD: 0000-0002-0316-548x
2019 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Health information systems are our technological response to the growing demand for health care. However, their success in their mission can be challenging due to the complexity of evaluating technological interventions in health care. In the series of studies compiled in this dissertation, we looked at the evaluation of these systems. We focused on the evaluation of factors that lead to success, where success is indicated by user satisfaction and can be induced by both intervention-specific and individual-specific factors.

Study 1 developed a method, called UVON, to elicit and organise the user-demanded qualities in the outcomes of the health information system intervention. Through the application of the UVON method in the FI-STAR project, an EU project which developed and deployed seven e-health applications in seven member countries, ten categories of quality and their subcategories were identified. These qualities formed two questionnaires, specific to the patient and health professional users. Through the questionnaires, the patients and health-professionals users evaluated and graded both the occurrence of those demanded qualities in the project outcomes and their general satisfaction.

Study 2 analysed the survey results to find out which of those ten qualities have the highest impact on satisfaction or can predict it better. Two partial least squares structural equation modelling (PLS-SEM) models were constructed, for the patient and health professionals, based on the Unified eValuation using ONtology (UVON) and survey outputs. The models showed that effectiveness is an important quality in creating satisfaction for both user groups. Besides, affordability for the health professionals and efficiency plus safety for the patients were the most influential. A satisfaction index is also introduced for simple and fast inferring of the changes in the outcome qualities.

Study 5 recruited outputs and learnings from studies 1 and 2 to design a system that partially automates the process of evaluating success factors in health information systems, making it continuous and real-time, and replacing hard-to-run surveys with automatically captured indicators and analytics.

Study 3 focused on individual-specific factors in using health information systems, particularly the technophilia personality trait. A short six-items instrument, called TechPH, was designed to measure technophilia in users, tuned for older users. The study recruited empirical data from the Swedish National Study on Aging and Care (SNAC) project. Two factors, labelled techAnxiety and techEnthusiams, are identified by the factor analysis method. A TechPH score was introduced as a scalar measurement of technophilia.

Study 4 elicited and discussed the ethical challenges of evaluating and researching health information systems. Both a scoping review and a novel systematic postulation approach were recruited to identify twenty ethical challenges. The identified ethical challenges were discussed and mapped into a three-dimensional space of evaluation stages, demanded qualities, and major involving entities (stakeholder and artefacts), which fosters further postulation of ethical challenges.

Place, publisher, year, edition, pages
Karlskrona: Blekinge Tekniska Högskola, 2019. , p. 340
Series
Blekinge Institute of Technology Doctoral Dissertation Series, ISSN 1653-2090 ; 14
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
URN: urn:nbn:se:bth-18799ISBN: 978-91-7295-387-1 (print)OAI: oai:DiVA.org:bth-18799DiVA, id: diva2:1366513
Supervisors
Available from: 2019-10-29 Created: 2019-10-29 Last updated: 2019-11-08Bibliographically approved
List of papers
1. Evaluating Health Information Systems Using Ontologies
Open this publication in new window or tab >>Evaluating Health Information Systems Using Ontologies
Show others...
2016 (English)In: JMIR Medical Informatics, ISSN 2291-9694, Vol. 4, no 2, article id e20Article in journal (Refereed) Published
Abstract [en]

Background: There are several frameworks that attempt to address the challenges of evaluation of health information systems by offering models, methods, and guidelines about what to evaluate, how to evaluate, and how to report the evaluation results. Model-based evaluation frameworks usually suggest universally applicable evaluation aspects but do not consider case-specific aspects. On the other hand, evaluation frameworks that are case specific, by eliciting user requirements, limit their output to the evaluation aspects suggested by the users in the early phases of system development. In addition, these case-specific approaches extract different sets of evaluation aspects from each case, making it challenging to collectively compare, unify, or aggregate the evaluation of a set of heterogeneous health information systems.

Objectives: The aim of this paper is to find a method capable of suggesting evaluation aspects for a set of one or more health information systems—whether similar or heterogeneous—by organizing, unifying, and aggregating the quality attributes extracted from those systems and from an external evaluation framework.

Methods: On the basis of the available literature in semantic networks and ontologies, a method (called Unified eValuation using Ontology; UVON) was developed that can organize, unify, and aggregate the quality attributes of several health information systems into a tree-style ontology structure. The method was extended to integrate its generated ontology with the evaluation aspects suggested by model-based evaluation frameworks. An approach was developed to extract evaluation aspects from the ontology that also considers evaluation case practicalities such as the maximum number of evaluation aspects to be measured or their required degree of specificity. The method was applied and tested in Future Internet Social and Technological Alignment Research (FI-STAR), a project of 7 cloud-based eHealth applications that were developed and deployed across European Union countries.

Results: The relevance of the evaluation aspects created by the UVON method for the FI-STAR project was validated by the corresponding stakeholders of each case. These evaluation aspects were extracted from a UVON-generated ontology structure that reflects both the internally declared required quality attributes in the 7 eHealth applications of the FI-STAR project and the evaluation aspects recommended by the Model for ASsessment of Telemedicine applications (MAST) evaluation framework. The extracted evaluation aspects were used to create questionnaires (for the corresponding patients and health professionals) to evaluate each individual case and the whole of the FI-STAR project.

Conclusions: The UVON method can provide a relevant set of evaluation aspects for a heterogeneous set of health information systems by organizing, unifying, and aggregating the quality attributes through ontological structures. Those quality attributes can be either suggested by evaluation models or elicited from the stakeholders of those systems in the form of system requirements. The method continues to be systematic, context sensitive, and relevant across a heterogeneous set of health information systems.

Place, publisher, year, edition, pages
JMIR Publications, 2016
Keywords
health information systems; ontologies; evaluation; technology assessment; biomedical
National Category
Information Systems
Identifiers
urn:nbn:se:bth-12114 (URN)10.2196/medinform.5185 (DOI)000387986800010 ()
Funder
EU, FP7, Seventh Framework Programme, 604691
Available from: 2016-06-16 Created: 2016-06-16 Last updated: 2019-10-29Bibliographically approved
2. Most Influential Qualities in Creating Satisfaction Among the Users of Health Information Systems: A Study in Seven EU Countries
Open this publication in new window or tab >>Most Influential Qualities in Creating Satisfaction Among the Users of Health Information Systems: A Study in Seven EU Countries
Show others...
2018 (English)In: JMIR Medical Informatics, Vol. 6, no 4, p. 3-21Article in journal (Refereed) Published
Abstract [en]

Background:

Several models suggest how the qualities of a product or service influence user satisfaction. Models, such as the Customer Satisfaction Index (CSI), Technology Acceptance Model (TAM), and Delone and McLean Information Systems Success (D&M IS), demonstrate those relations and have been used in the context of health information systems.

Objective:

We want to investigate which qualities foster greater satisfaction among patient and professional users. In addition, we are interested in knowing to what extent improvement in those qualities can explain user satisfaction and if this makes user satisfaction a proxy indicator of those qualities.

Methods:

The Unified eValuation using ONtology (UVON) method was utilised to construct an ontology of the required qualities for seven e-health applications being developed in the FI-STAR project, a European Union (EU) project in e-health. The e-health applications were deployed across seven EU countries. The ontology included and unified the required qualities of those systems together with the aspects suggested by the Model for ASsessment of Telemedicine applications (MAST) evaluation framework. Two similar questionnaires, for 87 patient users and 31 health professional users, were elicited from the ontology. In the questionnaires, user was asked if the system has improved the specified qualities and if the user was satisfied with the system. The results were analysed using Kendall correlation coefficients matrices, incorporating the quality and satisfaction aspects. For the next step, two Partial Least Squares Structural Equation Modelling (PLS-SEM) path models were developed using the quality and satisfaction measure variables and the latent construct variables that were suggested by the UVON method.

Results:

Most of the quality aspects grouped by the UVON method are highly correlated. Strong correlations in each group suggest that the grouped qualities can be measures which reflect a latent quality construct. The PLS-SEM path analysis for the patients reveals that the effectiveness, safety, and efficiency of treatment provided by the system are the most influential qualities in achieving and predicting user satisfaction. For the professional users, effectiveness and affordability are the most influential. The parameters of the PLS-SEM that are calculated allow for the measurement of a user satisfaction index similar to CSI for similar health information systems.

Conclusions:

For both patients and professionals, the effectiveness of systems highly contributes to their satisfaction. Patients care about improvements in safety and efficiency, while professionals care about improvements in the affordability of treatments with health information systems. User satisfaction is reflected more in the users' evaluation of system output and fulfilment of expectations, but slightly less in how far the system is from ideal. Investigating satisfaction scores can be a simple, fast way to infer if the system has improved the abovementioned qualities in treatment and care.

Place, publisher, year, edition, pages
JMIR Publications, 2018
Keywords
Health Information Systems, Telemedicine, Evaluation Studies as Topic, Consumer Behavior, Treatment Outcome, Safety, Efficiency, Health Care Costs, Ontology Engineering, Equation Models
National Category
Other Health Sciences
Identifiers
urn:nbn:se:bth-16998 (URN)10.2196/11252 (DOI)000454162600001 ()
Projects
MD3S
Note

Open access

Available from: 2018-09-13 Created: 2018-09-13 Last updated: 2019-10-29Bibliographically approved
3. A Novel Instrument for Measuring Older People's Attitudes Toward Technology (TechPH): Development and Validation
Open this publication in new window or tab >>A Novel Instrument for Measuring Older People's Attitudes Toward Technology (TechPH): Development and Validation
2019 (English)In: Journal of Medical Internet Research, ISSN 1438-8871, E-ISSN 1438-8871, Vol. 21, no 5, article id e13951Article in journal (Refereed) Published
Abstract [en]

Background: The use of health technology by older people is coming increasingly in focus with the demographic changes. Health information technology is generally perceived as an important factor in enabling increased quality of life and reducing the cost of care for this group. Age-appropriate design and facilitation of technology adoption are important to ensure functionality and removal of various barriers to usage. Development of assessment tools and instruments for evaluating older persons' technology adoption and usage as well as measuring the effects of the interventions are of high priority. Both usability and acceptance of a specific technology or service are important factors in evaluating the impact of a health information technology intervention. Psychometric measures are seldom included in evaluations of health technology. However, basic attitudes and sentiments toward technology (eg, technophilia) could be argued to influence both the level of satisfaction with the technology itself as well as the perception of the health intervention outcome. Objective: The purpose of this study is to develop a reduced and refined instrument for measuring older people's attitudes and enthusiasm for technology based on relevant existing instruments for measuring technophilia A requirement of the new instrument is that it should be short and simple to make it usable for evaluation of health technology for older people. Methods: Initial items for the TechPH questionnaire were drawn from a content analysis of relevant existing technophilia measure instruments. An exploratory factor analysis was conducted in a random selection of persons aged 65 years or older (N=374) on eight initial items. The scale was reduced to six items, and the internal consistency and reliability of the scale were examined. Further validation was made by a confirmatory factor analysis (CFA). Results: The exploratory factor analysis resulted in two factors. These factors were analyzed and labeled techEnthusiasm and techAnxiety. They demonstrated relatively good internal consistency (Cronbach alpha=.72 and .68, respectively). The factors were confirmed in the CFA and showed good model fit (chi(2)(8)=21.2, chi(2)/df=2.65, comparative fit index=0.97, adjusted goodness-of-fit index=0.95, root mean square error of approximation=0.067, standardized root mean square residual=0.036). Conclusions: The construed TechPH score showed expected relations to external real-world criteria, and the two factors showed interesting internal relations. Different technophilia personality traits distinguish clusters with different behaviors of adaptation as well as usage of new technology. Whether there is an independent association with the TechPH score against outcomes in health technology projects needs to be shown in further studies. The instrument must also be validated in different contexts, such as other countries.

Place, publisher, year, edition, pages
JMIR PUBLICATIONS, INC, 2019
Keywords
technophilia, aging, internet, health technology, eHealth
National Category
Nursing Public Health, Global Health, Social Medicine and Epidemiology
Identifiers
urn:nbn:se:bth-18031 (URN)10.2196/13951 (DOI)000469099700001 ()31124467 (PubMedID)
Note

open access

Available from: 2019-06-14 Created: 2019-06-14 Last updated: 2019-10-29Bibliographically approved
4. Ethical Challenges of Evaluating Health Information Systems
Open this publication in new window or tab >>Ethical Challenges of Evaluating Health Information Systems
(English)Manuscript (preprint) (Other academic)
Abstract [en]

Background

Evaluating and researching health information systems are interventions of their kind and might lead to ethical complexities and challenges. Most of those challenges are inherited from the more general fields of research and evaluation, health studies, and information systems studies. Beyond those challenges, this field has its particular traits, regarding the involved stakeholders, required values or qualities, or the process which can raise field-specific or context-specific ethical challenges.

Objectives

This paper reports and discusses some of the challenges of evaluating and researching health information systems by taking a systematic approach in finding, postulating, and analyzing them.

Method

Through a scoping review, a set of ethical challenges, regarding the evaluation and research of health information systems, were extracted. From the same set of articles, the acting entities, including stakeholders and artefacts, were identified. From a sample of seven cases of health information systems, a set of demanded impact qualities were extracted. From the literature, the evaluation stages were elicited. The acting entities, required qualities, and the evaluation stages were combined to create a three-dimensional space. The space contained the ethical challenges extracted from the scoping review and helped to postulate more items.

Results

The final list of identified items contains 20 possible ethical challenges that can be caused or raised by evaluating or researching health information systems and technologies. The ethical challenges are discussed, based on their probable stage of occurrence. The three-dimensional space and the method of populating it is proposed as an effective method in similar cases of discovering ethical challenges.

Conclusion

Evaluating or researching health information systems can raise ethical challenges, that we have identified 20 of them in this article. All the challenges were discussed, such as the actual value of evaluation, breach of privacy, risks for safety, problems with usability and accessibility, conflict of interests, problems with the informed consent, and miscommunication. The novel approach for elicitation of the ethical challenges introduced in this article might be applied in other similar studies.

National Category
Medical Ethics
Identifiers
urn:nbn:se:bth-18794 (URN)
Available from: 2019-10-29 Created: 2019-10-29 Last updated: 2019-10-29
5. Design of a Semi-Automated and Continuous Evaluation System: Customized for Application in e-Health
Open this publication in new window or tab >>Design of a Semi-Automated and Continuous Evaluation System: Customized for Application in e-Health
Show others...
(English)Manuscript (preprint) (Other academic)
Abstract [en]

Background and Objectives

Survey-based evaluation of a system, such as measuring user’s satisfaction or patient-reported outcomes, entails a set of burdens that limits the feasibility, frequency, extendability, and continuity of the evaluation. Automating the evaluation process, that is reducing the burden of evaluators in questionnaire curation or minimizing the need for explicit user attention when collecting their attitudes, can make the evaluation more feasible, repeatable, extendible, continuous, and even flexible for improvement. An automated evaluation process can be enhanced to include features, such as the ability to handle heterogeneity in evaluation cases. Here, we represent the design of a system that makes it possible to have a semi-automated evaluation system. The design is presented and partially implemented in the context of health information systems, but it can be applied to other contexts of information system usages as well.

Method

The system was divided into four components. We followed a design research methodology to design the system, where each component reached a certain level of maturity. Already implemented and validated methods from previous studies were embedded within components, while they were extended with improved automation proposals or new features.

Results

A system was designed, comprised of four major components: Evaluation Aspects Elicitation, User Survey, Benchmark Path Model, and Alternative Metrics Replacement. All components have the essential maturity of identification of the problem, identification of solution objectives, and the overall design. In the overall design, the primary flow, process-entities, data-entities, and events for each component are identified and illustrated. Parts of some components have been already verified and demonstrated in real-world cases.

Conclusion

A system can be developed to minimize human burden, both for the evaluators and respondants, in survey-based evaluation. This system automates finding items to evaluate, creating questionnaire based on those items, surveying the users' attitude about those items, modeling the relations between the evaluation items, and incrementally changing the model to rely on automatically collected metrics, usually implicit indicators, collected from the users, instead of requiring their explicit expression of their attitudes. The system provides the possibility of minimal human burden, frequent repetition, continuity and real-time reporting, incremental upgrades regarding environmental changes, proper handling of heterogeneity, and a higher degree of objectivity.

Keywords
Evaluation, Questionnaire, Information Systems, Patient Reported Outcome, User’s Attitude, Automation
National Category
Computer Systems
Identifiers
urn:nbn:se:bth-18795 (URN)
Available from: 2019-10-29 Created: 2019-10-29 Last updated: 2019-10-29

Open Access in DiVA

fulltext(5965 kB)8 downloads
File information
File name FULLTEXT02.pdfFile size 5965 kBChecksum SHA-512
31ad087295ff1ef63cd5c374b9f4a27db33e112d3cb67f9ccf5f59d3cf85cab97922768ee2e47e796cba0a37a8613f2f1c814cbc743b896ed6cb1da8969f60f2
Type fulltextMimetype application/pdf

Authority records BETA

Eivazzadeh, Shahryar

Search in DiVA

By author/editor
Eivazzadeh, Shahryar
By organisation
Department of Health
Other Electrical Engineering, Electronic Engineering, Information Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 17 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 96 hits
5678910118 of 32
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf