Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Correcting and complementing freeway traffic accident data using mahalanobis distance based outlier detection
Blekinge Institute of Technology, Faculty of Computing, Department of Creative Technologies.ORCID iD: 0000-0001-5824-425X
Blekinge Institute of Technology, Faculty of Computing, Department of Creative Technologies.
Blekinge Institute of Technology, Faculty of Computing, Department of Creative Technologies.
Blekinge Institute of Technology, Faculty of Computing, Department of Creative Technologies.ORCID iD: 0000-0003-0891-2859
2017 (English)In: Technical Gazette, ISSN 1330-3651, E-ISSN 1848-6339, Vol. 24, no 5, p. 1597-1607Article in journal (Refereed) Published
Abstract [en]

A huge amount of traffic data is archived which can be used in data mining especially supervised learning. However, it is not being fully used due to lack of accurate accident information (labels). In this study, we improve a Mahalanobis distance based algorithm to be able to handle differential data to estimate flow fluctuations and detect accidents and use it to support correcting and complementing accident information. The outlier detection algorithm provides accurate suggestions for accident occurring time, duration and direction. We also develop a system with interactive user interface to realize this procedure. There are three contributions for data handling. Firstly, we propose to use multi-metric traffic data instead of single metric for traffic outlier detection. Secondly, we present a practical method to organise traffic data and to evaluate the organisation for Mahalanobis distance. Thirdly, we describe a general method to modify Mahalanobis distance algorithms to be updatable. © 2017, Strojarski Facultet. All rights reserved.

Place, publisher, year, edition, pages
Strojarski Facultet , 2017. Vol. 24, no 5, p. 1597-1607
Keywords [en]
Accident data, Data labelling, Differential distance, Mahalanobis distance, Outlier detection, Traffic data, Updatable algorithm, Accidents, Data mining, Statistics, User interfaces, Mahalanobis distances, Data handling
National Category
Communication Systems Computer and Information Sciences
Identifiers
URN: urn:nbn:se:bth-15472DOI: 10.17559/TV-20150616163905ISI: 000417100300037Scopus ID: 2-s2.0-85032512786OAI: oai:DiVA.org:bth-15472DiVA, id: diva2:1156201
Note

Funded by National Natural Science Foundation of China

Funding nr. 61364019

Available from: 2017-11-10 Created: 2017-11-10 Last updated: 2018-11-01Bibliographically approved
In thesis
1. Automated Traffic Time Series Prediction
Open this publication in new window or tab >>Automated Traffic Time Series Prediction
2018 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Intelligent transportation systems (ITS) are becoming more and more effective. Robust and accurate short-term traffic prediction plays a key role in modern ITS and demands continuous improvement. Benefiting from better data collection and storage strategies, a huge amount of traffic data is archived which can be used for this purpose especially by using machine learning.

For the data preprocessing stage, despite the amount of data available, missing data records and their messy labels are two problems that prevent many prediction algorithms in ITS from working effectively and smoothly. For the prediction stage, though there are many prediction algorithms, higher accuracy and more automated procedures are needed.

Considering both preprocessing and prediction studies, one widely used algorithm is k-nearest neighbours (kNN) which has shown high accuracy and efficiency. However, the general kNN is designed for matrix instead of time series which lacks the use of time series characteristics. Choosing the right parameter values for kNN is problematic due to dynamic traffic characteristics. This thesis analyses kNN based algorithms and improves the prediction accuracy with better parameter handling using time series characteristics.

Specifically, for the data preprocessing stage, this work introduces gap-sensitive windowed kNN (GSW-kNN) imputation. Besides, a Mahalanobis distance-based algorithm is improved to support correcting and complementing label information. Later, several automated and dynamic procedures are proposed and different strategies for making use of data and parameters are also compared.

Two real-world datasets are used to conduct experiments in different papers. The results show that GSW-kNN imputation is 34% on average more accurate than benchmarking methods, and it is still robust even if the missing ratio increases to 90%. The Mahalanobis distance-based models efficiently correct and complement label information which is then used to fairly compare performance of algorithms. The proposed dynamic procedure (DP) performs better than manually adjusted kNN and other benchmarking methods in terms of accuracy on average. What is better, weighted parameter tuples (WPT) gives more accurate results than any human tuned parameters which cannot be achieved manually in practice. The experiments indicate that the relations among parameters are compound and the flow-aware strategy performs better than the time-aware one. Thus, it is suggested to consider all parameter strategies simultaneously as ensemble strategies especially by including window in flow-aware strategies.

In summary, this thesis improves the accuracy and automation level of short-term traffic prediction with proposed high-speed algorithms.

Place, publisher, year, edition, pages
Karlskrona: Blekinge Tekniska Högskola, 2018
Series
Blekinge Institute of Technology Doctoral Dissertation Series, ISSN 1653-2090 ; 10
Keywords
Machine Learning, Time Series, Traffic Engineering
National Category
Computer Sciences Transport Systems and Logistics
Identifiers
urn:nbn:se:bth-17210 (URN)978-91-7295-360-4 (ISBN)
Public defence
2018-11-30, J1650, Valhallav. 1, Karlskrona, 13:30 (English)
Opponent
Supervisors
Available from: 2018-11-02 Created: 2018-11-01 Last updated: 2018-12-14Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records BETA

Sun, BinCheng, WeiBai, GuohuaGoswami, Prashant

Search in DiVA

By author/editor
Sun, BinCheng, WeiBai, GuohuaGoswami, Prashant
By organisation
Department of Creative Technologies
In the same journal
Technical Gazette
Communication SystemsComputer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 157 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf