Featured Application This work has applied computer vision and deep learning technology to develop a real-time weapon detector system and tested it on different computing devices for large-scale deployment. Weapon detection in CCTV camera surveillance videos is a challenging task and its importance is increasing because of the availability and easy access of weapons in the market. This becomes a big problem when weapons go into the wrong hands and are often misused. Advances in computer vision and object detection are enabling us to detect weapons in live videos without human intervention and, in turn, intelligent decisions can be made to protect people from dangerous situations. In this article, we have developed and presented an improved real-time weapon detection system that shows a higher mean average precision (mAP) score and better inference time performance compared to the previously proposed approaches in the literature. Using a custom weapons dataset, we implemented a state-of-the-art Scaled-YOLOv4 model that resulted in a 92.1 mAP score and frames per second (FPS) of 85.7 on a high-performance GPU (RTX 2080TI). Furthermore, to achieve the benefits of lower latency, higher throughput, and improved privacy, we optimized our model for implementation on a popular edge-computing device (Jetson Nano GPU) with the TensorRT network optimizer. We have also performed a comparative analysis of the previous weapon detector with our presented model using different CPU and GPU machines that fulfill the purpose of this work, making the selection of model and computing device easier for the users for deployment in a real-time scenario. The analysis shows that our presented models result in improved mAP scores on high-performance GPUs (such as RTX 2080TI), as well as on low-cost edge computing GPUs (such as Jetson Nano) for weapon detection in live CCTV camera surveillance videos.

With the growing demand for image and video-based applications, the requirements of consistent quality assessment metrics of image and video have increased. Different approaches have been proposed in the literature to estimate the perceptual quality of images and videos. These approaches can be divided into three main categories; full reference (FR), reduced reference (RR) and no-reference (NR). In RR methods, instead of providing the original image or video as a reference, we need to provide certain features (i.e., texture, edges, etc.) of the original image or video for quality assessment. During the last decade, RR-based quality assessment has been a popular research area for a variety of applications such as social media, online games, and video streaming. In this paper, we present review and classification of the latest research work on RR-based image and video quality assessment. We have also summarized different databases used in the field of 2D and 3D image and video quality assessment. This paper would be helpful for specialists and researchers to stay well-informed about recent progress of RR-based image and video quality assessment. The review and classification presented in this paper will also be useful to gain understanding of multimedia quality assessment and state-of-the-art approaches used for the analysis. In addition, it will help the reader select appropriate quality assessment methods and parameters for their respective applications.

In this paper, we examine an adaptive deblocking deringing filter for mobile video sequences in H.263 format. The considered filter has been designed with reference to the constraints of computational complexity and working memory of mobile terminals. The post filter suggested by the International Telecommunications Union (ITU) in Recommendation H.263 App. III is also included as a reference. Given that fidelity metrics such as the peak signal-to-noise ratio (PSNR) do not necessarily correlate well with video quality as experienced by the user, we consider in this paper objective quality metrics that can incorporate knowledge about the user's perception into the quality assessment. Guidelines for choosing filter parameters in relation to user-perceived video quality are obtained from the numerical results.

The emergence of electric vehicles offers a promising approach to achieving a more sustainable transportation system, given their lower production of direct emissions. However, the limited driving range and insufficient public recharging infrastructure in some areas hinder their competitiveness against traditional vehicles with internal combustion engines. To address these issues, this paper introduces an ``iterative route cover optimization method'' to suggest  charging station locations in high-demand regions. The method samples routes from a route choice set and optimally locates at least one charging station along each  route. Through iterative resampling and optimal allocation of charging stations, the method identifies the potential recharging demand in a location or a region. We demonstrate the method's applicability to a transportation network of the southern part of Sweden. The results show that the proposed method is capable to suggest locations and geographical regions where the recharging demand is potentially high. 

Modern vehicles are to a large extent connected today, either directly by built-in navigation systems in the vehicles or indirectly by other devices such as mobile phones and GPS units. This enables the possibility to continuously collect traffic data in a cost-effective way. The increased access to detailed data allows practitioners and researchers to analyze the transportation system from various perspectives. The travel speed is a common descriptor of the traffic state, and it can be extracted from GPS data. By analyzing how the travel speed vary over time and detect anomalies among the measured travel speeds, it is possible to detect potential deficiencies in the transportation system, e.g., insufficient road capacity which may cause bottlenecks. Often, a weakness in the infrastructure is detected in a very late stage which means that extensive investments may be required to resolve the deficiency.

The purpose of the pilot study is to develop methods and models to detect deficiency in the transportation system and to identity travel speeds that deviates from the normal state, i.e., travel speeds that are considered as very low or very high with respect to the normal behavior. Thus, the starting point of the pilot study is to find appropriate ways to model the traffic state along the studied road segments by using measured travel speeds from a general point of view. Analysis of the traffic state allows the study of how the normal state of the road segments change of time to detect deficiency related to road capacity and road access which may occur if no changes are made, or to detect road segments where the normal state is unchanged.

Typically, slower travel speeds may be an indicator of that a deficiency along a road segment exists. Thus, we present a method to systematically partition measured travel speeds in low, normal, and high travel speeds. The method is robust and enable the possibility to compare different road segment with different attributes, such as number of lanes and free-flow travel speed, with each other. Furthermore, we present a new measurement to describe how the low travel speeds relates to the free flow travel speed, e.g., the speed limit. Existing measurements and indicators used today utilize travel speeds which range from low to high. Our proposed measurement uses low travel speed and free flow travel speed exclusively and aims to quantify the accessibility and condition of a road segment.

The pilot study also includes an initial attempt to apply cluster analysis to detect recurrent patterns along the studied road segments. Cluster analysis is in several contexts an effective method to group time series to detect recurrent patterns among the speed profiles. The purpose of using cluster analysis is to evaluate if speed profiles with similar behavior is related to, for instance, weekday or time of the day. Thus, cluster analysis may be used to detect road segments with recurring low travel speeds, and potentially be used to forecast when congestion or queues may occur.

The pilot study is mainly limited to travel speed data. The proposed methods and models show that it is possibly to solely use travel speed data to detect deficiencies in the transportation system. In particular, the pilot study shows the potential to detect deficiencies in the transportation system without additional data sources such as link flow data.

Traffic flows play a very important role in transportation engineering. In particular, link flows are a source of information about the traffic state, which is usually available from the authorities that manage road networks. Link flows are commonly used in both short-term and long-term planning models for operation and maintenance, and to forecast the future needs of transportation infrastructure. In this paper, we propose a model to study how traffic flow in one location can be expected to reflect the traffic flow in a nearby region. The statistical basis of the model is derived from link flows to find estimates of the distribution of traffic flows in junctions. The model is evaluated in a numerical study, which uses real link flow data from a transportation network in southern Sweden. The results indicate that the model may be useful for studying how large departing flows from a node reflect the link flows in a neighboring geographic region. 

Travel time reliability is vital for both road agencies and road users. Expected travel time reliability can be used by road agencies to assess the state of a transportation system, and by road users, to schedule their trips. Road network deficiencies, such as insufficient traffic flow capacity of a road segment or poor road design, have a negative impact on the reliability of travel times. Thus, to maintain robust and reliable travel times, the detection of road network deficiencies is vital. By continuously analyzing travel times and using appropriate travel time reliability measurements, it is possible to detect existing deficiencies or deficiencies that may eventually occur unless necessary actions are taken. In many cases, indices and measurements of travel time reliability are related to the distribution of the travel times, specifically the skewness and width of the distribution. The current paper introduces a median-based misery index for travel time reliability. The index is robust and handles travel times that follow a skewed distribution well. The index measures the relative difference between the slow travel speeds and the free-flow travel speed. The index is inspired by the median absolute deviation, and its primary application is to detect routes or road segments with potential road network deficiencies. To demonstrate the applicability of the index, we conducted an empirical case study using real travel speed data from the European route E4 in Sweden. The results from the empirical case study indicate that the index is capable of detecting road segments with slow travel speeds regardless of the travel speed distribution.

Travel times and travel time reliability are important indicators of the traffic state in a transportation system. Analyzing travel times and their reliability and how they vary over time makes it possible to identify existing road network deficiencies or deficiencies that may eventually occur unless necessary actions are taken. Road network deficiencies, such as inadequate road capacity or substandard road design, typically lead to congestion and trip delays for road users. A novel median-based misery index is proposed to highlight potential road network deficiencies in the transportation system. The proposed index offers a way to gauge travel time reliability, providing valuable insights into roads where improvements may be needed. The newly developed median-based misery index operates by computing the relative disparity between slow travel speeds and free-flow travel speeds. The median-based misery index is more robust to skewed distributions of travel times than the ordinary misery index. Our empirical case study includes a spatiotemporal analysis of travel speed data from a section of the European route E4 in Sweden. The case study results show that the new index may be used to detect peak periods when the traffic conditions for road users have deteriorated.

Adaptive gain equalizer (AGE) is a commonly used single-channel speech enhancement algorithm. AGE and its variants has been widely used for speech enhancement applications. There are two broad categories of these variants. The first deals with its improvement in time-frequency domain with readjustment of the used parameters and the second one deals with performing the main filtering operation in modulation frequency domain. This paper evaluates the working of AGE in modulation frequency domain with the use of a demodulation technique which solves the demodulation process as a convex optimization problem. The performance of the modified AGE is compared with the traditional AGE and another modulation frequency domain AGE based on demodulation using the spectral center-of-gravity. These used performance measures are Signal to Noise Ratio Improvement(SNRI), Spectral Distortion(SD) and Mean Option Score(MOS).

his paper presents a single channel speech enhancement technique based on sub-band modulator Kalman filtering for laryngeal (normal) and alaryngeal (Esophageal speech) speech signals. The noisy speech signal is decomposed into sub-bands and subsequently each sub-band is demodulated into its modulator and carrier components. Kalman filter is applied to modulators of all sub-bands without altering the carriers. Performance of the proposed system has been validated by Mean Opinion Score (MOS) for laryngeal and Harmonic to Noise Ratio (HNR) for alaryngeal speech. An improvement of 20% has been observed in MOS over sub-band Kalman filtering for laryngeal speech, while 3 to 4 dB enhancement in HNR has been observed for alaryngeal speech over the full-band Kalman filtering.

In real-time video streaming, video quality can be degraded due to network performance issues. Among other artefacts, video freezing and video jumping are factors that influence user experience. Service providers, operators and manufacturers are interested in evaluating the quality of experience (QoE) objectively because subjective assessment of QoE is expensive and, in many user cases, subjective assessment is not possible to perform. Different algorithms have been proposed and implemented in this regard. Some of them are in the recommendation list of the ITU Telecommunication Standardization Sector (ITU-T). In this paper, we study the effect of the freezing artefact on user experience and compare the mean opinion score of these videos with the results of two algorithms, the perceptual evaluation of video quality (PEVQ) and temporal quality metric (TQM). Both metrics are part of the ITU-T Recommendation J.247 Annex B and C. PEVQ is a full-reference video quality metric, whereas TQM is a no-reference quality metric. Another contribution of this paper is the study of the impact of different resolutions and frame rates on user experience and how accurately PEVQ and TQM measure varying frame rates.

Video streaming and multimedia applications are getting popular with the growth of networks. In real-time video streaming, video quality can be degraded due to network performance issues. Among other artifacts, freezing and frame dropping are factors that influence user experience. Service providers, operators, and researchers are interested to measure the Quality of Experience objectively. Different algorithms have been proposed and implemented in this regard. Some of them are in the recommendation list of the ITU Telecommunication Standardization Sector (ITU-T). In this paper, we study the effect of the freezing artifact on user experience and compare the mean opinion score (MOS) of these videos with the results of two algorithms, Perceptual Evaluation of Video Quality (PEVQ) and Temporal Quality Metric, both being part of ITU-T Recommendation J.247 Annex B and C, respectively. Another contribution of this paper is the investigation of the impact of different resolutions and frame rates on user experience.

In order to estimate subjective video quality, we usually deal with a large number of features and a small sample set. Applying regression on complex datasets may lead to imprecise solutions due to possibly irrelevant or noisy features as well as the effect of overfitting. In this work, we propose a No-Reference (NR) method for the estimation of the quality of videos that are impaired by both compression artifacts and packet losses. Particularly, in an effort to establish a robust regression model that generalizes well to unknown data and to increase Mean Opinion Score (MOS) estimation accuracy, we propose a frame-level MOS estimation approach, where the MOS estimate of a sequence is obtained by averaging the perframe MOS estimates, instead of performing regression directly at the sequence-level. Since it is impractical to obtain the actual perframe MOS values through subjective experiments, we propose an objective metric able to do this task. Thus, our proposed NR method has the dual benefit of offering improved sequence-level MOS estimation accuracy, while giving an indication of the relative quality of each individual video frame.

In this work, we propose a No-Reference (NR) bitstream-based model for predicting the quality of H.264/AVC video sequences, a effected by both compression artifacts and transmission impairments. The concept of the article is based on a feature extraction procedure, where a large number of features are calculated from the impaired bitstream. Many of the features are mostly proposed in this work, while the specific c set of the features as a whole is applied for the first time for making NR video quality predictions. All feature observations are taken as input to the Least Absolute Shrinkage and Selection Operator (LASSO) regression method. LASSO indicates the most important features, and using only them, it is able to estimate the Mean Opinion Score (MOS) with high accuracy. Indicatively, we point out that only 13 features are able to produce a Pearson Correlation Coefficient of 0:92 with the MOS. Interestingly, the performance statistics we computed in order to assess our method for predicting the Structural Similarity Index and the Video Quality Metric are equally good. Thus, the obtained experimental results verifi ed the suitability of the features selected by LASSO as well as the ability of LASSO in making accurate predictions through sparse modeling.

In many applications and environments for mobile communication there is a need for reference free perceptual quality measurements. In this paper a method for prediction of a number of quality metrics is proposed, where the input to the prediction is readily available parameters at the receiver side of a communications channel. Since the parameters are extracted from the coded video bit stream the model can be used in user scenarios where it is normally difficult to estimate the quality due to the reference not being available, as in streaming video and mobile TV applications. The predictor turns out to give good results for both the PSNR and the PEVQ metrics.

In the mobile communication area there is a demand for reference free perceptual quality measurements in video applications. In addition low complexity measurements are required. This paper proposes a method for prediction of a number of well known quality metrics, where the inputs to the predictors are readily available parameters at the decoder side of the communication channel. After an investigation of the dependencies between these parameters and between each parameter and the quality metrics, a set of parameters is chosen for the predictor. This predictor shows good results, especially for the PSNR

One of the main contributions to the quality of experience in streaming services or in two-way communication of audio and video applications is synchronization. This has been shown in several studies and experiments but methods to measure synchronization are less frequent, especially for situations without internal access to the application and independent of platform and device. In this paper we present a method for measuring synchronization skewness as well as delay for audio and video. The solution incorporates audio and video reference streams, where audio and video frames are marked with frame numbers which are decoded on the receiver side to enable calculation of synchronization and delay. The method has been verified in a two-way communication application in a transparent network with and without inserting known delays, as well as in a network with 5 and 10 % packet loss levels. The method can be used for both streaming and two-way communication services, both with and without access to the internal structures, and enables measurements of applications running on e.g. smartphones, tablets, and laptops under various conditions.

In this paper an adaptive filter that removes de-blocking and de-ringing artifacts and also enhances the sharpness of decoded video, which may be caused by zeroing high-frequency DCT coefficients, is presented. The solution is designed with consideration of Mobile Equipment with limited computational power and memory. Also, the solution is computationally scalable to be able to handle limited computational resources in different user cases. In the paper it is shown that the adaptive filter always keeps or increases the image quality, compared to the original decoded sequences, and that the amount of sharpening decreases with an decrease of bit-rate to limit amplification of coding artifacts or noise.

The growing consumer interest in video communication has increased the users' awareness in the visual quality of the delivered media. This in turn increases, at the service provider end, the need for intelligent methodologies of optimal techniques for adapting to varying network conditions. Recent studies show that constraints on the bandwidth of transmission media should not always be translated to an increase in compression ratio to lower the bitrate of the video. Instead, a suitable option for adaptive streaming is to scale down the video temporally or spatially before encoding to maintain a desirable level of perceptual quality, while the viewing resolution is constant. Most of the existing studies to examine these scenarios are either limited to low resolution videos or lack in provisioning of subjective assessment of quality. We present here the results of our campaign of subjective quality assessment experiments done on a range of spatial and temporal resolutions, up to VGA and 30 frames per second respectively, under a number of bitrate conditions. The analysis shows, among other things, that keeping the spatial resolution is perceptually preferred among the three parameters that have impact on the video quality, even in the case with high temporal activity.

An image enhancement method that reduces speckle noise and preserves edges is introduced. The method is based on a new nonlinear multiscale reconstruction scheme that is obtained by successively combining each coarser scale image with the corresponding modified interscale image. Simulation results are included to demonstrate the performance of the proposed method.

This paper evaluates speech enhancement by filtering in the modulation frequency domain, as an alternative to filtering in conventional frequency domain. Adaptive Gain Equalizer (AGE) is a commonly used single-channel speech enhancement algorithm. A recently introduced class of signal transformations called modulation transform has successfully made its place alongside classical time/frequency representations. This paper presents an implementation of AGE within modulation system, for the purpose of enhancing the speech signal. The successful implementation of the proposed system has been validated with various performance measurements, i.e., Signal to Noise Ratio Improvement (SNRI), Mean Opinion Score (MOS) and Spectral Distortion (SD). A spectrogram analysis is also presented to further substantiate the performance of this work

This paper presents bitstream-based features for perceptual quality estimation of HEVC coded videos. Various factors including the impact of different sizes of block-partitions, use of reference-frames, the relative amount of various prediction modes, statistics of motion vectors and quantization parameters are taken into consideration for producing 52 features relevant for perceptual quality prediction. The used test stimuli constitutes 560 bitstreams that have been carefully extracted for this analysis from the 59, 520 bistreams of the large-scale database generated by the Joint Effort Group (JEG) of the Video Quality Experts Group (VQEG). The obtained results show the significance of the considered features through reasonably accurate and monotonic prediction of a number of objective quality metrics. © 2015 IEEE.

Reduced-reference (RR) and no-reference (NR) models for video quality estimation, using featuresthat account for the impact of coding artifacts, spatio-temporal complexity, and packet losses, are proposed. Thepurpose of this study is to analyze a number of potentially quality-relevant features in order to select the mostsuitable set of features for building the desired models. The proposed sets of features have not been used in theliterature and some of the features are used for the first time in this study. The features are employed by the leastabsolute shrinkage and selection operator (LASSO), which selects only the most influential of them toward per-ceptual quality. For comparison, we apply feature selection in the complete feature sets and ridge regression onthe reduced sets. The models are validated using a database of H.264/AVC encoded videos that were subjec-tively assessed for quality in an ITU-T compliant laboratory. We infer that just two features selected by RRLASSO and two bitstream-based features selected by NR LASSO are able to estimate perceptual qualitywith high accuracy, higher than that of ridge, which uses more features. The comparisons with competingworks and two full-reference metrics also verify the superiority of our models.

In the video encoding process, the motion estimation usually consumes a large part of the encoder computations. This paper presents motion estimation techniques, targeted mainly for MPEG-4 video encoding but also applicable for other video codecs e.g. H.264. A high quality adaptive algorithm with adjustable complexity, based on partially blind prediction for motion estimation, is proposed.The computational complexity of motion estimation is reduced with minor loss in the video quality. In the paper, the quality metrics PSNR, BD PSNR and PEVQ are used, and the possible trade off between complexity and visual quality is studied.

The growing need of quick and online estimation of video quality necessitates the study of new frontiers in the area of no-reference visual quality assessment. Bitstream-layer model based video quality predictors use certain visual quality relevant features from the encoded video bitstream to estimate the quality. Contemporary techniques vary in the number and nature of features employed and the use of prediction model. This paper proposes a prediction model with a concise set of bitstream based features and a machine learning based quality predictor. Several full reference quality metrics are predicted using the proposed model with reasonably good levels of accuracy, monotonicity and consistency.

There is a growing need for robust methods for reference free perceptual quality measurements due to the increasing use of video in hand-held multimedia devices. These methods are supposed to consider pertinent artifacts introduced by the compression algorithm selected for source coding. This paper proposes a model that uses readily available encoder parameters as input to an artificial neural network to predict objective quality metrics for compressed video without using any reference and without need for decoding. The results verify its robustness for prediction of objective quality metrics in general and for PEVQ and PSNR in particular. The paper also focuses on reducing the complexity of the neural network.

The field of perceptual quality assessment has gone through a wide range of developments and it is still growing. In particular, the area of no-reference (NR) image and video quality assessment has progressed rapidly during the last decade. In this article, we present a classification and review of latest published research work in the area of NR image and video quality assessment. The NR methods of visual quality assessment considered for review are structured into categories and subcategories based on the types of methodologies used for the underlying processing employed for quality estimation. Overall, the classification has been done into three categories, namely, pixel-based methods, bitstream-based methods, and hybrid methods of the aforementioned two categories. We believe that the review presented in this article will be helpful for practitioners as well as for researchers to keep abreast of the recent developments in the area of NR image and video quality assessment. This article can be used for various purposes such as gaining a structured overview of the field and to carry out performance comparisons for the state-of-the-art methods.

Advancements in the video processing area have been proliferated by services that require low delay. Such services involve applications being offered at various temporal and spatial resolutions. It necessitates to study the impacts of related video coding conditions upon perceptual quality. But most of studies concerned with quality assessment of videos affected by coding distortions lack in variety of spatio-temporal resolutions. This paper presents a work done on quality assessment of videos encoded by state-of-the-art H.264/AVC standard at different bitrates and frame rates. Overall, 120 test scenarios for video sequences having different spatial and temporal spectral information were studied. The used coded bistreams in this work and the corresponding subjective assessment scores have been made public for the research community to facilitate further studies

The growing popularity of adaptive streaming-based video delivery nowadays has raised the interest about the user's perception when experiencing quality adaptation. The impact of the video content characteristics on user's perceptual quality has already become evident. The aim of this study is to investigate the influence of this factor on the quality of experience of adaptive streaming scenarios. Our results show that the perceptual quality of adaptation strategies applied on videos with high spatial and low temporal amount of activity is significantly lower compared to the other content types.

In wireless networks, due to limited bandwidth and packet losses, seamless and ubiquitous delivery of high-quality video streaming services is a big challenge for the operators. In order to improve the process of online video quality monitoring, the presence of no reference (NR) objective video quality assessment (VQA) methods is required. In some networks, the video decoder on the reception side adopts a mechanism in which last correctly received frame is frozen and displayed on video display terminal until the next correct frame is received. This phenomenon, employed as an error concealment technique, can cause a perceptual jerkiness on the video display terminal. In this paper, we have proposed an enhanced model of objective VQA based on the estimation of jerkiness. A study of three contemporary NR methods, used for objective VQA and online monitoring of videos, has been included along with subjective VQA tests. The subjective tests were performed for a set of video sequences with specific spatial and temporal information. The proposed NR method is based on our careful observations from the subjective test results and our main focus is to cater the effect of multiple frame freeze impairments in video steaming. Comparison with other NR methods shows that the proposed method performs better, in terms of estimating the impact of multiple frame freezing impairments, and has more affinity with the subjective test results.

A digital Video Stabilization (DVS) system removes the unwanted shaking in the videos acquired by hand-held cameras and preserves the panning. In this paper, a digital video stabilization system is proposed based upon adaptive cerebellar model articulation controller (CMAC) filtering. A CMAC is a manifestation of the associative memory learning structure present in the cerebellum of human being. Adaptive CMAC filtering has favorable properties of small size, good generalization, rapid learning and dynamic response. Thus, it is more suitable for high-speed signal processing applications. The adaptive CMAC is used to adjust the coefficients of IIR filter employed in the proposed model. The training of CMAC is based upon fuzzy rule. The efficacy of the proposed adaptive CMAC filtering has been validated by evaluating it on a set of test video sequences.