This paper presents a contribution to better interpretation of the results we get from GenAI models, more specifically, better interpretation of the mistakes that they make. We have conducted an analysis of 644 (from GPT-4o) + 4858 (from ARIA) mistakes made by two models on a key-value extraction task, and found that they may be categorised into three mutually exclusive groups. These groups are; i problems identifying the requested information, p problems presenting the correct information, and s skewed training data. These categories could be used to indicate which action a user could take to reduce the number of mistakes. Further, we have found a strong correlation between the suggested categories and the Ratcliff/Obershelp pattern recognition score between the generated result and the expected result; all faulty results containing minor mistakes are more than 60% similar to the expected result. Only mistakes based on lack of identifying what was requested had less than 60% similarity to the expected result. 

We evaluate Forward Error Correction (FEC) codes in the context of a novel routing protocol HDARP+ for airborne networks. HDARP+ uses directional antennas and dynamic FEC coding to avoid detection by adversaries. The use of FEC coding is dynamic in the sense that different FEC codes, or no FEC code, will be used depending on the relative position of friendly and adversary aircraft. Due to the real-time restrictions in airborne networks, encoding and decoding must be fast and done through table lookup. Since we use table lookup, the FEC codes must be short. We evaluate two types of short FEC codes: Reed-Solomon (RS) codes, and FEC codes found using greedy search (called GS codes). The results show that the RS codes are better than the GS codes at handling error bursts. However, the GS codes are more flexible when it comes to finding attractive trade-offs between the code's ability to increase the number of the cases when hostile detection can be avoided (related to the coding gain), the code rate and the amount of memory required for implementing lookup tables.

This paper presents two experiments testing the capabilities of the models GPT-4-vision-preview and GPT-4o, to transcribe images of randomly generated number-strings. We noticed a tendency for longer number strings to be read inaccurately while transcribing invoices using the models, so we created an experiment with images with minimal noise to see what amount of digits these models are able to transcribe with full accuracy. We further tested weather the mistakes that occur in transcribing longer strings are reoccurring when the images are re-tested. We analyzed the character of the mistakes made, and weather any digits are over represented among the ones involved in mistakes. We found, among more results described in the paper, that the models are 100% accurate up to 75 digits per image, that the same mistakes are reoccurring when the same image is rerun, and that hallucinations are only 23% of all mistakes made by the models. 

With the rapid growth of internet technologies, IT businesses are transferring to cloud-based systems, and cloud-based services are in high demand among internet users. Therefore, appropriate allocation of resources in cloud computing environments is essential. The companies can reduce costs by saving energy by dynamically scaling up or down the number of active servers. In this context, this study presents a machine learning-based model for accurate prediction of CPU utilization. Previous studies employed timestamp-based data to predict CPU utilization in cloud computing, while the proposed work uses incoming user requests to predict CPU workload so that a timely decision can be made to scale up or scale down the servers in a cloud computing environment. The proposed model is based on several machine learning algorithms that are stacked into a single model called the stacking model for CPU workload prediction. The effectiveness of the proposed stacking model was tested on several evaluation metrics to validate its performance. Furthermore, the performance of the proposed stacking model is also compared with other state-of-the-art machine learning models such as support vector machines (SVM), decision trees (DT), random forests (RF), gradient boosting, and extreme gradient boosting (XGBoost). 

Data stream mining is a research area that has grown enormously in recent years. The main challenge is extracting knowledge in real-time from a possibly unbounded data stream. Clustering, a process in which groupings within the data are identified, is a valuable technique to extract and identify underlying structures of the data. An open question in stream clustering is how to evaluate the proposed algorithms. In this survey, we review the literature in the domain to identify common methodologies, datasets, and evaluation measures, used to evaluate the algorithms. We provide a short summary of the stream clustering algorithms in the literature, but our primary focus lies in the survey of cluster validation relevant to the evaluation of data stream clustering algorithms. We begin our literature review with the inception of clustering incrementally, namely with the introduction of the balanced iterative reducing and clustering using hierarchies (BIRCH) algorithm. We identify that the evaluation methodologies primarily focus on performance, and that aspects such as cluster quality are rarely considered. Performance has been the focal point of all evaluations, both in terms of computational performance and accuracy, since the inception of clustering data streams. We also identify that issues in the conventional clustering domain are present in the data stream clustering. However, minor additions to the evaluation methods can improve both the applicability and usefulness of the algorithms. 

This paper provides details and analysis of findings from three experiments on the capabilities of the OpenAI chatbot ChatGPT-4 with vision capabilities as a dual-purpose tool for Optical Character Recognition (OCR) and key-value data extraction tasks. Our results could be relevant to any task where one is interested in extracting key information from images with approximately equal complexity, and are scalable to be used for large datasets. We did the main experiments in the OpenAI user interface for the model, and a smaller experiment using the API. The experiment used a dataset comprising 1000 digital invoices alongside 1000 photographic images of real receipts, collected from a broad spectrum of market sectors within Sweden. The main experiment gave us a significant accuracy rate, achieving $\mathbf{9 9. 8 \%}$ in extracting critical financial information from the digital invoices. Similarly, when applied to the photographic images of receipts, it maintained a high accuracy level of $\mathbf{9 9. 5 \%}$. The smaller experiment gave us an accuracy of $\mathbf{9 4. 4 \%}$ These findings are particularly noteworthy not only because of the high accuracy levels but also due to the model's effectiveness in performing both OCR and key-value extraction tasks as a one-step process. This dual functionality underscores the model's potential as a highly efficient and reliable solution in automating financial data extraction and processing tasks. 

In the competitive business-to-business (B2B) landscape, retaining clients is critical to sustaining growth, yet customer churn presents substantial challenges. This paper presents a novel approach to customer churn prediction using a modified Transformer architecture tailored to multivariate time-series data. We suggest that analyzing customer behavior patterns over time can indicate potential churn. Our findings suggest that while uncertainty remains high, the proposed model performs competitively against existing methods. The Transformer architecture achieves a top decile lift of almost 5 and 0.77 AUC. We assess the model’s confidence by employing conformal prediction, providing valuable insights for targeted anti-churn campaigns. This work highlights the potential of Transformers to address churn dynamics, offering a scalable solution to identify at-risk customers and inform strategic retention efforts in B2B contexts.

Domain-Specific Hardware Accelerators (DSHA) are natural components in the evolution of general computers. However, designing and simulating hardware in Hardware Description Languages (HDL) often requires more effort for the developers and might not be suitable in all scenarios, which makes high-level language-based software simulators for computer hardware attractive. Yet, choosing which simulation framework to use can be challenging due to the lack of comparative studies of high-level language-based simulators. This paper presents a comparative evaluation of state-of-the-art simulation frameworks that simulate computer hardware in high-level languages like C++. The contemporary simulators used in this study were selected from the 79 articles introducing novel AI accelerators referenced in our previous survey. We have identified six simulators that are suitable for AI accelerator evaluation, and provide a deeper analysis of three of them. © 2024 IEEE.

This study introduces a novel systematic approach to address the challenge of labeled data scarcity for fault detection and diagnosis (FDD) in District Heating (DH) systems. To replicate real-world DH fault scenarios, we have created a controlled laboratory emulation of a generic DH substation integrated with a climate chamber. Furthermore, we present an FDD pipeline using an isolation forest and a one-class support vector machine for fault detection alongside a random forest and a support vector machine for fault diagnosis. Our research analyzed the impact of data sampling frequencies on the FDD models, revealing that shorter intervals, such as 1-min and 5-min, significantly improve FDD performance. We provide detailed information on six scenarios, including normal operation, a minor valve leak, a valve leak, a stuck valve, a high heat curve, and a temperature sensor deviation. For each scenario, we present their signature, quantifying their unique behavior and providing deeper insights into the operational implications. The signatures suggest that, while variable, faults have a consistent pattern seen in the generic DH substation. While this work contributes directly to the DH field, our methodology also extends its applicability to a broader context where labeled data is scarce. © 2024 The Authors

We present a method, including tool support, for bibliometric mining of trends in large and dynamic research areas. The method is applied to the machine learning research area for the years 2013 to 2022. A total number of 398,782 documents from Scopus were analyzed. A taxonomy containing 26 research directions within machine learning was defined by four experts with the help of a Python program and existing taxonomies. The trends in terms of productivity, growth rate, and citations were analyzed for the research directions in the taxonomy. Our results show that the two directions, Applications and Algorithms, are the largest, and that the direction Convolutional Neural Networks is the one that grows the fastest and has the highest average number of citations per document. It also turns out that there is a clear correlation between the growth rate and the average number of citations per document, i.e., documents in fast-growing research directions have more citations. The trends for machine learning research in four geographic regions (North America, Europe, the BRICS countries, and The Rest of the World) were also analyzed. The number of documents during the time period considered is approximately the same for all regions. BRICS has the highest growth rate, and, on average, North America has the highest number of citations per document. Using our tool and method, we expect that one could perform a similar study in some other large and dynamic research area in a relatively short time.