The intersection of econometrics, statistics, and corporate finance has led to significant advancements in algorithmic trading in the cryptocurrency markets. While machine learning has been widely applied to predict price movements, most research focus solely on directional classification without considering the possibility to use the prediction confidence score of the models as a mean to increase the accuracy of the predictions further. This thesis investigates whether incorporating the confidence scores from machine learning classifiers can improve the prospect of positive trading outcomes in algorithmic cryptocurrency trading. As a result, it contributes to both academic understanding of market efficiency and serve as a basis for future algorithmic trading strategies. We employ an ensemble of two Random Forest models to predict the directional movement of cryptocurrency prices, increasing the accuracy by filtering out low-confidence predictions. The approach is benchmarked against Chen et al. (2020) using the same historical Bitcoin market dataset and further validated on ten cryptocurrencies from January 2022 to December 2023 with 5-minute trading data samples from the ByBit exchange. The results demonstrate that strategies incorporating confidence thresholds yield higher prediction accuracy than those that do not, though they produce fewer trade signals. Our method outperforms the top results achieved in the benchmark study with an accuracy of 72%, while maintaining 44% of the output samples, and with further increasing levels of accuracy for higher confidence score thresholds. Thus, indicating that confidence-based filtering can enhance both prediction accuracy and in extension also trading performance. In conclusion, combining two Random Forest models in an ensemble and incorporating confidence score thresholds significantly improve the accuracy of cryptocurrency price movement predictions compared to individual models. The method outperforms previous research for higher confidence thresholds and provides algorithmic traders with a tool to further increase prediction accuracy, however at the cost of decreasing number of trading opportunities for increasing accuracy. It thus demonstrates a shortcoming in the Efficient Market Hypothesis (Fama, 1970) and provides support for a more evolutionary approach to understanding markets – especially those that are less mature – such as the Adaptive Market Hypothesis (Lo, 2004). Future research could extend this work by investigating other prediction windows to further increase accuracy and output size detention, and by extending the method to also predict the level of magnitude of price changes, enabling more advanced trading strategies and enhanced risk management.