Travel time reliability is vital for both road agencies and road users. Expected travel time reliability can be used by road agencies to assess the state of a transportation system, and by road users, to schedule their trips. Road network deficiencies, such as insufficient traffic flow capacity of a road segment or poor road design, have a negative impact on the reliability of travel times. Thus, to maintain robust and reliable travel times, the detection of road network deficiencies is vital. By continuously analyzing travel times and using appropriate travel time reliability measurements, it is possible to detect existing deficiencies or deficiencies that may eventually occur unless necessary actions are taken. In many cases, indices and measurements of travel time reliability are related to the distribution of the travel times, specifically the skewness and width of the distribution. The current paper introduces a median-based misery index for travel time reliability. The index is robust and handles travel times that follow a skewed distribution well. The index measures the relative difference between the slow travel speeds and the free-flow travel speed. The index is inspired by the median absolute deviation, and its primary application is to detect routes or road segments with potential road network deficiencies. To demonstrate the applicability of the index, we conducted an empirical case study using real travel speed data from the European route E4 in Sweden. The results from the empirical case study indicate that the index is capable of detecting road segments with slow travel speeds regardless of the travel speed distribution.