Objective: This study introduces an innovative end-to-end deep learning pipeline designed to automatically classify and order fetal ultrasound standard planes in alignment with the guidelines of the Canadian Association of Radiologists, while also assessing the diagnostic usability of each view. The primary objective is to address the manual and cumbersome challenges that interpreting radiologists encounter in the existing obstetric ultrasound workflow. Methods: We compiled a diverse dataset, comprising 33,561 de-identified two-dimensional obstetrical ultrasound images acquired from January 1, 2010, to June 1, 2020. This dataset was categorized into 19 distinct classes associated with standard planes and further partitioned into training, validation, and testing subsets via a 60:20:20 stratified split. The standard plane and diagnostic usability networks are founded on a convolutional neural network framework and employ the benefits of transfer learning. Results: The standard plane classification network demonstrated promising results by achieving 99.4 % and 98.7 % for accuracy and F1 score, respectively. Subsequently, the diagnostic usability network demonstrated strong performance, registering 80 % accuracy and an 82 % F1 score. Notably, this study is the first to investigate whether deep learning methods can surpass sonographers in the standard plane labeling task, with some instances revealing the algorithm’s capacity to rectify sonographer mislabeled planes. Conclusion: The results highlight the algorithm’s potential to be integrated into a clinical setting by serving as a reliable assistive tool, alleviating the cognitive workload faced by radiologists and enhancing efficiency and diagnostic outcomes in the current obstetric ultrasound process.