Optimizing Production Engineering: Data Science and ML Solutions for Scalable Data Pipelines in Supply Chain Software

Abstract

In the era of Industry 4.0, optimizing production engineering through intelligent systems has become a strategic priority for supply chain-driven industries. This study investigates the integration of Data Science and Machine Learning (ML) solutions within scalable data pipelines to enhance production performance and decision-making in supply chain software platforms. A hybrid methodology was employed, combining real-time data pipeline engineering using Apache Kafka and Airflow with predictive modeling through algorithms such as Random Forest, XGBoost, ARIMA, and Prophet. Empirical analysis was conducted across multiple industrial case studies, evaluating the system on key performance indicators (KPIs) such as production throughput, machine downtime, and inventory turnover. The results revealed notable improvements in operational accuracy, with Prophet outperforming ARIMA in demand forecasting and Random Forest achieving 92.4% accuracy in equipment failure prediction. Scalable data pipelines ensured high throughput and low latency, supporting seamless real-time ML deployment. Statistical analysis confirmed the significance of performance gains, with production efficiency increasing by 9.3% and forecast error decreasing by over 38%. This study provides a practical, data-driven framework for optimizing production workflows and establishes a foundation for AI-enabled supply chain transformation. The findings highlight the critical role of ML and data engineering in advancing modern production systems and driving digital resilience in industrial operations.