Glass-box Automated Driving: Insights and Future Trends

Challenges

Without any doubt, full black-box models have demonstrated compelling capabilities in controlling complex systems such as autonomous vehicles. This achievement cannot be neglected, as it represents a significant opportunity to address long-term practical problems. However, glass-box models offer the opportunity to debug and refine these systems by providing full control and transparency. This allows for a deeper understanding and improvement of each system’s behavior. 

The first significant challenge in building a glass-box control system for autonomous driving lies in managing the complexity of the environment. As previously mentioned, the challenge does not stem from driving along a predefined path at a precise velocity but rather from interacting effectively with the surrounding environment. Autonomous driving requires processing vast amounts of real-time data from sensors such as cameras, LiDAR, radar, and GPS. The decision-making process entails intricate interactions between perception, prediction, and control systems, which are often modeled using complex deep learning or neural networks—methods that are inherently difficult to interpret. Figure~\ref{fig:control} illustrates the control flow from low-level to high-level driving models.The concept is that any state-of-the-art controller, from PID to Lyapunov-based controllers, can effectively and precisely manage vehicle speed and steering angle, provided a simplified physical model of the vehicle is known. The assumptions required for these controllers to function are often unrealistic and ineffective at predicting dynamic environmental changes. The higher the complexity of the vehicle model, the better and more effective the controller. This capability is sufficient for driving in structured environments. The system’s internal components, highlighted with a dashed line in Fig.~\ref{fig:control}, can be fully explainable and interpretable PIDs. Literature offers a robust foundation of analytical analyses and formal solutions for the problem of following reference signals, enabling deterministic control in these scenarios.

As the research community approached this problem, it quickly became evident that addressing it required multiple levels of abstraction. At the very low level, classical automatic controllers, such as PID or model predictive controllers, perform their tasks effectively. However, tackling vehicle perception and routing with the same level of detail as low-level control, represented in the outer cycle of  Fig.~\ref{fig:control}, is computationally and practically infeasible.

The routing problem itself can be subdivided into two distinct components: the generation of kinematically or dynamically feasible trajectories for short-range vehicle control, and waypoint generation for high-level vehicle routing. The former requires solving differential equations, which are computationally demanding and often limited to localized regions. The latter typically employs simpler, non-physically compliant algorithms, such as Dijkstra’s algorithm or rapidly-exploring random trees (RRT). Attempting to solve differential equations for every point along a long path is not only computationally prohibitive but also ineffective, as it assumes a static environment, thereby reducing the system's ability to adapt to dynamic changes. Conversely, ignoring differential equations altogether can result in unfeasible paths that violate the vehicle's physical constraints. 

The routing problem exemplifies the necessity for different levels of abstraction to address complex challenges in autonomous driving. Each level retains its own interpretability: long-range planning may sacrifice physical precision but provides computational efficiency, while short-range trajectory generation maintains a detailed physical interpretation. This balance allows the system to adapt to a dynamic environment while ensuring feasibility at a local level.

On a different level, the perception problem, which includes tasks such as object detection and segmentation, is predominantly addressed using black-box models. On one hand, these models excel in providing detection capabilities that can even surpass human performance in certain scenarios, such as low-light conditions or when integrating data from multiple sensor sources. An illustrative example is shown in the two images in Fig.~\ref{fig:control} which are extracted from the IseAuto dataset and include the addition of a black-box segmentation result. 

On the other hand, they lack an understanding of the semantic meaning of objects within a scene, treating a tree and a person merely as objects without recognizing their distinct roles or contextual importance. Contextual importance is inherently challenging to capture, as even human observers may assign varying interpretations and significance to similar objects in a scene. An interpretable and explainable model should not aim to produce verbose descriptions of a scene but rather to classify objects accurately while accounting for their high-level roles and importance. For instance, in such a framework, errors or flaws in detecting vehicles or pedestrians, compared to non-critical objects like rocks or foliage, could be identified, debugged, and addressed in a targeted manner.