A groundbreaking AI system called OmniPredict can predict pedestrian behavior before it happens — a leap from detection to anticipation that could dramatically reduce traffic accidents and reshape autonomous vehicle safety.
You’re standing at a busy intersection. A car slows as you approach the curb. No driver looks back at you. Instead, software inside the vehicle is deciding what you are about to do next. That moment captures the goal behind OmniPredict, a new artificial intelligence system designed to anticipate pedestrian behavior before it happens — not react after movement begins.
Developed by Dr. Srinkanth Saripalli and his team at Texas A&M University’s College of Engineering alongside researchers at Korea Advanced Institute of Science and Technology, OmniPredict leverages GPT-4o — the same multimodal large language model powering advanced chatbots — to reason across images, text, and structured data. Unlike older systems that merely track motion, this AI interprets intent, offering vehicles a proactive edge in predicting whether pedestrians will cross streets, look toward cars, or remain stationary.
The research, published in Computers & Electrical Engineering, represents one of the first efforts to use such models for real-time pedestrian forecasting. “Cities are unpredictable. Pedestrians can be unpredictable,” said Saripalli. “Our new model is a glimpse into a future where machines don’t just see what’s happening, they anticipate what humans are likely to do, too.”
OmniPredict ingests four types of input simultaneously: scene context images, local context images of pedestrians, bounding box data describing position and size, and ego-vehicle speed. By analyzing sixteen past video frames, it predicts pedestrian actions approximately one second ahead — enabling vehicles to adjust their behavior preemptively rather than reactively.
The model was trained to identify four key behaviors: whether a pedestrian would cross, if they were walking or standing still, if they were partially hidden, and if they were looking toward the vehicle. Spatial cues were converted into textual prompts so the model could reason about them like written instructions — forcing structured outputs instead of vague explanations.
Unlike traditional neural networks reliant on memory states, OmniPredict evaluates all inputs together using attention mechanisms that weigh subtle cues — hesitation, body orientation changes — before making decisions. This allows it to respond faster and generalize better across different environments, critical traits for real-world deployment.
The model was tested against two widely respected datasets in pedestrian behavior research: JAAD (over 82,000 annotated frames) and WiDEVIEW (recorded on university campuses). Without any task-specific training, OmniPredict achieved 67 percent accuracy in predicting whether pedestrians would cross — surpassing existing models by roughly 10 percent and securing the highest area-under-the-curve score among all tested models.
While some supervised systems achieved higher detection rates, they required extensive training and fine-tuning. OmniPredict matched or surpassed their accuracy without additional overhead. Even when challenged with partially hidden pedestrians or people looking directly at vehicles, its performance remained strong — suggesting robustness under real-world conditions.
Performance depended heavily on visibility. Smaller figures in the frame reduced accuracy because less visual detail was available. As pedestrians occupied more of the image, predictions improved significantly. Ablation tests revealed that removing global scene context caused the largest drop in performance — underscoring how crucial understanding environmental context is for accurate prediction.
In challenging scenes — snow, wet pavement, multiple pedestrians — OmniPredict often succeeded where other models failed. It detected early movement cues and head orientation signals that indicated intent. However, heavy shadows, severe occlusion, or cyclists sometimes led to incorrect predictions. Researchers note separating pedestrians from similar road users remains difficult without explicit labeling.
The implications extend beyond road safety. In military or emergency settings, the ability to read posture, stress signals, or hesitation could offer earlier warnings and improve situational awareness. “We are opening the door for exciting applications,” Saripalli said. “For instance, the possibility of a machine to capably detect, recognize and predict outcomes of a person displaying threatening cues could have important implications.”
Crucially, OmniPredict emphasizes interpretability. When asked to explain its decisions, it often provided clear reasoning tied to movement patterns and environmental context — essential for building trust in safety-focused AI systems.
This research marks a fundamental shift: from reactive to anticipatory AI. Autonomous systems may soon reduce accidents and improve traffic flow by understanding human motives — not just motions. For developers, it demonstrates that general-purpose multimodal models can rival specialized systems without costly retraining. This approach could lower barriers for deploying advanced safety tools across cities and influence how AI assists humans in high-risk scenarios — from disaster response to security operations.
Research findings are available online in the journal Computers & Electrical Engineering.
Stay ahead of the curve. Get the fastest, most authoritative analysis of breaking technology news delivered straight to your inbox. Subscribe to onlytrustedinfo.com’s daily newsletter today.
