The pursuit of high fuel efficiency and low emissions has inspired a lot of research efforts on automotive powertrain hybridization. Targeted at developing a real-time hybrid energy management strategy, a stochastic dynamic programming extremum seeking (SDP-ES) optimization algorithm with both the system states and output feedback is investigated in this paper. This SDP-ES algorithm utilizes a state-feedback control, which is offline generated by the stochastic dynamic programming (SDP), as a reference term to ensure the approximate global energy optimality and battery state-of-charge (SOC) sustainability. And in real-time, this algorithm injects a local feedback term via extremum seeking (ES), which is a non-model-based nonlinear optimization method, to compensate the control commands from the SDP and generate more fuel-efficient operation points along the specific SOC sustaining line, by leveraging the real-time measurement of system outputs (fuel consumption and emissions). The simulation results show the SDP-ES algorithm can provide desirable improvement of fuel economy based on the original SDP.