While Vertical-External-Cavity-Surface-Emitting-Lasers (VECSELs) have been successfully used as ultrafast laser sources with pulse durations in the hundreds of femtosecond regime, the dynamics within the semiconductor gain structure are not yet completely understood. With the high carrier densities inside the semiconductor, nonequilibrium effects such as kinetic-hole burning are expected to play a major role in pulse formation dynamics. Moreover, the nonlinear phase change by the intense light field can induce a complex dispersion, which may potentially limit the achievable pulse durations. To shed light on such nonequilibrium dynamics, we perform in-situ characterization of mode-locked VECSELs. We probe the gain media as well as the intracavity absorber with a femtosecond fiber laser source. For measuring temporal characteristics, we employ an asynchronous optical sampling technique by phase-locking the repetition rate of the VECSEL to a multiple of the probe laser with an adjustable offset frequency. This allows for probing dynamics from femtosecond to nanosecond time scales with scan rates up to hundreds of Hertz without compromise of measurement precision which can be introduced by mechanical delays covering such large temporal windows. With a resolution in the femtosecond range, we characterize gain depletion by the intracavity pulse as well as the gain recovery timescales for different power levels and operation regimes.