Phase-locked electromagnetic transients in the terahertz (THz) spectral domain have become a unique contact-free probe of the femtosecond dynamics of low-energy excitations in semiconductors. Access to their nonlinear response, however, has been limited by a shortage of sufficiently intense THz emitters. Here we introduce a novel high-field source for THz transients featuring peak amplitudes of up to 108 MV/cm. This facility allows us to explore the non-perturbative response of semiconductors to intense fields tailored with sub-cycle precision. In a first experiment intense transients drive Rabi-oscillations between excitonic states in Cu2O, implying exciting perspectives for future THz quantum optics. At electric fields beyond 10 MV/cm, we observe the breakdown of the power expansion of the nonlinear polarization in bulk semiconductors. Furthermore, we employ the intense magnetic field components of our transients to coherently control spin waves in antiferromagnetically ordered solids. Finally, intersubband cavity polaritons in semiconductor microcavities are exploited to push light-matter coupling to an unprecedented ultrastrong and sub-cycle regime.