Modern wireless systems and standards increasingly rely on OFDM for high-throughput communications. However, these systems are often highly vulnerable to selective jamming attacks, particularly when a jammer targets (part of) the known frame preamble. In this paper, we consider one of the most disruptive jamming attacks against the preamble-based frequency offset (FO) estimation in IEEE 802.11a/n/ac/ax systems and develop four techniques to mitigate this attack. Two of these techniques are based on randomly changing the first half of the standard frame preamble at the transmitter while maintaining its backward compatibility with legacy receivers. Specifically, we design a set of new preamble waveforms that satisfy the expected characteristics of a preamble in 802.11 systems. The other two techniques take a receiver-based approach and exploit the parts of the preamble that are not under attack to estimate the FO. We conduct extensive simulations and illustrative USRP experiments to study the effectiveness of these countermeasures.