4D computed tomography (4D-CT) is an important modality in medical imaging due to its ability to resolve patient anatomy motion in each respiratory phase. Conventionally 4D-CT is accomplished by performing the reconstruction for each phase independently as in a CT reconstruction problem. We propose a new 4D-CT reconstruction algorithm that explicitly takes into account the temporal regularization in a non-local fashion. By imposing a regularization of a temporal non-local means (TNLM) form, 4D-CT images at all phases can be reconstructed simultaneously based on extremely under-sampled x-ray projections. Our algorithm is validated in one digital NCAT thorax phantom and two real patient cases. It is found that our TNLM algorithm is capable of reconstructing the 4D-CT images with great accuracy. The experiments also show that our approach outperforms standard 4D-CT reconstruction methods with spatial regularization of total variation or tight frames.