We present a formalism based on first principles of quantum electrodynamics at nonzero temperature which permits to calculate the Casimir-Polder interaction between an atom and a graphene sheet with arbitrary mass gap and chemical potential, including graphene-coated substrates. The free energy and force of the Casimir-Polder interaction are expressed via the polarization tensor of graphene in (2+1)-dimensional space-time in the framework of the Dirac model. The obtained expressions are used to investigate the influence of the chemical potential of graphene on the Casimir-Polder interaction. Computations are performed for an atom of metastable helium interacting with either a free-standing graphene sheet or a graphene-coated substrate made of amorphous silica. It is shown that the impacts of the nonzero chemical potential and the mass gap on the Casimir-Polder interaction are in opposite directions by increasing and decreasing the magnitudes of the free energy and force, respectively. It turns out, however, that the temperature-dependent part of the Casimir-Polder interaction is decreased by a nonzero chemical potential, whereas the mass gap increases it compared to the case of undoped, gapless graphene. The physical explanation for these effects is provided. Numerical computations of the Casimir-Polder interaction are performed at various temperatures and atom-graphene separations.