We calculate magnetic field fluctuations above a conductor with a nonlocal response (spatial dispersion) and consider a large range of distances. The cross-over from ballistic to diffusive charge transport leads to reduced noise spectrum at distances below the electronic mean free path, as compared to a local description. We also find that the mean free path provides a lower limit to the correlation (coherence) length of the near field fluctuations. The short-distance behavior is common to a wide range of materials, covering also semiconductors and superconductors. Our discussion is aimed at atom chip experiments where spin-flip transitions give access to material properties with mesoscopic spatial resolution. The results also hint at fundamental limits to the coherent operation of miniaturized atom traps and matter wave interferometers.