The dynamics of inheritance of histones and their associated modifications across cell divisions can have major consequences on maintenance of the cellular epigenomic state. Recent experiments contradict the long-held notion that histone inheritance during replication is always local, suggesting that active and repressed regions of the genome exhibit fundamentally different histone dynamics independent of transcription-coupled turnover. Here we develop a stochastic model of histone dynamics at the replication fork and demonstrate that differential diffusivity of histones in active versus repressed chromatin is sufficient to quantitatively explain these recent experiments. Further, we use the model to predict patterns in histone mark similarity between pairs of genomic loci that should be developed as a result of diffusion, but cannot originate from either PRC2 mediated mark spreading or transcriptional processes. Interestingly, using a combination of CHIP-seq, replication timing and Hi-C datasets we demonstrate that all the computationally predicted patterns are consistently observed for both active and repressive histone marks in two different cell lines. While direct evidence for histone diffusion remains controversial, our results suggest that dislodged histones in euchromatin and facultative heterochromatin may exhibit some level of diffusion within "Diffusion-Accessible-Domains" (DADs), leading to redistribution of epigenetic marks within and across chromosomes. Preservation of the epigenomic state across cell divisions therefore might be achieved not by passing on strict positional information of histone marks, but by maintaining the marks in somewhat larger DADs of the genome.