A coarse-grained model has been built to study the effect of the interfacial interaction between spherical filler particles and polymer on the mechanical properties of polymer nanocomposites. The polymer is modeled as bead-spring chains, and nano-fillers grafted with coupling agent are embedded into the polymer matrix. The potential parameters for polymer and filler are optimized to maximally match styrene-butadiene rubber reinforced with silica particles. The results indicated that, to play a noticeable role in mechanical reinforcement, a critical value exists for the grafting density of the filler–polymer coupling agent. After reaching the critical value, the increase of grafting density can substantially enhance mechanical properties. It is also observed that the increase of grafting density does not necessarily increase the amount of independent polymer chains connected to fillers. Instead, a significant amount of increased grafting sites serve to further strengthen already connected polymer and filler, indicating that mechanical reinforcement can occur through the locally strengthened confinement at the filler–polymer interface. These understandings based on microstructure visualization shed light on the development of new filler polymer interfaces with better mechanical properties.