The cellular basis for site-specific inflammation remains unclear. In human fingers, proximal interphalangeal (PIP) joints are preferentially affected by inflammatory arthritis, whereas distal interphalangeal joints are spared, providing a model to investigate the predilection of inflammation to distinct sites. Here we combine single-cell RNA sequencing, imaging and X-ray tomography to examine cellular composition, spatial organization and structure of finger joints during fetal development. PIP joints had a larger synovial volume and were enriched for PI16+ 'universal' fibroblasts. These cells were located in perivascular regions and at developing tendon-ligament interfaces. PI16+ fibroblasts exhibited both a shared inflammatory and cell-type-specific response to cytokine stimulation, suggesting that the combination of their spatial location and transcriptional responses promote inflammation. We suggest that differences in the stoichiometry of mesenchymal cells established in utero, including the key role of PI16+ fibroblasts, is a general principle that drives inflammation susceptibility across tissues.