Chromatin modification and higher-order chromosome structure play key roles in gene regulation, but their functional interplay in controlling gene expression is elusive. We have discovered the machinery and mechanism underlying the dynamic enrichment of histone modification H4K20me1 on hermaphrodite X chromosomes during C. elegans dosage compensation and demonstrated H4K20me1's pivotal role in regulating higher-order chromosome structure and X-chromosome-wide gene expression. The structure and the activity of the dosage compensation complex (DCC) subunit DPY-21 define a Jumonji demethylase subfamily that converts H4K20me2 to H4K20me1 in worms and mammals. Selective inactivation of demethylase activity eliminates H4K20me1 enrichment in somatic cells, elevates X-linked gene expression, reduces X chromosome compaction, and disrupts X chromosome conformation by diminishing the formation of topologically associating domains (TADs). Unexpectedly, DPY-21 also associates with autosomes of germ cells in a DCC-independent manner to enrich H4K20me1 and trigger chromosome compaction. Our findings demonstrate the direct link between chromatin modification and higher-order chromosome structure in long-range regulation of gene expression.