Step junctions are often present in layered materials, i.e. where single-layer regions meet multilayer regions, yet their effect on thermal transport is not understood to date. Here, we measure heat flow across graphene junctions (GJs) from monolayer-to-bilayer graphene, as well as bilayer to four-layer graphene for the first time, in both heat flow directions. The thermal conductance of the monolayer-bilayer GJ ranges from ∼0.5 to 9.1 × 108 W m-2 K-1 between 50 K to 300 K. Atomistic simulations of such a GJ device reveal that graphene layers are relatively decoupled, and the low thermal conductance of the device is determined by the resistance between the two distinct graphene layers. In these conditions the junction plays a negligible effect. To prove that the decoupling between layers controls thermal transport in the junction, the heat flow in both directions was measured, showing no evidence of thermal asymmetry or rectification, within experimental error bars. For large-area graphene applications, this signifies that small bilayer (or multilayer) islands have little or no contribution to overall thermal transport.