A search is presented for the pair production of heavy vectorlike quarks, TT or BB, that decay into final states with jets and no reconstructed leptons. Jets in the final state are classified using a deep neural network as arising from hadronically decaying W/Z bosons, Higgs bosons, top quarks, or background. The analysis uses data from the ATLAS experiment corresponding to 36.1 fb-1 of proton-proton collisions with a center-of-mass energy of s=13 TeV delivered by the Large Hadron Collider in 2015 and 2016. No significant deviation from the Standard Model expectation is observed. Results are interpreted assuming the vectorlike quarks decay into a Standard Model boson and a third-generation-quark, T→Wb,Ht,Zt or B→Wt,Hb,Zb, for a variety of branching ratios. At 95% confidence level, the observed (expected) lower limit on the vectorlike B-quark mass for a weak-isospin doublet (B, Y) is 950 (890) GeV, and the lower limits on the masses for the pure decays B→Hb and T→Ht, where these results are strongest, are 1010 (970) GeV and 1010 (1010) GeV, respectively.