The brain operates like a dynamic 3d Ising spin-glass coupled to memory. Memory induces non-equilibrium criticality in the spin-glass regardless of the effective temperature or the underlying frustration profile. Note this is not a feature of the equilibrium model, where frustration dictates the critical behavior. We extend the memory dynamics for optimization problems on multiple graph structures such as the bipartite graph (RBM) and the random 3-hypergraph (3-SAT). This has direct applications to unsupervised learning and factorization. We conclude by showing rigorously several dynamical features of the memory dynamics, including the absence of periodic orbits and dissipativity.