Given an arbitrary directed graph $E$ and field $K$, we study the partially ordered set $Spec(L_K(E))$ of prime ideals of the corresponding Leavitt path algebra $L_K(E)$, partially ordered by set inclusion. We work towards classifying which posets appear in this way, for general graphs $E$, row-finite graphs $E$, and finite graphs $E$. By considering special sets of vertices (namely, the maximal tails) we are able to determine the structure of the prime ideals, and prove that every countable poset that has the DCC and locally has the ACC is both of the form $Spec(L_K(E))$ and $Spec_\gamma(L_K(E))$ for some countable, row-finite graph $E$ and for an arbitrary field $K$, where $Spec_\gamma(L_K(E))$ is the collection of prime, graded ideals of $L_K(E)$. If we only look at the finite graphs $E$, by doing explicit constructions we find that the posets that arise for $Spec_\gamma(L_K(E))$ are precisely the finite posets, the posets that arise for $Spec(L_K(E))$ are those finite posets, with sets of infinite cardinality $max\{\aleph_0,|K|\}$ inserted at arbitrary locations throughout the poset, and that the posets that arise for $Spec(L_K(E))\backslash Spec_\gamma(L_K(E))$ look like the finite posets, except that each point is replaced with infinitely many non-comparable ones.