The manufacturers generally go to 36 or 37 (3-3.5 mens). Some will have a few 35 offereings.
There are a few manufacturers that have some 34's (Vittoria, Northwave), but I think they also advertise those as size 3 equivalent. I did find one website that specializes in kids outdoor gear and advertised these kids shoes from Answer at size 32
(mens 2). SixSixOne is another shoe manufacturer that may have smaller sizes, but 33 seems to be lower limit.
+1 on the bungee or between the pedals as the method to keep the toes up.
We are using toeclips with a hard plastic strap. We do not bother to cinch them down, they give a little extra hold from slipping off and it keeps their feet better positioned on the pedal. It has worked fine.
Let us know if you find something better.