Never heard of huge chunks of gravel on that arm of the trail. Are you familiar with the feeder arm from the Trenton Area to just south of Milford, NJ? It goes along the Delaware River and is primarily crushed limetine. I rode the entire stretch between Lambertville and the end near Milford on Easter weekend. Totally fine. You could probably even do it with 23c tires. I have in the past. I don't recommend riding north of Frenchtown because the trail abruptly end in the middle of nowhere before you get to Milford. (There is a environmental clean up site between the trail's abrupt end and the town of Milford.)
On nice feature of the feeder arm is that it passes through Lambertville, Stockton and Frenchtown, where you can get food and drink. It also passes by Bull's Island Recreation Area, where you will find water, bathrooms, a large pinic area and a boat ramp where you can take a dip in the river. There is also a pedestrian bridge to PA there with a nice view of the river. At the PA side of that bridge is the Black Bass Hotel. It serves lunch and has outdoor seating overlooking the river. Depending on how far you want to ride, you could start at the NJ side of Washington's Crossing or in L'Ville, head north, then ride back. Shorter drive than Bound Brook.