This business about the average speed goldfinch is capable of is absolute BS. For a start, short randonnees have an average speed of 15km/h which works out to around 9.6mph. The really long ones have even lower averages. Admittedly, that includes everything such as stops for food and eating. But then, an event like the one being discussed is similar.
Those averages do not prevent people from riding them, especially the short ones (50, 100 and 150km) and as far as I am concerned, any ride organiser should use that average speed as a guide on which to organise their rides for the slowest participant.
We are not talking about a race. This is just an organised ride. Yes, there are going to be participants who tear off into the distance at 20 or 25mph. All power to them (literally and figuratively). But I will remind goldfinch that one of the secrets behind successful randonneuring is: Ride your own ride.
The only thing is, you might need to be happy in your own company. You might indeed be the slowest on the ride; then again, you might not be, and you could easily hook up with someone else who rides about your own pace, and you could have delightful days on the road that pass quickly because of the conversations you have.
It's also completely different being part of an event, and doing practice rides by yourself.
I definitely vote to stick with it. Get out there and ride. As I said before, do back to back rides at distances you feel you can currently handle, and ramp them up by 5 or 10 miles every week between now and then.