The showpiece of cycling in Victoria, BC is the Galloping Goose bike trail. It's a nice trail, quite fast and convenient where it is paved, but I find it highly overrated due to its lack of night-time lighting and to the fact that most of it is unpaved. The section that runs from the BC Ferries terminal at Swartz Bay to the Victoria city limit is called the Lochside Trail. It's the flattest way to get between the ferry and downtown Victoria, but it contains a number of significant unpaved sections. Avoiding the gravel adds maybe 4 kilometers and 10 minutes to the ride (or adds a lot of climbing and riding along the main highway), but I would always avoid the gravel. All of the regular club rides that went along the Lochside would also assiduously avoid the gravel.
I don't like riding on gravel because it gets the bike dirty! I like having a clean bike.
L.