I lived in Victoria for many years. The closest mtb park from downtown Victoria (about 15 km) is a place called the Dump (or Hartland). There are very nice and technical trails. If you are into touring there is more than 100 km long railroad trails. Those trails are also perfect for training. There are also good trails at Mt. Tzouhalem close to Duncan.

These are some trail pictures that I posted last year.

And here you can find lots of information.