If I had to keep the price down really low, the following items comes in at $42.77.

This flashlight
These batteries
This charger
This handlebar mount
This taillight

I'd actually get two of the taillight. That light is very similar to the Planet Bike Superflash. I have one and it's held up well for me. I feel that I want TWO taillights because you never know when one has failed unless you have eyes in the back of your head.

This setup is nearly identical to what I have except I have a P7 flashlight by MTE which is about twice as much money (and is about twice as bright) but the flashlight I linked to above is quite good for the price, about as bright as a Dinotte 200L (for 6x the price).

This is a very good setup for < $50, I don't think you can really beat it.

The caveat is that this WILL take 3 to 4 weeks to arrive from China. You can get this stuff from closer sources but it will drive the price up into the $80 to $100 range for the whole package.