For night riding, you'll need lights.
For a road ride headlight, you don't need a super high-lumen wide-beam light (like some MTB trail lights), just something that illuminates a narrow beam well. I really like my Cycle Torch Night Owl for this purpose; compact, simple, hours of battery life even when bright enough to be comfortable for speedy road bike riding.
You'll also want a rear blinky light. Think about maybe using it during the day, too.

Get a bottle cage. Aesthetically, you'll want to either match the component color (usually either black or metallic silver) or the trim color (i.e. what's used for frame lettering and/or paint on the inside of the wheel rims).
Also have a water bottle. Aesthetically, you'll either want the bottle to match the frame's primary paint color, or go with a transparent bottle. If you go with a transparent bottle, consider matching beverage color to the frame's primary paint color.

Glasses. Definitely have sunglasses for day riding; sunburning your eyeballs isn't pleasant. It's not a terrible idea to also have clear glasses for night riding, because taking a moth to an eyeball also isn't pleasant.

You'll want a basic mechanical toolkit. I like to use a saddle bag.
-At least one spare inner tube.
-At least two tire levers.
-A patch kit. At the very least some glueless patches (glorified stickers), they suck but they add almost no weight and take up almost no room.
-A basic multi-tool.
You'll also want some way to inflate your tires. I mostly use pumps. If you go with a mini-pump, they usually come with mounting brackets that can fit underneath a bottle cage, so a single set of bottle cage bosses can hold both a pump and a water bottle.
Some bike shops carry starter kits with all of the above, like the Bontrager Flat Pack.
You'll also want to know how to actually use the stuff in your kit, particularly how to fix a flat tire.

It's also nice to have a floor pump with a pressure gauge at home.