It can be done. A race bike of the 1960s years ago was not too different in terms of weight, to a modern touring bike.
You can time trial on any touring bike. A midweight club-style tourer such as a Trek 520 is better than an expedition style one.
If you want to go lighter, then a light-touring bike will be more agile but not as good for loaded touring.
Fit some 25mm tyres and an aero bar. A set of lighter racing wheels will be better and you need to ensure you have higher gearing than suitable for loaded touring. You may want to get a small triple chainset and have a set of larger rings for your racing, or a replacement chainset.