I have seen such a book, in Icelandic. I haven't really looked for it in English, but I would think it must have been translated. There are quite a few locations known around the country which have geothermal pools suitable for bathing; some are very popular, others are hardly known. Some are well accessible, others would require you to go far off the beaten track.
Magnus Thor
Iceland