Its not legal just like its technically not legal to take a bike out of the trash that someone threw out.
I dont think anyone would have any real beef about it seeing how you described it though, maybe you can take a photo of the bike should you decide to take it, that way if someone down the road is like "hey thats my bike!" you can show em the photo and say you were sure it was abandoned. and offer to give it back if they want it of course (minus your new wheels!). Thats probably what I would do anyways.