You are right. The mechanical part of LED lights is often harder than the electronic part. I forgot that you started the idea on the vintage forum so I see your point on the looks. But then my question is : Do you really need the stand light function? Don't get me wrong, I'm not trying to tell you what to do, but the point is that without that your circuit can become really simple and small. For the heat sink there are different solutions. You can use thermal glue and depending on which led you use, you may not need a big heat sink anyway. (I don't know by heart which leds that would be. I have read through many websites over the last months and some DIY projects out there mention that the need for cooling is very limited.)

The typical supercap is 1F at 5.5V and cost a few $. But I have not seen many succesful DIY, if any. The general idea is that if you need a stand light it is easier to add a small battery light. Don't know if that is an option for you. I'm just trying to help with ideas.