I'll skip the portion about possible surgical repair/cleanup and just address the "easy" potential fixes...

- Make sure your bike is setup right to minimize any further damage to your knee. For many people this means a medical bike fit to ensure a safer setup.
- If the arthritis includes a noticeable decline in synovial fluid you can actually get injections of artificial fluid into the joint to help smooth functioning and lessen pain.