Aahh the eternal question of moving up before you can compete! Back in the old days when I first started racing there was no cat 5. I made to the 3's due to some crashes in crits. Once I arrived in the 3's I never finished another crit (got pulled every time). I wasn't a great cat 4 but circumstances got me to the 3's.
My feeling about moving up from 5 to 4 is you should at least be in the top 10 consistently in the 5's before moving up. Otherwise you are going to be off the back of a decent 4's field pretty quickly.