With a few exceptions, such as Dia Compe BRS brakes, all the return force comes from the caliper's return spring. So sluggish return can usually traced back to either something in the caliper, or most commonly cable friction.

You can eliminate the caliper quick enough by squeezing it closed with your hand around the shoes, then letting go. Since this doesn't involve the cable, it should snap back open. If it doesn't then there's friction within the caliper or a damaged/weak spring. OTOH of the caliper works OK, but there's still sluggish return when using the lever, then odds favor cable friction as the cause.

Of course there could also be a damaged lever or friction within the lever, but this is the least likely cause.