There can be a lot of strain on the E-brake cable proper if they're strung through an eye bolt resulting in a sharp bend.

Better is to make up a short length of tubing, flared at each end or at least smoothed over where the cable enters and leaves.
Bend the tubing a bit after you make it so as to put a very gentle bend into the cable spreading the bending load over several inches.

I've seen simple open-faced shoes made from 1/8" or so, a curve bent into it for the cable to feed across.
These shoes also have one or two tabs for bolting them down.

Depending on the situation you may be able to connect a light spring from one cable to the other so that there is tension on the cable, but not too much.
Alternatively, a couple light springs pulling each cable outward individually.

A little tension will go a long way toward getting rid of cable slap.