Don't rule out the other side. Sometimes a brake "hangs up" on application, causing a pull to the other side.
Loose wheel bearing, slack in a suspension bushing, contaminated pads, all can cause your symptoms. Clean everything up, reassemble, and check everything.