Clipless pedals/Wonky Feet

The reason I ask is that it sounds like your cleats are loose. Cleat misalignment will be more apparent on the side that sees more action, i.e. the side you clip in and out of when approaching traffic lights, etc.

Re-align the cleats and get some quality allen keys to torque up the fixing screws. Those fixing screws are prone to loosening off after a few rides. It's normal; the fixings/cleats compress the sole, leading to a slack fit.

do you clip out on the left-hand side first?
What type of pedals do you use?

if you're getting pain then yes, your foot is probably being held in the wrong position. the first thing to try is exactly what you've suggested: adjust the cleat so that your foot position is about the same as it would be when you're not clipped in.

it's possible you could 'train' your leg to adopt a straight position but depends on whether it's just a muscle imbalance or something more structural.