Bargain upgrades for your bike

Losing a few lbs of your body weight and riding more is the biggest improvement I've had, and it's free... Spending on shiny upgrades is much easier though!

Getting your position sorted is also a good one, you spend less time with post-ride aches and have the benefit of knowing you're getting the best from yourself.

Second hand SIDI shoes if they fit. Very hollow saddle (SMP IZM Cobb), for no perineum pressure to allow low aero position, set forward seat post for a more TT position ( not for everyone but I think KOPS may be a myth), Spinachi aerobars or other shortish tribars, Two budget 18650 LEDs strapped to bars using cable grip perhaps for winter evenings, Strava so everyday is a race day (but take care!), Ghostracer but take even more care.

Tyres, spend money on something decent like Open Corsas SC, much better than GP4000s, and you can get the for around £10 more than a pair of GP4000S closest you'll get to a tub on a clincher rim.

+1 on gp4000's. You can get a pair for 60 quid and in my mind they're easily the biggest bang for buck

Agreed on the tires.
Try new handlebar tape - feels like a new bike.
I recently got one of those gizmos (these blue plastic boxes with rotating brushes) that help you clean the chain properly - what a difference a properly cleaned chain makes!

Tyres. Ultremo, GP4000S or similar.

Well-cut clothing and a comfy saddle.

For some people a proper bikefit can make a huge difference.

Depending on the wheels you have, there's often room to upgrade here. But I wouldn't describe a nice wheelset a bargain.

Otherwise just ride it, that's what it's for. Carbon parts like stems and bottle cages are expensive and don't add to the experience.

Tyres are the best 'bang per buck' upgrade.
Try a set of GP4000's

Tyres made a nice difference to me, really improved speed and comfort.

Wheels, best upgrade there is. £4-450 will get you something that's awesome.
Don't neglect your tyres and tubes too - Getting a decent set of e.g. Conti GP4000s and using the race weight inner tubes saves you a couple of hundred grams at the rim of your wheels, where it affects you the most.