Im already using lerp to increase speed but It slows down too much when camera gets closer to player & too fast when camera gets further, there should a way to adjust this?
How to increase speed of camera following, as camera gets closer to the player, beyond lerp setting?
That’s exactly what lerp is for.
It’s math, so… adjust it as you like. 
Maybe you’ll be interested in tweens: