atAt server:
$ ls -lh /home
$ sudo chmod 700 /home/$USER
It was directory permission issue, it. It was 777 at server, so I changed it back to 700;700 this. This fixed my issue with ssh password less login failure even after copying $USER/.ssh/id_rsa.pub toto server $USER/.ssh/authorized_keys.