Take a close look at the spindle that comes with the new handle. The spindle has a notch at each end for the set screw to tighten on to. Make sure you are installing the handle set screw in the notch at each end of the spindle.
I know how to install it properly.
Set screws are both in the notched grove.
But it does not sty tight.
It comes loose after couple of times of use.
What can I do?
What I've done in this situation was to put a few drops of a liquid lock washer on the screw threads. This acts as a type of adhesive to keep the screw tight, but can still be removed it you need to take out the set screw in the near future. You should be able to find this at a local hardware store.