Even Nephi, a prophet, encountered such setbacks. He recognized the joy he felt from the spirit, so much that he says his "soul delighteth in the things of the Lord." (2 Nephi 4:16) But his "heart exclaimeth...because of the temptations and the sins that do so easily beset [him]." (2 Nephi 4:17-18)
A prophet even recognizes that as much as he loves the spirit and the Lord, he sometimes falters and has to work to once again progress and have the spirit with him. He cries out "Awake, my soul! No longer droop in sin. Rejoice, O my heart, and give place no more for the enemy of my soul." (2 Nephi 4:28)
So when we make a mistake, we can't give up or get disheartened. The key is to pick ourselves back up, with a new determination to serve God and be worthy of His Spirit. Making our peaks a little higher than our last peak, and then when we falter, not falling as far as we did before. That will be the key to our progression in returning to live with our Father in Heaven.