The kind of atheism that the psalmist decries here is the practical (or impractical) atheism that outwardly admits God’s existence, but refuses to call on him when times get bad. The psalmist calls on his people to rejoice in the God who restores, to be glad in the process, before the restoration happens. Otherwise, God’s people will act like others, allowing corruption and evil to continue, and excusing it out of fear.
LORD, by faith we accept your rescue. We choose to believe in your existence and help.
Psalm 53