The bottom line answer to your question is no, lust is not grounds for divorce.
If you look at the context of the Lord Jesus' remarks about lust in Matthew 5, what you see is that He is "pulling back the rug," so to speak, on outward sins to expose the underlying problem, which is sin in the heart. Murder doesn't start with murder; it starts with sinful anger in the heart (vv. 21-22). Don't be as concerned about the proper words in taking an oath; be people of such integrity that your simple word alone will suffice and no oath is necessary (vv. 33-37). Instead of carefully measuring the retaliatory consequences of an offense against you, give in and don't fight back (vv. 38-42). Instead of loving your neighbor and hating your enemy, love your enemies and pray for them (vv. 43-44).
The main point to all of these illustrations in the Sermon on the Mount is that a sinful heart lurks behind every offensive action. By shining the light of His perfection on our dark hearts, the Lord very effectively makes us aware of how short we fall of God's standard of righteousness. That's why we need to receive Christ's righteousness, since we have none of our own.
So the point about lust is made to expose the sinful motives of the heart, showing that even before one actually enters into an adulterous relationship, there's a heart problem that's just as serious in God's eyes as acting on it.
But if the Lord had meant to set lust as an acceptable ground for divorce, He would have said so. He doesn't play games with us to keep us guessing about what pleases Him. Personally, this makes sense to me, since a person can fall into the sin of lust for another person, and repent and receive forgiveness, without his/her spouse ever getting hurt. They don't ever need to know about it; it's a internal war of the one struggling with lust. As sinful creatures, we are going to struggle with various sins all our lives. But there's still a big difference in consequence between fighting the internal battle against the sin nature and going out and acting on it.
Furthermore, engaging in sexual immorality is an external act that can be proven by witnesses and/or testimony. Experiencing lust is internal, and can only be judged by another without any proof. Only God can know whether someone truly lusts or not. Kind of hard to hold up in a court of law!
I hope this helps.