"I've done plenty of "cherishing others and wishing the happiness" but maybe because I would have contact with them it would fly out the window. "
Stop! You can do yourself some real damage with that sort of nonsense. Seriously, any "practice" that involves attempting to gin up positive feelings for the jerks, sociopaths and vexatious hangers-on that we run into on our path is as far away from real spirituality as self-flagellation.
"I would love to go into everything he did and said but that would be non-virtuous although I'm not sure how."
You're right to be not sure about it because whoever said that was "non-virtuous" is clueless. What is non-virtuous are the emotional projections involving anger, frustration and even hatred that make it impossible to see the real situation and can cause us to act in terribly destructive ways.
Cheap sentimentality and false compassion can do the same.
On the other hand, its highly virtuous (and rather practical too..) to make as thorough and dispassionate an assessment of the situation as you can, and then figure out how you might best extract yourself from it. Han Shan called it the "Sword of Discrimination." Not just mindfulness, but a constant sharp inquisitiveness: What the hell is going on? Why am I feeling this, that or the other.. why do I feel this obligation, that obligation.. what are my real obligations?
A Chan Master once undertook the instruction of a novice monk who was having great difficulty in detaching himself from the persons of his former, secular life. "You cannot serve the Dharma until you sever these bonds," said the Master. "You must destroy these possessive relationships! Kill them! Regard them as if they no longer existed!"
The novice asked, "But my parents? Must I slay them, too?"
And the Master replied, "Who are they to be spared?"
"And you, Master," said the novice, "must I kill you, too?"
And the Master smiled and said, "Don't worry. There is not enough of me left for you to get your hands on."
He wasn't kidding