The house was under surveillance, and he got surveilled. He loses his job at a Catholic university, and he suffers embarrassment, yes; but the prostitutes have to go to jail. Which seems unfair.
The owner of the business captures some of the unfairness:
[The owner] said that after speaking with his attorney, Thomas H. Ramsay, he concluded that he could not “afford to roll the dice at age 62” and risk greater punishment, so he accepted a plea bargain.
He said police repeatedly referred to him as the “brothel operator,” while treating Sargent “with deference.”
“If you watch the taped interview, the police are almost apologetic with this guy,” Clark said of Sargent. “They told him, ‘You just happened to be in the wrong place at the wrong time,’ and they agreed to contact him at his office, not his home.”