Here is a great article by Jayson Stark of ESPN on the subject.
The problem with shutting him down is that it doesnt guarantee you anything. There is simply not enough information out there in any kind of valid sample size. He might go on to have a long career. He also might blow out his arm on his first pitch next season. The other pitchers in the rotation might not be as healthy next season and it wont matter how well he pitches they arent getting back to the playoffs.
So many things have to go right to win a pennant and then a Championship that I dont think you voluntarily shut down your ace when its one of those magical years where everything is breaking right and you have a real shot at it.
I do give Rizzo a lot of credit for standing up in what he believes in even though its getting him a ton of criticism.