Ta có: $(a+1)(b+1)=4$ và $3\le\dfrac{(a+b)^2}{4}+(a+b) \Rightarrow a+b\ge 2$.
Đặt: $P=\dfrac{3a}{b+1}+\dfrac{3b}{a+1}+\dfrac{ab}{a+b}-\dfrac{3}{2}-a^2-b^2$
Ta có: $P=\dfrac{3a(a+1)+3b(b+1)}{(a+1)(b+1)}+\dfrac{3}{a+b}-\dfrac{5}{2}-a^2-b^2$
$=\dfrac{3}{4}(a^2+b^2)+\dfrac{3}{4}(a+b)+\dfrac{3}{a+b}-\dfrac{5}{2}-a^2-b^2 $
$=\dfrac{-1}{4}(a^2+b^2)+\dfrac{3}{4}(a+b)+\dfrac{3}{a+b}-\dfrac{5}{2}$
$=\dfrac{-1}{4}[(a+b)^2-2ab]+\dfrac{3}{4}(a+b)+\dfrac{3}{a+b}-\dfrac{5}{2}$
$=\dfrac{-1}{4}(a+b)^2+\dfrac{1}{2}[3-(a+b)]+\dfrac{3}{4}(a+b)+\dfrac{3}{a+b}-\dfrac{5}{2}$
$=\dfrac{-1}{4}(a+b)^2+\dfrac{1}{4}(a+b)+\dfrac{3}{a+b}-1$
$=-\dfrac{[(a+b)-2][(a+b)^2+(a+b)+6]}{4(a+b)}\le0$