Ta có:
$(\sqrt2x-y)^2\ge0 \Leftrightarrow 2x^2+y^2\ge2\sqrt2xy \Rightarrow xy\le\dfrac{1}{2\sqrt2-2}$
$(\sqrt2x+y)^2\ge0 \Leftrightarrow 2x^2+y^2\ge-2\sqrt2xy \Rightarrow xy\ge\dfrac{1}{2\sqrt2+2}$
Lại có:
$4x^4+y^4-2x^2y^2$
$=(2x^2+y^2)^2-6x^2y^2$
$=(1+2xy)^2-6x^2y^2$
$=-2x^2y^2+4xy+1$
Xét hàm $f(t)=-2t^2+4t+1$ trên đoạn $\left[\dfrac{1}{2\sqrt2+2};\dfrac{1}{2\sqrt2-2}\right]$
$\min f(t)=3\sqrt2-\dfrac{5}{2} \Leftrightarrow xy=t=\dfrac{1}{2\sqrt2+2}$
$\max f(t)=3 \Leftrightarrow xy=t=1$