expr = α β + α^2 + β^2 + α β^2/2 + α^2 β;
vars = {α, β};
Another way is to use PolynomialDegree by Dennis M Schneider and DeleteCases:
PolynomialDegree = ResourceFunction["PolynomialDegree"];
DeleteCases[expr, x_ /; PolynomialDegree[x, vars] > 2]
α^2 + α β + β^2
Comments about the use of pattern α^i_.*β^j_.:
This pattern matches any product of α and β with optional exponents. If the exponents are not specified, they default to 1, e.g.:
ReplaceAll[α β^3, α^i_. β^j_. :> {i, j}]
{1, 3}
Note that in FullForm α is represented without an explicit exponent, while β^3 shows the exponent:
FullForm[α β^3]
Times[α, Power[β, 3]]
Finally, compare the pattern with and without the use of the point, taking FullForm in each case:
FullForm[α^i_ β^j_]
FullForm[α^i_. β^j_.]
Times[Power[α, Pattern[i, Blank[]]], Power[β, Pattern[j, Blank[]]]]
Times[Power[α, Optional[Pattern[i, Blank[]]]], Power[β, Optional[Pattern[j, Blank[]]]]]
ResourceFunction["MultivariateTaylorPolynomial"][x + x*y + y^2 + x*y^2 - x^2*y, {x, y}, 2] Out[42]= x + x y + y^2$\endgroup$