What term specifies a filter's maximum ability to reject signals outside its passband?

Answer: D. Ultimate rejection

Ultimate rejection is the most a filter can reduce signals outside its passband. Insertion loss is about signals inside the passband.

Tip: Ultimate rejection = maximum rejection outside.