An ICP is a list of people you will say yes to. It is also, more importantly, a list of people you will say no to. The second list is harder to keep.
Early on every paying customer feels like validation. They aren't, if they're not in the ICP. They are a future churn event with a logo attached. They will pull the roadmap toward features only they need.
We've turned down customers that wanted to pay us. It feels insane every time. It is the single highest-leverage thing a small team can do.