After you delete an app password that's used for Azure Multi-Factor Authentication, the app password appears to continue to work.
This problem occurs because the token that's acquired after a user successfully signs in by using an app password continues to work until the token expires. The token works only on devices on which the user successfully signed in.
Wait for the token to expire. This may take from 8 to 24 hours, depending on the service that the user is accessing. This practice follows the same guidelines for when passwords are changed or when users are deleted.