Select the key frames you want to remove (see Selecting key frames).

Choose Edit → Clear or press Delete.

If you are using the key frame panel (see Working with the key frame panel) you can click the X too.