Does removing from the Timeline delete the task?

No. The × badge on a task bar only sets Show In Timeline to No. The task remains in the Gantt and task list. See Remove from Timeline.