I am fine with either colour or number, although I think number makes more sense. Like someone said, some cities have many lines and eventually you run out of highly distinguishable colours.
For example, Shanghai currently has 12 lines, with multiple (at least 5) under construction. They use numbers along with unofficial colour pattern as well. Do you see how line 8 and 9, 1 and 6 are highly similar? It is easy to make mistakes even for non-colour blind people, especially when in a hurry.
Shanghai Metro
You know what is horrible? Toronto has this subway line called "Yonge-University-Spadina line", a name as retarded as it gets. Half of the line is called "the Spadina line" when only two stops are on a street called Spadina ave.