Suppose that I have 2 possible models that I might try to build. I can train them both on my data set, but how do I get a sense as to which is better?

When you have the split, you can then tell which idea seems to work better.