The Line Between Toy and Game

I've written before about the difference between games and puzzles, and I'd like to make comments on another of Greg Costikyan's distinctions. A game has a goal, while a toy does not. Toys rapidly get boring until you eventually invent a game. SimCity is a toy, but it's easy to add a goal and make a game of it (e.g. build the biggest metropolis).

Video games justify themselves as games by giving the player an explicit goal. Kill all the bad guys. Capture the flag. Get the most points. Put the ball in the goal.

But it seems to me like there's no way to enforce these goals. "Winning" is ultimately decided by the consensus of the players. If the players all decide that the winner is whoever has the biggest negative score, who's to stop them? When playing Super Smash Bros. on stock mode, the game proclaims that whoever survives to the end is the winner, but my friends and I often decide that the winner is whoever had the most kills.

So are all games toys? If I come up with a definitive answer, I'll post again.