It's a full week of partying if you want to.
The infield is the frat party. Its cheap, and you get what you pay for. It's a lot cleaner in the seats, but it's still a party.
Most locals go to Oaks or Therby. Some are even pushing that back to Werby. Other locals just leave town all together.
I think it's definitely worth doing at least once.