I've been on the receiving end of this, I was president of a kart club at a local speedway when a kart flipped, the red flag came out, another competitor ignored it and ended up hitting the wall. Back then, the wall was boiler plate, and a piece of rebar went through his shoulder. We ended up being sued as well as the track.
Short story, the guy lost, he signed a waiver saying that if the track was not safe he would not race on it, he signed one for the club and one for the track (read one sometime..it covers everything). We paraded a bunch of witnesses saying he ignored the red and that racing was dangerous..etc etc. Fortunately the club had an attorney that raced and of course the track had a retained attorney.
The bad part was the track said if it ever happened again, we would lose the venue, and the club had to reimburse the track for their legal fees. This track was owned by an old school racing family and they cut us a great deal on rent, they jacked the rent up as well, although not too badly.
Since then the walls have been made out of concrete, the track closed, and was reopened under another individual, and the karters still use it.
IMO, no one is holding a gun to your head, you should've read the waiver, racing is dangerous (the first kart race I attended at Daytona, a guy was impaled on his steering column when the kart went under the armco backwards). If you don't think it's safe, pack your stuff and leave, you always have that option. If you are on the track, you have decided you want to be there and it's worth the risk. These guys that race, then get hurt, then sue the track is garbage..keep it up and insurance won't be available or the cost will be driven too high, and maybe even lose the venue.
Sorry dude, it was your personal decision, nobody else's.