I really like the checkbox (or slider) suggestion - I think it could be very helpful all around.  I've helped review some submissions, and I try to apply consistent criteria.  I based my criteria on the instructions, but there's certainly room for interpretation particularly when it comes to how to weight criteria.  So it would probably both help me review faster and score more accurately if it was "+1 for OSS, +1 for end-user, -2 for content itself = average" type thing, and this could likely both let the track assemblers reweight these criteria to give balance to a track, and also provide some simple actionable feedback for submitters (e.g. "your strongest factor was that the topic was interesting, but the weakest factor was that people worried it was going to turn into a sales pitch").