Facing one of the league's top defenses, the Warriors struggled early and scratched a season-low 32 points in the first half of Wednesday