Remove some useless ones, add some sensible ones.

Separate flags into architecture-dependent, compiler-dependent and other
flags and only set them when really needed.  Also fix various
inconsistent or wrong flags.

If -m32 is provided on a 64-bit machine, we still use the (non-existing)
64-bit configuration.

If-conversion produces several miscompares.