Immorality is frequently motivated by a readiness to conform to law and convention in opposition to our own values. In these cases, it’s not that we care too little about others; it’s that we care too much. More specifically, we care too much about how we stack up in the eyes of others.
Source: Evildoers Often Care Too Much about Others, Not Too Little | Foundation for Economic Education