Understanding, Exploiting and Improving Inter-view Relationships