Neighbors oppose apartment plan at historic Milford property