A three-story apartment building with 51 units will replace a parking lot on Depot Street, across from the outlet mall.