Architecting Inference Services