Amazon Interview Question

Explain gradient descent, batch norm, how to accelerate, how to parallel, how to change the batch size when parallel and how to save memory for running.