Searchmetrics Interview Question

I should design a distributed web-crawler