Common Crawl is a nonprofit foundation dedicated to building and maintaining an open crawl of the web in order to enable a new wave of innovation in business, education, and research.
commoncrawl.org