No one processes and provides more earth observation data than NASA. NASA provides access to more than 17.5 petabytes of earth observation data across more than 11,000 unique data products from its satellite and aerial missions. In FY 2016, NASA delivered more than 1.51 billion earth science data products to more than 3 million data users around the world. Managing this data is a huge task. We are working with NASA’s Earth Science Data System (ESDS) Program to build Cumulus, a cloud-based prototype to ingest, process, catalog, archive, and distribute NASA’s Earth Data streams.

Cumulus is one of a suite of tools that NASA is developing to better leverage cloud computing for data storage, processing, and access. We are working closely with NASA’s ESDS team as well as several of the Distributed Active Archive Centers (DAACs) that ensure Earth Data missions are available and accessible to users. We are working with these teams to prototype and refine a system that can be used to process, archive, and publish data as varied as lightning strike datasets and MODIS scenes.

Cumulus will be open source. We are working with NASA to transfer ownership of Cumulus to NASA and to provide the code and documentation through their platforms. We will keep you posted when the code is available. We hope that Cumulus will be useful to other organizations building imagery publishing pipelines.