On our last Frictionless Data community call on August 26th we had Amber York and Adam Shepherd from BCO-DMO giving a presentation on Frictionless Data Pipelines for Ocean Science.
BCO-DMO is a biological and chemical oceanography data management office, working with scientists to make sure that their data is publicly available and archived for everyone else to use.
BCO-DMO processes around 500 datasets a year, with all sorts of variability. In the beginning the staff was writing ad hoc scripts and software to process that data, but that quickly became a challenge, as the catalogue continued to grow in both size and the variety of data types it curates.
Having worked for several years with Frictionless Data, BCO-DMO identified the Data Package Pipelines (DPP) project in the Frictionless toolkit as key to overcoming those challenges and achieving its data curation goals.
Together with the Frictionless Data team at Open Knowledge Foundation, BCO-DMO developed Laminar, a web application to create Frictionless Data Package Pipelines. Laminar helps data managers process data efficiently while recording the provenance of their activities to support reproducibility of results
You can learn more on the project here (opens new window). If you would like to dive deeper and discover all about Frictionless Data Pipelines, you can watch Amber York’s and Adam Shepherd’s presentation:
# Other agenda items from our hangout
# Frictionless Hackathon on 7-8 October!
Join the Frictionless Data community for a two-day virtual event to create new project prototypes based on existing Frictionless open source code. It’s going to be fun!
We are currently accepting project submissions, so if you have a cool project in mind, using based on existing Frictionless open source code, this could be an excellent opportunity to prototype it, together with other Frictionless users from all around the world. You can pitch anything - your idea doesn’t need to be complete/fully planned. We can also help you formulate a project if you have an idea but aren’t sure about it. You can also submit ideas for existing projects you need help with!
# Join us next month!
Our next meeting will be on September 30th, exceptionally one hour later than usual. We will hear a presentation from Daniella Lowenberg and Cassiano Reinert Novais dos Santos on the Frictionless Data validation implemented for the Dryad application.
You can sign up here (opens new window).
Do you want to share something with the community? Let us know when you sign up!
# Call recording:
On a final note, here is the recording of the full call: