A few months ago, I found out about Frictionless, a framework for data management to describe, extract, validate, and transform tabular data. It is very simple to use, and provides a sweet Python toolkit to complete several data tasks. This Open Knowledge Foundation short video presents a high-level introduction about the toolkit and its purpose.

While experimenting with the tool, I enrolled in the Frictionless data Hackaton to peek at the toolkit, community and ecosystem a little more. During the event, I created a tutorial in Google Collab on how to use Frictionless Data and its Excel plugin to generate metadata and describe Transparency International’s dataset for the Corruption Perception Index, a dataset I’ve been curious to explore for some time already. The tutorial covers all steps from installing the required python modules to actually exploring and describing the data.

Learning how to use the tools and creating (and reviewing, and re-reviewing) the documentation “on the fly” was an interesting experience. It took me more time than I usually need to create similar pieces of documentation, but in the other hand everything felt fresh, with that curiosity that accompanies all new things. Everything in the tutorial were tasks I actually wanted to perform, in order to properly explore and describe the data.

Frictionless is a very straightforward framework for describing data, already adopted by a number of institutions and projects. However, as of now it’s not that widespread yet, something that will make data tasks standardized indeed, and therefore much simpler, as intended by its creators.

If you want to try the framework yourself, the tutorial I created is now available at Frictionless framework website under Notebooks > Frictionless Excel (here’s the direct link). There, you can also explore other tutorials, for several data tasks and formats.

Have you used Frictionless or other data management frameworks before? Was my tutorial and this documentation useful to you? Want to share your own thoughts? Contact me at hello@mirianbr.com and let’s chat! 👋🏼