How to Document and Organize Datasets in Obsidian


Dear Scholar,

I am currently conducting very data-intensive research that involves records from thousands of forest plots and measurements of millions of individual trees conducted over 50 years in New Zealand. A lot of data to analyze and find out how forests are changing. This massive data collection is aggregated from many sources, then cleaned, arranged, combined, filtered, recombined, processed, transformed, recleaned, and rearranged again. A very messy process that far surpasses the capacities of my brain.

To overcome this limitation, I started using Obsidian's Canvas feature to document and organise this process. The result is one of the most useful canvases in my research! If your research uses some data in some way, check out this week's article:

While this example primarily deals with CSV files and scripts that transform them, you can be creative and apply the same method to other domains. For instance, you could organize your systematic literature review with the same methods, but instead of forest data, you have papers! The key here is that you can document how datasets relate to one another and where you store them simultaneously.

Join the Upcoming Webinar

The Academic Knowledge Management Webinar 2.0 is coming up in a week, on May 3rd. Today's article is in line with what you will learn in this webinar: How to organize your research. Before the webinar, I always ask some participants to share their desires with me and adjust the webinar accordingly. Here is what the current survey is saying:

In this webinar, therefore, I will present to you how I organize large and complex projects with Obsidian. Here is a preview of my current strategy:

It has multiple connected elements (the first four shown in the screenshot):

  1. Open Directions: Notes describing what I need to do next, automatically colour coded by progress.
  2. Results: Notes describing key findings to use in a paper later.
  3. Follow-Ups: - Additional questions to ask and refine results
  4. Multiple canvases to capture various aspects of the project.
  5. Subtasks to break down big directional tasks.
  6. Progress: Notes capturing day-to-day progress.
  7. Reading Canvas: Organizes papers (see this blog post)
  8. ...and a few more bells and whistles.

This system gives me clarity and guidance every day when I show up at work and sit down in front of my computer and the gigabytes of forest data waiting to reveal their secrets.

Ready to transform how you organise research and boost your productivity and sanity? This webinar will include all the templates you need to set up the system yourself.

Free webinar ticket with course bundle

Those who decide to go all-in and get the Essential Course Bundle will, receive a free webinar ticket. This bundle includes the three most popular and important courses on note-taking, lit reviews and academic writing.

Wishing you a wonderful weekend,

Ilya Shabanov, The Effortless Academic

Don't want to receive emails like this: Unsubscribe from newsletter only .
Want to unsubscribe from all future emails?
Unsubscribe from everything.
Your
Preferences.

The Effortless Academic

Literature Review Tools, Note-Taking Strategies and AI tutorials for the modern academic. Publish more with less effort and supercharge your career.

Read more from The Effortless Academic

Dear Scholar, When doing innovative research, you need to do a lot of reading, looking for a tiny grain of insight or a gap that fits your research perfectly and allows you to identify new publishable insight. Most of what you read, though, is not strictly relevant. Looking for important, impactful questions can feel like searching for a needle in a haystack. A new tool we are reviewing this week promises to help you find that needle using AI: Anara.com. Anara can combine insights from dozens...

Sponsor: Sourcely Sourcely is your AI-powered academic search assistant, offering access to over 200 million papers and advanced search filters. It streamlines research by finding credible sources, summarizing them, and exporting citations instantly—helping you save time and improve quality. Learn more about Sourcely Get 40% off with code Effortless40 (annual)or 20% off with Effortless20 (monthly) Dear Scholar, When I do a literature review, one of my biggest struggles is to keep papers...

Dear Scholar, This week, we are reviewing a different AI tool to help with academic writing. Instead of helping you to edit, WriteWise helps you start—and it does so by grounding its entire drafting system in the linguistic theories of John Swales. If you’ve ever taught students to identify the “research space” or structure an introduction using Swales’ CARS model, you understand what makes WriteWise different to ChatGPT or Paperpal. Writewise is designed to create first drafts, not finished...