How to Document and Organize Datasets in Obsidian


Dear Scholar,

I am currently conducting very data-intensive research that involves records from thousands of forest plots and measurements of millions of individual trees conducted over 50 years in New Zealand. A lot of data to analyze and find out how forests are changing. This massive data collection is aggregated from many sources, then cleaned, arranged, combined, filtered, recombined, processed, transformed, recleaned, and rearranged again. A very messy process that far surpasses the capacities of my brain.

To overcome this limitation, I started using Obsidian's Canvas feature to document and organise this process. The result is one of the most useful canvases in my research! If your research uses some data in some way, check out this week's article:

While this example primarily deals with CSV files and scripts that transform them, you can be creative and apply the same method to other domains. For instance, you could organize your systematic literature review with the same methods, but instead of forest data, you have papers! The key here is that you can document how datasets relate to one another and where you store them simultaneously.

Join the Upcoming Webinar

The Academic Knowledge Management Webinar 2.0 is coming up in a week, on May 3rd. Today's article is in line with what you will learn in this webinar: How to organize your research. Before the webinar, I always ask some participants to share their desires with me and adjust the webinar accordingly. Here is what the current survey is saying:

In this webinar, therefore, I will present to you how I organize large and complex projects with Obsidian. Here is a preview of my current strategy:

It has multiple connected elements (the first four shown in the screenshot):

  1. Open Directions: Notes describing what I need to do next, automatically colour coded by progress.
  2. Results: Notes describing key findings to use in a paper later.
  3. Follow-Ups: - Additional questions to ask and refine results
  4. Multiple canvases to capture various aspects of the project.
  5. Subtasks to break down big directional tasks.
  6. Progress: Notes capturing day-to-day progress.
  7. Reading Canvas: Organizes papers (see this blog post)
  8. ...and a few more bells and whistles.

This system gives me clarity and guidance every day when I show up at work and sit down in front of my computer and the gigabytes of forest data waiting to reveal their secrets.

Ready to transform how you organise research and boost your productivity and sanity? This webinar will include all the templates you need to set up the system yourself.

Free webinar ticket with course bundle

Those who decide to go all-in and get the Essential Course Bundle will, receive a free webinar ticket. This bundle includes the three most popular and important courses on note-taking, lit reviews and academic writing.

Wishing you a wonderful weekend,

Ilya Shabanov, The Effortless Academic

Don't want to receive emails like this: Unsubscribe from newsletter only .
Want to unsubscribe from all future emails?
Unsubscribe from everything.
Your
Preferences.

The Effortless Academic

Literature Review Tools, Note-Taking Strategies and AI tutorials for the modern academic. Publish more with less effort and supercharge your career.

Read more from The Effortless Academic

Conceptual Literature Review Dear Reader, A literature review starts with reading. But what is reading other than unpacking packaged information into a set of ideas that link together in meaningful ways in your head? What if we could start the process with the mindmap? This is what I call a conceptual literature review. Given a few papers, it uses AI to extract concepts and claims and create an interlinked information network. The output is an Obsidian vault with 100s of connections and...

Consensus (Sponsor) Consensus is one of the most advanced AI literature search tools utilising both the reference network and AI semantic similarity to find relevant papers and summarise their findings. I've used Consensus almost daily while writing my PhD. » Try Consensus Discount Codes:effortlessacademic (25% off) The most minimal knowledge management system for beginners (+ video walkthrough) Dear Reader, I recently introduced a colleague to Obsidian and have pitched it to our lab many...

Paperpal (Sponsor) For researchers juggling multiple tools to check grammar, plagiarism, and AI detection, Paperpal now brings everything together in one place. With the new full paper check, you can upload your draft and run all three checks at once, helping you catch issues early and move to submission with greater confidence.» Try Paperpal Document Check Discount Codes:ILYAPRIME30 (30% off) What are AI skills, and how to use them well? Dear Reader, In the early days of AI (...long...