Overview

This EmacsConf 2023 talk demonstrates a collaborative workflow for research data processing and documentation in Emacs. The speakers use Org mode as the central document format and combine it with companion packages for knowledge graph visualization, literate programming, and collaborative editing.

Starting from the National Research Data Infrastructure Germany (NFDI), the talk shows how to retrieve information from Wikidata, clean and process it with different programming languages, visualize relationships, and preserve the work through exportable documentation.

Topics covered

Org mode as a plain-text environment for scientific writing, organization, and publishing
org-roam and org-roam-ui for linking notes and visualizing a knowledge graph
org-babel for literate programming and self-documenting code
SPARQL queries against Wikidata
Data cleaning and processing with shell, Python, awk, and R
Collaborative editing in Emacs using CRDT
Exporting Org documents to formats such as PDF, HTML, and plain text

Speakers

Jonathan Hartman is a trained data scientist and works at the IT Center of RWTH Aachen University, Germany.

Lukas C. Bossert is a trained classical archaeologist and deputy head of the department “Research Process and Data Management” at the IT Center of RWTH Aachen University.

Chapter markers

00:00 — Introduction
01:16 — Org Mode
02:18 — Working together
06:27 — Data cleaning
08:04 — Processing
12:36 — Visualization
14:01 — Preserve

Resources

Tags :

Documentation From User Experience

Marvin Lorber
Documentation , Software , RDM

This post is a condensed version of a talk at our Data Compentcy College

If you regularly use scientific software written by others, or tried to replicate interesting research that relies on software, you have probably also invested weeks of work to solve a software problem or even given up on a software because of missing documentation. Finding a project that might be the solution to your problem and then failing to run the code is frustrating. Being unable to run a project you have built yourself years ago is even worse. Having experienced all those setbacks myself in the past I want to use this post to channel that frustration to fuel solutions for better documentation for our current and future projects.

Carpentries Workshop - Introduction to Python

C. Gonzalez
Training , Workshop , Python

Empowering Researchers with Foundational Computing Skills: Join the Upcoming Carpentries Workshop

In today’s fast-paced research environment, the ability to harness computational tools effectively can make a world of difference. Whether you’re managing data or automating tasks, having the right skills can significantly streamline your work. That’s where The Carpentries come in — a global initiative comprising the Software Carpentry, Data Carpentry, and Library Carpentry communities. These communities are dedicated to equipping researchers with essential computational and data science skills, helping them to work smarter, not harder.

FDM-Werkstatt - Into the RDM-Toolbox!

A. Janz
Event , RDM

The Center of Data Litercacy (German: “Zentrum für Datenkompetenz”) DKZ.2R was officially launched mid November 2023. Already a month later we joined forces with fdm.nrw to organise the very first DKZ.2R-event (find the call for participation here). The “FDM-Werkstatt – Into the RDM-Toolbox” took place from March 18 to 20, 2024 at the IT Center of RWTH Aachen University. In total, 50 participants from all over Germany took part in the workshops. Many of them brought their own topics of interest with them and presented it in one of the 12 sessions. The contents of the sessions ranged from low-level introductions and RDM-basics to elaborate and in-detail coding sessions. For three days we worked together, discussed use cases and new RDM tools. But we also enjoyed the social program such as a tour of the AiX Cave and the server room of the ITC’s High Performance Computing Center. There was a good balance between cognitively demanding workshop sessions and more relaxing social events and lunch breaks which hopefully resulted in an enjoyable and rewarding experience for all participants. On the very last day of the workshop we offered a session to especially discuss ideas and directions for the DKZ.2R. The feedback we got in this session will help us in moving forward with the DKZ.2R and making a lasting impact for future researchers.

Collaborative data processing and documenting using org-babel