Overview

This EmacsConf 2023 talk demonstrates a collaborative workflow for research data processing and documentation in Emacs. The speakers use Org mode as the central document format and combine it with companion packages for knowledge graph visualization, literate programming, and collaborative editing.

Starting from the National Research Data Infrastructure Germany (NFDI), the talk shows how to retrieve information from Wikidata, clean and process it with different programming languages, visualize relationships, and preserve the work through exportable documentation.

Topics covered

  • Org mode as a plain-text environment for scientific writing, organization, and publishing
  • org-roam and org-roam-ui for linking notes and visualizing a knowledge graph
  • org-babel for literate programming and self-documenting code
  • SPARQL queries against Wikidata
  • Data cleaning and processing with shell, Python, awk, and R
  • Collaborative editing in Emacs using CRDT
  • Exporting Org documents to formats such as PDF, HTML, and plain text

Speakers

Jonathan Hartman is a trained data scientist and works at the IT Center of RWTH Aachen University, Germany.

Lukas C. Bossert is a trained classical archaeologist and deputy head of the department “Research Process and Data Management” at the IT Center of RWTH Aachen University.

Chapter markers

  • 00:00 — Introduction
  • 01:16 — Org Mode
  • 02:18 — Working together
  • 06:27 — Data cleaning
  • 08:04 — Processing
  • 12:36 — Visualization
  • 14:01 — Preserve

Resources

Related Posts

A Survival Guide to Research Data Sharing Services in the Rhine-Ruhr Region

A Survival Guide to Research Data Sharing Services in the Rhine-Ruhr Region

A Survival Guide to Research Data Sharing Services in the Rhine-Ruhr Region

There are a lot of reasons why collaborating with other researchers on scientific projects is great! It provides new perspectives and gives you the chance to benefit from other people’s knowledge and input. When it comes to sharing and exchanging data across multiple locations and devices however, researchers are often disoriented and don’t know which tools, cloud services and so on are safe to share data in a secure and ethical way.

Read More
How To: Good Scientific Practice

How To: Good Scientific Practice

“Scientific integrity forms the basis for trustworthy research”, so it says in the Guidelines for Safeguarding Good Research Practice of the DFG, the German Research Foundation. As a major funder of research in Germany the DFG, as well as many other funders of research in Germany and the European Union, requires researchers to follow a certain set of rules conducting their research. These rules are called “good scientific practice” and have to be followed by researchers to be viable for funding. According to the guidelines researchers are required to “document all information relevant to the production of a research result as clearly as is required by and is appropriate for the relevant subject area to allow the result to be reviewed and assessed”. But good scientific practice is not done by documenting your research. It also includes i.a. protecting the personality rights of your subjects and handling research data in an appropriate manner by e.g. “back(-ing) up research data and results made publicly available, as well as the central materials on which they are based and the research software used, by adequate means according to the standards of the relevant subject area, and retain them for an appropriate period of time.” This is where Research Data Management (RDM) comes in. Of course RDM is much more than just creating a backup of your data on a USB-Stick and handing it over to anyone asking for it. “Good scientific practice” in RDM follows the FAIR principles:

Read More
Call for participation

Call for participation

Call for participation!

The Data Literacy Center Rhine-Ruhr (DKZ.2R) issues a call for participation in its “rent-an-expert” project! We offer support for ambitious research projects of PhD students and early postdocs dealing with Data Science and Artificial Intelligence, High Performance Computing and Simulation, and Research Data Management. As the DKZ.2R is funded by the German Federal Ministry of Education and Research (BMBF) as well as the EU, this offer is free of charge!

Read More