Posts about new Renku features, platform design, reproducibility in data science, solutions to engineering challenges, and Renku use-cases.
Renku has evolved considerably over the past several years; from our strong initial focus on computational reproducibility to our current goals of building connections within and across communities. As we engaged with various research communities, we understood that one of the main struggles researchers face in collaborative projects is consolidating a scattered collection of resources across various providers, tools, and technologies.
Do not miss the opportunity to meet other Renku Users and learn how they leverage Renku for data science, research, and teaching. Moreover, we will share updates about the upcoming new version of Renku, Renku 2.0, and the features prioritised for the second half of 2024. You are also invited to voice your challenges in an open discussion about collaborative open research, and to bring your suggestions for new features in Renku.
On June 27th between 13:00 CET and 17:00 CET, the SDSC team developing Renku invites you to a hybrid Meeting held both in Bern and via Zoom.
Working with your data in Renku is faster and simpler: you can write and read your data from your external storage services, such as S3, Azure Blob or WebDAV among others.
RenkuLab interactive sessions can access data hosted in a cloud storage system, and the storage is simply mounted as another folder in your session. You can concentrate on developing your code to analyse your data, rather than worrying about moving data around! Plus, when you add a cloud storage to your project, that configuration is available to all project members, so you can share the results with your colleagues or even with the broader community. Who has access is still completely in your control: the access to cloud storage is controlled by the storage provider, not Renku.
To add cloud storage to your Renku project, go to your project's Settings and find the new Cloud Storage tab. Check out Renku documentation for more details on how to configure cloud storage for your project.
If you use SSH sessions via the CLI, you can use cloud storage there too. You just need to configure cloud storage for your project on RenkuLab.io, and those storages will be mounted in your remote session.
Never lose track of important projects again! Pin a project by clicking the new pin icon in the top left corner of the project image. You can pin up to 5 projects, and these will stay at the top of the dashboard, above your recently viewed projects.
We are proud to release our completely redesigned landing page! Our new page better describes Renku and its capabilities, and features multiple entry points for getting started with the platform.
Renku allows you to work either locally with the Renku CLI or directly in the cloud with RenkuLab. What if you wish to continue working in your local environment, but do not have adequate compute resources on your machine? An SSH session on RenkuLab can help you there: you can develop code on your computer and run it in the cloud with SSH sessions. One of the motivations to work locally and execute in the cloud is to be able to work from your favorite IDE while accessing additional compute resources. In this post, we discuss:
On RenkuLab you can now pause sessions and later resume them exactly where you left off. All of your work in progress, including files, data, and environment changes not saved to git, are resumed right as you left them!
The first thing you’ll notice in RenkuLab is that the session stop button has been replaced with a pause button.
Paused sessions will be maintained for 15 days, after which they (and any unsaved work) will be deleted permanently.
You can resume paused sessions from the Dashboard. Not sure if you have any work left un-committed from your last session? The un-saved work indicator on the Dashboard will let you know!
Session pausing replaces RenkuLab's branch-based auto-save mechanism. Most users do not have to do anything to transition from auto-saves to persistent sessions. However, if your last session went into an auto-save, you can still retrieve that work by using Start with Options and selecting your most recent auto-save branch. If your project contains auto-save branches that you do not need anymore, you can safely delete them.
Recreating exactly the result of a data analysis should be easy because, after all, it has been done once before — by a computer! In reality, those of us who work with data often cannot replicate even our own results several months later, much less reproduce work done by others. The reason for this is also clear: repeating an analysis requires keeping careful records of the steps and the exact execution environment.
This record keeping can be tedious if done manually, but, in recent years, a rich ecosystem of freely available tools facilitating reproducibility has flourished. The adoption of these tools has been slow, however, since a particular technical skill set is necessary to use them, and for many scientists and data analysts, the time investment required is simply too large.
At the Swiss Data Science Center, we are building a platform called Renku that aims to lower the barriers to leveraging this ecosystem. Our vision is to provide a platform where data analysis projects can be discussed, repeated, and verified, and split up into components which can be individually shared, reused, and recombined; we aim to do all this while imposing as small of a disruption as possible to the user’s established way of working. We will try to illuminate the concepts and technology behind Renku and demonstrate its use through a series of articles, starting with this one.