ydf

A dotfiles manager+

Be ready to work in just a few minutes on your Fresh OS

Declare your working environment and Automate its configuration

New member on the team?, reproduce your colleague working environment and start working now

👉 A working environment declared in a simple directory structure

Table of Contents

Click on the menu right before README.md as shown in the image below.

About

Are you tired of dealing with dotfiles of tools you are not using, are you tired of installing a bunch of messy and dirty configs and executing an elephant script.

This solution brings you a simple way to declare and install idempotently the tools you need along with its configurations, following the principles of high cohesion and low coupling. Turn the chaos to order, if you install the configuration, you install the tool because those belong to the same package.

With this solution you can create multiple selections of packages for your different needs, for example, you can create a packages selection for your laptop, desktop, servers, different operating systems, etc.

Declaring your working environment give you some benefits: transparency and control over it, allowing you to easily reproduce it on a new machine or fresh OS, you can share it with others, so they can reproduce your working environment, you can version it with git.

What you write you can read, share, save and reproduce, it is simply there, it exists.

Tested OS

It has been tested on the following linux distributions, click to show the dependencies:

Manjaro

Runtime Dependencies:

# SYSTEM: Linux manjaro-gnome 6.5.3-1-MANJARO #1 SMP PREEMPT_DYNAMIC Wed Sep 13 12:21:35 UTC 2023 x86_64 GNU/Linux
# PACMAN
bash 5.1.016-3

Optional Dependencies:

# SYSTEM: Linux manjaro-gnome 6.5.3-1-MANJARO #1 SMP PREEMPT_DYNAMIC Wed Sep 13 12:21:35 UTC 2023 x86_64 GNU/Linux
# PACMAN
yay 12.1.3-1
snapd 2.60.3-1
docker 1:24.0.5-1
docker-compose 2.20.3-1
# GITHUB
yunielrc/yzsh

Ubuntu

Runtime Dependencies:

# SYSTEM: Linux ubuntu-gnome 6.2.0-26-generic #26~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Jul 13 16:27:29 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
# APT
bash 5.1-6ubuntu1

Optional Dependencies:

# SYSTEM: Linux ubuntu-gnome 6.2.0-26-generic #26~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Jul 13 16:27:29 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
# APT
containerd.io 1.6.24-1
docker-buildx-plugin 0.11.2-1~ubuntu.22.04~jammy
docker-ce-cli 5:24.0.6-1~ubuntu.22.04~jammy
docker-ce 5:24.0.6-1~ubuntu.22.04~jammy
docker-compose-plugin 2.21.0-1~ubuntu.22.04~jammy
flatpak 1.14.4-1~flatpak1~22.04.1
# GITHUB
yunielrc/yzsh

It should work on any other linux distribution, but it has not been tested.

Install

Install git, make and vim

Clone the repository and switch to ydf directory

git clone https://github.com/yunielrc/ydf.git && cd ydf

Select the latest stable version

git checkout "$(git tag --sort='version:refname' | grep -Po '^v\d+\.\d+\.\d+$' | tail -n 1)"

Install on Manjaro

Install optional dependencies and ydf on home directory

make install-opt-manjaro && make install-tohome

Install on Ubuntu

Install optional dependencies and ydf on home directory

make install-opt-ubuntu && make install-tohome

Install on Any Linux Distro (Minimal installation)

Minimal installation without optional dependencies

make install-tohome

⚠️ Attention: Instructions that rely on optional dependencies can't be used with the minimal installation. For each instruction you want to use install its dependency.

Instruction	Optional dependency
@snap	snapd
@flatpak	flatpak
docker-compose.yml	docker, docker-compose
@yay	yay
*.plugin.zsh	yzsh
*.theme.zsh	yzsh

Configure

Edit the config file:

• Set the variable YDF_PACKAGE_SERVICE_DEFAULT_OS to manjaro or ubuntu according to your distro. If you have any other distro don't set this variable.

vim ~/.ydf.env

YDF Package

What is a package?

A package is a directory containing files and directories in which some have a special meaning for the interpreter. ydf is an interpreter.

What are the directories and files with special meaning?

Here is an example of a package with 19 directories and files with special meaning, those are instructions that work on any linux distribution:

package1
├── preinstall           # Script executed before all instructions
├── @flatpak             # Install <package1> with flatpak
├── @snap                # Install <package1> with snap
├── install              # Script executed on install
├── docker-compose.yml   # Run docker compose up -d
├── package1.plugin.zsh  # Install yzsh plugin
├── package1.theme.zsh   # Install yzsh theme
├── homeln/              # Create symlinks on home for the first level files and
|                        # directories inside this directory
├── homelnr/             # Create symlinks on home for all files inside this
|                        # directory
├── homecp/              # Copy all files to home directory
├── rootcp/              # Copy all files to root directory
├── homecat/             # Concatenate all files with those existing in home
├── rootcat/             # Concatenate all files with those existing in root
├── homecps/             # Evaluate variables in files and copy them to home
├── rootcps/             # Evaluate variables in files and copy them to root
├── homecats/            # Evaluate variables in files and concatenates them with
|                        # those existing in home
├── rootcats/            # Evaluate variables in files and concatenates them with
|                        # those existing in root
├── dconf.ini            # Load gnome dconf settings
└── postinstall          # Script executed after all instructions

The instructions can be grouped in 4 categories:

• Scripts instructions: preinstall, install, postinstall. These instructions are shell scripts that are executed by bash.

• Package manager instructions: @flatpak, @snap. These instrucions are plain text files, the file can have inside one or more package names that are going to be installed. The file can be empty, in this case the package package1 is going to be installed.

• Directory instructions: homeln, homelnr, homecp, rootcp, homecat, rootcat, homecps, rootcps, homecats, rootcats. These instructions are directories that contains files that are going to be symlinked, copied or concatenated to the home or root directory. For those that end with s all the variables inside each file are substituted with the values defined in the envsubst.env file that is inside the packages directory.

• Tool files instructions: docker-compose.yml, dconf.ini, package1.plugin.zsh, package1.theme.zsh. These instructions are files that are going to be used by a tool. For example docker-compose.yml is going to be used by docker compose. The package1.plugin.zsh is a plugin that is going to be installed inside the YZSH data directory and used by YZSH.

These instructions only work for archlinux, manjaro linux an others arch distros:

package2
├── @pacman
└── @yay

These instructions only work for debian, ubuntu and others debian distros:

package3
├── @apt
└── @apt-get

👉 If you want support for others package managers you can open an issue or create a pull request.

You can check out some examples of packages at: tests/fixtures/packages

What is a YDF Packages Directory

The packages directory is the declaration of your working environment. It's a directory that contains a list of packages and the envsubst.env file, besides it can have one or more packages selection files.

Below is shown the recommended approach of a packages directory:

~/.ydf-packages       # packages directory
├── bat/              # package
├── bmon/             # package
├── htop/             # package
├── aws-cli-v2/       # package
├── mpv/              # package
├── ....              # package
├── envsubst.env      # substitution variables
├── pc-gaming.pkgs    # packages selection
├── laptop-work.pkgs  # packages selection
└── ....              # packages selection

The envsubst.env file has the variables that are evaluated in the files inside of Directory instructions that end with s.

The packages selection are plain text files that contains a list of packages one per line.

The packages directory is where the interpreter is going to look for packages, substitution variables and packages selection.

👉 You can check out my packages directory at: https://github.com/yunielrc/.ydf-packages

Below is shown the approach of only one big package with all dotfiles inside, this is the classic approach used by the most well-known dotfiles managers:

~/.ydf-packages/all/homelnr
├── .vedv.env
├── .viminfo
├── .vultr-cli.yaml
├── .xinitrc
├── .ydf.env
├── .yzsh
│   ├── aliases.zsh
│   ├── environment.zsh
│   └── .yzsh.env
├── .zsh_plugins.zsh
├── .zshrc
└── ....

The advantages of this approach are:

1. simplicity
2. ease of creation

The disadvantages of this approach are:

1. you can't make a custom pick of packages to install
2. loss of the relationship between tool installation and configuration
3. it breaks the principle of high cohesion and low coupling
4. no suitable for sharing, because #1
5. no packages selection
6. no substitution variables
7. no instructions other than homelnr

Usage

Reload your shell to load the new PATH.

exec $SHELL

Show the help

ydf --help

# command output:
Usage:
ydf COMMAND

A tool for managing ydotfiles

Flags:
  -h, --help    Show this help

Management Commands:
  package   Manage packages

Run 'ydf COMMAND --help' for more information on a command.

Add packages to your packages directory

Before adding a package to your packages directory you must create a git repository.

cd ~/.ydf-packages
git init
git remote add origin git@github.com:<your_user>/.ydf-packages.git

Open the packages directory in your favorite code editor .

code ~/.ydf-packages

Create a package and add instructions to it. You can check out my packages directory at: https://github.com/yunielrc/.ydf-packages

Add variables to the ~/.ydf-packages/envsubst.env if apply.

Test that the package works.

ydf package install <package>

Verify that the software was installed and configured correctly.

Create a packages selection if apply and add the package.

echo "<package>" >> ~/.ydf-packages/<packages_selection>.pkgs

Commit the changes.

cd ~/.ydf-packages
git add .
git commit -m "Add <package>"
git push -u origin master

Install packages

When you reinstall your OS or on a new machine you can:

Install packages from a packages selection file

ydf package install <packages_selection>.pkgs

Install some packages

ydf package install <package1> <package2> ...

Install one package

ydf package install <package>

👉 You can test the installation of the packages on a virtual machine before install them on a real one.

It's recommended to check out vedv at https://github.com/yunielrc/vedv for working with virtual machines.

Contributing

Contributions, issues and feature requests are welcome!

Manjaro dev dependencies

# SYSTEM: Linux yuniel-pc 6.1.53-1-MANJARO #1 SMP PREEMPT_DYNAMIC Wed Sep 13 14:10:57 UTC 2023 x86_64 GNU/Linux
# YAY
vedv-git 0.2.3-1
make 4.4.1-2
python-pre-commit 2.20.0-3
shfmt 3.7.0-1
shellcheck 0.9.0-47
nodejs 20.6.1-1
npm 10.1.0-1
bash-bats 1.10.0-2
bash-bats-assert-git 2.1.0-1
bash-bats-file 0.4.0-2
bash-bats-support-git 0.3.0-1
fd 8.7.0-1
# NPM
@commitlint/cli@17.7.1
@commitlint/config-conventional@17.7.0
commitizen@4.3.0
cz-conventional-changelog@3.3.0

Ubuntu dev dependencies

# SYSTEM: Linux user-MS-7D43 6.2.0-33-generic #33~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Sep  7 10:33:52 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
# APT
fd-find 8.3.1-1ubuntu0.1
make 4.3-4.1build1
pre-commit 2.17.0-1
# SNAP
node 18.18.0
shellcheck v0.9.0
shfmt 3.5.1
# NPM
@commitlint/cli@17.7.1
@commitlint/config-conventional@17.7.0
bats@1.10.0
commitizen@4.3.0
cz-conventional-changelog@3.3.0
# GITHUB
bats-core/bats-support
bats-core/bats-assert
bats-core/bats-file
yunielrc/vedv

Configure dev environment

Copy config samples

cp .env.sample .env
cp .ydf.env.sample .ydf.env

Edit the config file .env:

• Set the variable HOST_OS to manjaro or ubuntu according to your distro. If you have any other distro don't set this variable.

• Set the variable TEST_OS to manjaro or ubuntu according to distro that It's going to be tested.

vim ~/.env

Install dependencies for Manjaro

The command below install development dependencies for Manjaro

make install-dev-manjaro

Install dependencies for Ubuntu

The command below install development dependencies for Ubuntu

make install-dev-ubuntu

Install dependencies for Any Linux Distro

For any other linux distribution install development dependencies manually.

Configure vedv

Check out: https://github.com/yunielrc/vedv#configure

Workflow

Code

Write your code

Run Tests

All tests are run within a virtual machine, this ensures that your operating system will never be modified.

The first time the image need to be downloaded and builded for development, this process take a while, below are shown the download and build time for the supported Linux distros at 90Mbps:

Distro	download	build
manjaro	5m 9.11s	13m 4.74s
ubuntu	10m 13.611s	8m 33.755s

Run Unit Testing for one component

make test-suite u="$(fd utils.bats)"

Run Unit Testing for one function

make test-name n='text_file_to_words' u="$(fd utils.bats)"

Run Integration Testing for one function

make test-name n='install_one_from_dir' u="$(fd ydf-package-service.i.bats)"

Run Functional Testing for one function

make test-name n='20homecats' u="$(fd ydf-package-command.f.bats)"

Run All Unit Tests

make test-unit
# 2.331s on manjaro
# 2.304s on ubuntu

Run All Integration Tests

make test-integration
# 53.475s on manjaro
# 1m 11.739s on ubuntu

Run All Functional Tests

make test-functional
# 1m 52.301s on manjaro
# 1m 31.900s on ubuntu

Run All tests

make test-all
# 1m 43.793s on manjaro
# 2m 17.157s on ubuntu

Commit

This project uses Conventional Commits.

For commiting use the command below

make commit

Show your support

Give a ⭐️ if this project helped you!