Skip to content

Latest commit

 

History

History
584 lines (489 loc) · 18.9 KB

01.filesystem.structure.md

File metadata and controls

584 lines (489 loc) · 18.9 KB
Raw

Basic

Print current working directory

user@ubuntu:~$ pwd
/home/user

List files & directories

List files/directories in current working directory:

user@ubuntu:~$ ls
Desktop  Documents  Downloads  Music  Pictures  Public  Templates  Videos

List relative path content:

user@ubuntu:~$ ls Pictures/
image1.jpg  image2.png  image3.jpg

List absolute path content:

user@ubuntu:~$ ls /home/user/Pictures/
image1.jpg  image2.png  image3.jpg

Use long listing format, and order files using last modification time:

user@ubuntu:~$ ls -lrth
total 40K
drwxr-xr-x 2 user user 4,0K mai   11 20:04 Videos
drwxr-xr-x 2 user user 4,0K mai   11 20:04 Templates
drwxr-xr-x 2 user user 4,0K mai   11 20:04 Public
drwxr-xr-x 2 user user 4,0K mai   11 20:04 Pictures
drwxr-xr-x 2 user user 4,0K mai   11 20:04 Music
drwxr-xr-x 2 user user 4,0K mai   11 20:04 Downloads
drwxr-xr-x 2 user user 4,0K mai   11 20:04 Desktop
-rw-rw-r-- 1 user user   14 mai   11 20:20 myfile
-rw-rw-r-- 1 user user   45 mai   11 20:20 myfile2
drwxr-xr-x 2 user user 4,0K mai   11 20:21 Documents

Include hidden files when listing:

user@ubuntu:~$ ls -la
total 96
drwxr-xr-x 14 user user 4096 mai   11 20:20 .
drwxr-xr-x  5 root root 4096 mai   11 20:03 ..
-rw-------  1 user user   87 mai   11 20:11 .bash_history
-rw-r--r--  1 user user  220 mai   11 20:03 .bash_logout
-rw-r--r--  1 user user 3771 mai   11 20:03 .bashrc
drwxrwxr-x  5 user user 4096 mai   11 20:04 .cache
drwx------ 14 user user 4096 mai   11 20:11 .config
drwx------  3 user user 4096 mai   11 20:04 .dbus
drwxr-xr-x  2 user user 4096 mai   11 20:04 Desktop
-rw-r--r--  1 user user   26 mai   11 20:04 .dmrc
drwxr-xr-x  2 user user 4096 mai   11 20:21 Documents
drwxr-xr-x  2 user user 4096 mai   11 20:04 Downloads

...[SNIP]...

Understand ls -l output : https://unix.stackexchange.com/a/103118/207157

Absolute path vs Relative path

Absolute path is a path that starts with / (which we call it root in Linux). Example : /home/user/Pictures/.

Relative path is a path that doesn't start with /, and is calculated according to current working directory. Example : If I'm inside /home/user/, and I want to list the content of /home/user/Pictures/ using a relative path, I'll just type the command ls Pictures/.

Change current working directory

Command cd is used to change the current working directory.

Change current working directory to directory Pictures/:

user@ubuntu:~$ pwd
/home/user
user@ubuntu:~$ cd Pictures/
user@ubuntu:~/Pictures$ pwd
/home/user/Pictures

Go to parent directory :

user@ubuntu:~$ pwd
/home/user
user@ubuntu:~$ cd ..
user@ubuntu:/home$ pwd
/home
user@ubuntu:/home$

Go back to previous location :

user@ubuntu:~$ pwd
/home/user
user@ubuntu:~$ cd /etc/cron.d/
user@ubuntu:/etc/cron.d$ pwd
/etc/cron.d
user@ubuntu:/etc/cron.d$ cd -
/home/user
user@ubuntu:~$ pwd
/home/user

Create files & directories

Create an empty file:

user@ubuntu:~$ touch file1

Create an empty file by using an absolute path:

user@ubuntu:~$ touch /home/user/Documents/myfile1

Create a directory:

user@ubuntu:~$ mkdir directory1

Create an hierarchy of directories:

user@ubuntu:~$ mkdir -p dir1/subdir2/subdir3

Copy files & directories

Copy the file myfile to the directory /home/user/Downloads:

user@ubuntu:~$ cp myfile /home/user/Downloads/

Copy the directory dir1 to the directory /home/user/Downloads:

user@ubuntu:~$ cp -r dir1/ /home/user/Downloads/

Show verbose output while copying:

user@ubuntu:~$ cp -rv dir1/ /home/user/Downloads/
'dir1/' -> '/home/user/Downloads/dir1'
'dir1/subdir2' -> '/home/user/Downloads/dir1/subdir2'
'dir1/subdir2/subdir3' -> '/home/user/Downloads/dir1/subdir2/subdir3'

When copying files and directories, we can use absolute and relative paths for both source and destination.

Move files & directories

Move the file myfile to the directory /home/user/Downloads:

user@ubuntu:~$ mv myfile /home/user/Downloads/

Move the directory dir1 to the directory /home/user/Downloads:

user@ubuntu:~$ mv dir1/ /home/user/Downloads/

Show verbose output while moving:

user@ubuntu:~$ mv -v dir1/ /home/user/Downloads/
'dir1/' -> '/home/user/Downloads/dir1'

When moving files and directories, we can use absolute and relative paths for both source and destination.

Create links to files & directories

Create a symbolic relative link to a file:

user@ubuntu:~$ ln -sr Downloads/myfile mylink
user@ubuntu:~$ ls -l mylink 
lrwxrwxrwx 1 user user 16 mai   12 20:16 mylink -> Downloads/myfile

Create a symbolic absolute link to a file:

user@ubuntu:~$ ln -s /home/user/Downloads/myfile mylink
user@ubuntu:~$ ls -l mylink 
lrwxrwxrwx 1 user user 27 mai   12 20:34 mylink -> /home/user/Downloads/myfile

The simple explanation to the difference between absolute and relative symbolic links :

  • An absolute symbolic link will stay valid no matter when you move the symbolic link, but it will become invalid as soon as you you move the file.
  • A relative link stays valid as long as you keep the same relative path between the link and the file. It means, even if you have to move the file, just move the link with it, in a way that keeps the same relative path between them.

Symbolic links to directories are created the same way as symbolic links to files.

Delete files & directories

Delete a file:

user@ubuntu:~$ rm myfile

Delete a directory:

user@ubuntu:~$ rm -r dir1/

Show verbose output while deleting:

user@ubuntu:~$ rm -rv dir1/
removed directory 'dir1/subdir2/subdir3'
removed directory 'dir1/subdir2'
removed directory 'dir1/'

We can use both absolute and relative when deleting files and directories.

Show disk usage of files & directories

Show disk usage of a file:

user@ubuntu:~$ du -sh Videos/movie-2020.mp4
812M     Videos/movie-2020.mp4

Show disk usage of a directory:

user@ubuntu:~$ du -sh /home/user/Pictures/
400K    /home/user/Pictures/

Show disk usage of all files and directories in the current directory, and sort them using their size.

user@ubuntu:~$ du -ah ./ | sort -h
...[SNIP]...
36K     ./.config/openbox
44K     ./.config/pulse/38c787e34637445597c7567da81abdfd-card-database.tdb
48K     ./.cache/lxsession/Lubuntu/run.log
52K     ./.cache/lxsession/Lubuntu
56K     ./.cache/lxsession
76K     ./.config/pulse
100K    ./.cache
120K    ./Pictures/image1.jpg
120K    ./Pictures/image2.png
156K    ./Pictures/image3.jpg
232K    ./.config
400K    ./Pictures
23M     ./Videos
23M     ./Videos/movie-2020.mp4
24M     .

We can use both absolute and relative paths of files and directories.

Advanced

What's using my disk space ?

The command ncdu allows us to show interactively the disk usage of a directory and all of it's sub files/directories. We can then move through these files & directories, and delete any one of them using the letter d.

This is very useful to find big files and directories that are consuming disk space.

asciicast

This command isn't installed by default, and we need to install it using the OS package manager.

  • On Debian based distributions (like Ubuntu) : 
    sudo apt-get update && sudo apt-get install ncdu -y
  • On Red Hat based distributions (like Fedora): 
    dnf install ncdu

Show files' types

The command file is used to get the type of a file:

user@ubuntu:~$ file Videos/video.mp4 
Videos/video.mp4: ISO Media, MP4 Base Media v1 [IS0 14496-12:2003]
user@ubuntu:~$ file Pictures/image1.jpg 
Pictures/image1.jpg: JPEG image data, JFIF standard 1.02, aspect ratio, density 1x1, segment length 16, Exif Standard: [TIFF image data, big-endian, direntries=6, xresolution=86, yresolution=94, resolutionunit=2, software=Adobe Photoshop CS3 Windows, datetime=2008:03:04 14:28:07], baseline, precision 8, 1024x640, frames 3
user@ubuntu:~$ file mylink 
mylink: symbolic link to /home/user/Downloads/myfile

Show files & directories informations

The command stat displays some informations about a file/directory like owner & group, last access/change time:

user@ubuntu:~$ stat file1
  File: 'file1'
  Size: 0               Blocks: 0          IO Block: 4096   regular empty file
Device: 801h/2049d      Inode: 414450      Links: 1
Access: (0664/-rw-rw-r--)  Uid: ( 1002/    user)   Gid: ( 1002/    user)
Access: 2020-05-12 01:08:04.015458094 +0200
Modify: 2020-05-12 01:08:04.015458094 +0200
Change: 2020-05-12 01:08:04.015458094 +0200
 Birth: -

Understand stat output : https://www.thegeekstuff.com/2009/07/unix-stat-command-how-to-identify-file-attributes/

List directory content as a tree

The command tree displays recursively the content of a directory as a tree:

user@ubuntu:~$ tree
.
├── Desktop
├── Documents
│   └── myfile1
├── Downloads
│   └── myfile
├── file1
├── Music
├── myfile2
├── mylink -> /home/user/Downloads/myfile
├── ncdu.cast
├── Pictures
│   ├── image1.jpg
│   └── image2.png
├── Public
├── Templates
└── Videos
    └── video.mp4

8 directories, 9 files

This command isn't installed by default, and we need to install it using the OS package manager.

  • On Debian based distributions (like Ubuntu) : 
    sudo apt-get update && sudo apt-get install tree -y
  • On Red Hat based distributions (like Fedora): 
    dnf install tree

List partitions/filesystems

List partitions and their mounting paths (if they are mounted):

user@ubuntu:~$ lsblk 
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda      8:0    0   20G  0 disk 
├─sda2   8:2    0    1K  0 part 
├─sda5   8:5    0  510M  0 part [SWAP]
└─sda1   8:1    0 19,5G  0 part /
sdb      8:32   1  7,2G  0 disk 
└─sdb1   8:33   1  7,2G  0 part /media/user/myUSB

Display informations about mounted file systems:

user@ubuntu:~$ mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=990764k,nr_inodes=247691,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=204132k,mode=755)
/dev/sda1 on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /sys/fs/cgroup type tmpfs (rw,mode=755)
/dev/sdb1 on /media/user/myUSB type fuseblk (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096,uhelper=udisks2)

Display partitions/filesystems disk space usage

The command df allows us to display the disk space usage on our partitions.

Display a partition disk space usage:

user@ubuntu:~$ df -H /
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        21G   16G  3,6G  82% /
user@ubuntu:~$ df -H /media/user/myUSB
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdc1       7,8G  7,5G  304M  97% /media/user/myUSB

Display all partitions/filesystems disk space usage:

user@ubuntu:~$ df -H
Filesystem      Size  Used Avail Use% Mounted on
udev            1,1G     0  1,1G   0% /dev
tmpfs           210M  6,4M  203M   4% /run
/dev/sda1        21G   16G  3,6G  82% /
/dev/sdc1       7,8G  7,5G  304M  97% /media/user/myUSB
tmpfs           1,1G   95k  1,1G   1% /dev/shm
tmpfs           5,3M  4,1k  5,3M   1% /run/lock
tmpfs           1,1G     0  1,1G   0% /sys/fs/cgroup
cgmfs           103k     0  103k   0% /run/cgmanager/fs
tmpfs           210M   21k  210M   1% /run/user/1000
tmpfs           210M     0  210M   0% /run/user/1002

Mount & Unmount partitions/filesystems

The command udisksctl allows us to mount and unmount partitions using a simple procedure and without the need to root permissions.

Mount & Unmount a partition using udisksctl:

user@ubuntu:~$ udisksctl mount -b /dev/sdb1
Mounted /dev/sdb1 at /media/user/myUSB.
user@ubuntu:~$ udisksctl unmount -b /dev/sdb1
Unmounted /dev/sdb1.

The commands mount & umount are more sophisticated, and allow us to mount in more advanced ways.

Mount the partition /dev/sdb1 to the directory my_mount_dir using mount:

user@ubuntu:~$ sudo mount /dev/sdb1 /home/user/my_mount_dir
[sudo] password for user:
user@ubuntu:~$ lsblk
Filesystem      Size  Used Avail Use% Mounted on
...[SNIP]...
sdb           8:32   1   7,2G  0 disk 
└─sdb1        8:33   1   7,2G  0 part /home/user/my_mount_dir

Unmount the previously mounted partition using umount:

user@ubuntu:~$ sudo umount /home/user/my_mount_dir
[sudo] password for user:
user@ubuntu:~$ lsblk
Filesystem      Size  Used Avail Use% Mounted on
...[SNIP]...
sdb           8:32   1   7,2G  0 disk 
└─sdb1        8:33   1   7,2G  0 part

Get filesystem type

Get the filesystem type using lsblk:

user@ubuntu:~$ lsblk -f /dev/sda1
NAME FSTYPE LABEL UUID                                 MOUNTPOINT
sda1 ext4         ec65c3c5-3d36-4642-8848-589decc8d5e0 /
user@ubuntu:~$ lsblk -f /dev/sdb1
NAME   FSTYPE LABEL UUID                                 MOUNTPOINT
sdb
└─sdb1 ntfs   myUSB 58B9DD4B057949F4                     /media/user/myUSB

Get the filesystem type + some extra informations using file:

user@ubuntu:~$ sudo file -sL /dev/sdb1
[sudo] password for user:
/dev/sdb1: DOS/MBR boot sector, code offset 0x52+2, OEM-ID "NTFS    ", sectors/cluster 8, Media descriptor 0xf8, sectors/track 63, heads 255, hidden sectors 2048, dos < 4.0 BootSector (0x80), FAT (1Y bit by descriptor); NTFS, sectors/track 63, sectors 1046527, $MFT start cluster 4, $MFTMirror start cluster 65407, bytes/RecordSegment 2^(-1*246), clusters/index block 1, serial number 058b9dd4b057949f4; contains Microsoft Windows XP/VISTA bootloader BOOTMGR

Format partitions

Format a partition with Fat32 filesystem:

user@ubuntu:~$ sudo mkfs.vfat /dev/sdb1 -F 32 -n myDisk
mkfs.fat 3.0.28 (2015-05-16)
mkfs.fat: warning - lowercase labels might not work properly with DOS or Windows
user@ubuntu:~$ lsblk -f /dev/sdb1 
NAME FSTYPE LABEL  UUID                                 MOUNTPOINT
sdb1 vfat   myDisk 56B3-C0AB

Format a partition with NTFS filesystem, using quick formatting:

user@ubuntu:~$ sudo mkfs.ntfs /dev/sdb1 -L myDisk --fast
[sudo] password for user: 
Cluster size has been automatically set to 4096 bytes.
Creating NTFS volume structures.
mkntfs completed successfully. Have a nice day.
user@ubuntu:~$ lsblk -f /dev/sdb1
NAME FSTYPE LABEL  UUID                                 MOUNTPOINT
sdb1 ntfs   myDisk 5B9A993902137688

Fix corrupted partitions/filesystems

Fix a corrupted ext4 partition/filesystem:

user@ubuntu:~$ sudo fsck.ext4 /dev/sdb1
e2fsck 1.42.13 (17-May-2015)
Superblock has an invalid journal (inode 8).
Clear<y>? yes
*** ext3 journal has been deleted - filesystem is now ext2 only ***

Resize inode not valid.  Recreate<y>? yes
Pass 1: Checking inodes, blocks, and sizes
Root inode is not a directory.  Clear<y>? yes
Pass 2: Checking directory structure
...[SNIP]...

Fix a corrupted ntfs partition/filesystem:

user@ubuntu:~$ sudo ntfsfix /dev/sdb1 
Mounting volume... OK
Processing of $MFT and $MFTMirr completed successfully.
Checking the alternate boot sector... OK
NTFS volume version is 3.1.
NTFS partition /dev/sdb1 was processed successfully.

ntfsfix doesn't exist by default in the OS, and we need to install it using the OS package manager:

  • On Debian based distributions (like Ubuntu) : 
    sudo apt-get update && sudo apt-get install ntfs-3g -y

Find duplicate files

Display list of duplicate files and their size in a directory:

user@ubuntu:~$ fdupes -S -r Videos/
23995556 bytes each:
Videos/video.mp4
Videos/video - copy.mp4

Let the user choose which copy to preserve, and delete the rest of duplicates:

user@ubuntu:~$ fdupes -S -r -d Videos/
[1] Videos/video.mp4
[2] Videos/video - copy.mp4

Set 1 of 1, preserve files [1 - 2, all] (23995556 bytes each): 1

   [+] Videos/video.mp4
   [-] Videos/video - copy.mp4

Automatically keep the first copy, and delete the rest of duplicates:

user@ubuntu:~$ fdupes -S -r -d -N Videos/

   [+] Videos/video.mp4
   [-] Videos/video - copy.mp4

List opened files in a directory

List open files in a directory, and which process is using these files:

user@ubuntu:~$ lsof +D Videos/
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
vlc     8869 user   17r   REG    8,1 23995556 402513 Videos/video.mp4

It is very useful when you need to unmount a partition, and your OS tells you that there are processes using files in this partition. We can use this command to find out which are these processes, and stop them.